Indeed. The main problem is that there is no comprehensive definition of what "trans" is or means. Leaving aside for the moment the objective fact that human beings can't change sex, how is a woman supposed to tell if an obviously male person in their single sex space is a threat or not? They can't, which is why we have this long-recognised social contract. Women are conditioned to see men as a threat. It's a sense that has developed over thousands of years.
I like to think of myself as a good man who would never hurt a woman but how would they know that? They don't, that's why all men should stay out of women's spaces.
Women are not a shield for men with fetishes and mental health issues. Third spaces are the obvious solution in toilets, sports etc but they are rejected as it doesn't make these men feel validated. It's the violation of women's boundaries that gets them off. If they were any sort of woman they would understand women's concerns and fears but they aren't so they don't. This should be about men being more accepting of gender non-conforming men. It should be a problem for women to sort out.
This post from author Jane Harris shows just how difficult it is to define trans. All these groups fall under the umbrella but have nothing in common.