Does he have a penis? Can he rape women? In that case, women need a safe space from him.
In places like India, the reason women still go to the fields to defecate instead of using the toilets set up by the government - because without a separate men's and women's section, too many women were getting raped when they went to use the bathroom. There is a reason women need safe spaces.
Trans individuals often also need safe spaces. It's too bad that there are people who are bullies who make women, trans people, etc., feel unsafe. Giving trans and other non-binary people a safe space does not mean that you take those spaces away from women, however.
Scientifically, BTW, human beings are mammals. According to the Encyclopedia Britannica, mammals are defined as: Mammal, a vertebrate animal whose young are nourished with milk from special mammary ... themselves to the female's body in the area of her mammary glands.