// So it comes down to is a woman transitioning to be a male is not seen as a threat to males, but the other way round it is a threat. Logic would say this is wrong, but feeling and instinct don't. //
It's not illogical at all. When over 90% of violent offences and sexual offences are carried out by males, and the majority of victims are females and children, it's not illogical that the male is viewed as a potential threat.
Imagine, Zacs, if you were alone in the Gents and a female dressed as a male came in. You might regard this as surprising/odd/amusing/insensitive/insert your own emotion. Now imagine you are a female alone in the Ladies and male dressed as a female comes in. This is quite, quite different. I would be at the very least wary, but also possibly intimidated, scared, vulnerable. I hate to generalise but I don't think I'd be overstepping the mark to suggest that most females would share similar feelings.
It's not that we walk about in a permanent, conscious state of fear, but it's second nature to us to be wary. And it is why we need 'safe spaces' though I detest the term. It is also one of the reasons why I dispute that any male can know what being a female 'feels like'.