Like I said, Naomi, it's about respecting the identity and the right to be transgender in public, or not. Or did you overlook that?
If you tell someone they have to go the gents', then it follows that you see them as a gent. So either we rebrand the toilets by sexual organs, or we match toilet use to gender identity, or we reform the way in which public toilets are organised (and, for that matter, as shoota suggested, public changing room facilities).
It is a big issue to feel accepted by society for some people, and shouldn't be dismissed out of hand as pandering, or for any other reason.