Well, because of the "tiny minority". They are law-abiding citizens with a right to travel, after all, and if it creates some sort of horrific mess whereby they have to identify legally as a gender other than that they identify themselves as, then it's something of a mess.
That said, again I'm suggesting that rather than dismiss the idea out of hand we should use Australia as an example and see how it works in practice before rushing to implement/ dismiss it. May turn out to have unfortunate repercussions in the form of security implications -- I doubt that is true but if it were then we should find out before adopting a change on behalf of what is admittedly a small minority.
It's frustrating, though, to see an idea designed to cater to a group of people that, however small, still exists and deserves legal recognition, being dismissed out of hand as "lunatic" -- or "pathetic", which is the word my Dad used last night.