In the long run, as and when it comes up, I would have to go to one toilet or the other. In one, I would feel uncomfortable and (possibly) a few others also might. In the other, I wouldn't feel nearly as uncomfortable, but possibly a few others might. In both cases, all I'm interested in is getting it over and done with. So far as I can tell, this is true with pretty much everybody. And the ones for whom it's not true are probably already abusing it. It's not like toilet use is policed currently; so far as I can tell, it's a self-enforced rule borne of etiquette.
Again, the basic point I'm trying to make is that "no transwomen in the ladies'" is based not on any actual dangers, or abuses, but just on the perception of a risk that doesn't actually exist. Or, as far as it does, it's not worth penalising the law-abiding majority for.
As it happens, this is an issue that could affect me personally. In a sense, since this is hardly the first time I've thought about it, it already has done. As and when it comes up, I'll go to the loo I feel most comfortable in. Thankfully, in this country most people are too polite to complain in person anyway.