The men can do what the hell they like (and if they can`t do it in Saudi, they come to London to do it) but they`re terrified of that status quo ending if women have any power whatsoever so hence, they`re scared of letting them drive because that might just be the start...
pretty much the same fears were expressed as cars became more common in the west over the last century: that greater mobility would lead to more sexual activity. I dare say you could find some archbishop (and indeed secular leaders) making the same point if you wanted to dig back through the records, though I don't think they were concerned with the gender of the drivers.
Once I worked with a sweet old feller who linked the rising divorce rate to electric washing machines. Once the dolly-tub went out women had too much time to think stuff over.
Pesky hormones. BTW you're all conceived female. Just thought I'd remind you.