A few times, during laddish conversations, I have heard male friends or colleagues say 'I can't tell if she's attractive; I'm married.'
Back when I was single, I used to think 'What a load of rubbish!' Now, I am married I still think it's a load of rubbish. It is entirely clear to me whom I fancy - my ability to judge the attractiveness of other women hasn't decreased one iota. And why would it? Moreover, if it did, it would mean that everytime you told your partner that they were pretty or sexy, you'd be lying, because these concepts rely on comparisons. My wife is equally capable of finding other people attractive, incidentally. We're both grown up enough to accept this is normal. Of course, if I got caught cheating, my chances of fathering would be forever removed - and fair enough, really.
I have never had to struggle to control the urge to sleep with someone in the way that you describe, so I can't really imagine what it must be like to find it that hard to resist (and well done for doing so incidentally).
The sexual urge does decrease in men with age, as you'll know, your sexual peak is at about 18 (how unfair that seems now I'm in my thirites, but then I suppose there is more to life than sitting around interfeering with yourself...), so it will probably slowly decrease.
I think you make the point when you say you'll stay because you don't find many girls like that.