It's a fair point, but never forget that this is not really a 'talent show', it is first and foremost a TV show, and that is an important difference.
If it was simply an audition, 75% of the contestants wouldn;t get to sing because they don't look right - no fatties, no geeks, no oddballs would ever get heard, they would walk in, be looked at, and walk out - because that is how auditions work.
The UK adores the X-Factor winner - how couldn;t they, that person is in their living room at prime-time for about three montsh - it is afterwards when reality sets in.
That is why the McManus's and Waller's and Brooksteins of this world don;t hack it, and the Leona's and Girls Aloud's do.
The public don't want fat and / or geeky with a sad life, they want gorgeous with killer songs - it's as simple as that.