I *can* claim we were poor, at least at the time. Or poorish. Not in a position to afford expensive foreign holidays, nor to holiday outside term-time. Nothing to do with snobbery.
I don't buy the "why them and not us?" counter, by the way. Firstly, this ruling applies to "you" as well as "them". Secondly, presumably the reason that "you" didn't take holidays during terms is perhaps because, well, you can afford not to? A rule like this penalises the poor, who have little choice in the matter, far more than it does the rich, who have the freedom money grants them. Not that I mean to envy, but it's hardly a valid complaint if you can afford it.
Rules are for all, but I don't agree with blanket rules that have no consideration whatsoever for circumstances. Never have done, never will do. Wherever possible, a rule should care about who it's being applied to. And if it does not, then it's a bad rule and should be changed.