You wouldn't catch me saying "haitch" either. But one of the things I tend to feel it's worth moving away from is using language as a means to judge people. Better to embrace the variety than to enforce some kind of gold standard that many people, for whatever reason, don't meet.
It's arbitrary anyway: as one of my links outlined, there is at least a plausible argument to be made that "haitch" was the original form. But we gradually dropped h's from the beginnings of words, and then only later started to restore them (eg to "hat", "hotel", "herb" etc); and, in that sense, haven't yet got around to restoring it at the font of "aitch".
There are times when it's helpful to have a universal standard that all people can refer to, for sure, especially in written language in formal documents that benefit from being as widely accessible as possible. But as long as it's clear what's meant, spoken language oughtn't be so tightly guarded. Everybody knows that "haitch" and "aitch" are the same letter (or, say, "skedyool" and "shedyool" both being the same word schedule, though I much prefer the second myself).