This was such an interesting question and I did not know the answer, so I started reserching a bit.
Apparently it was indeed the mother of queen Elizabeth II.
Hitler said:
"If Churchill is the man in Europe I must fear most, then surely she is the woman I have most to fear of in Europe!"
During the Second World War she refused to leave London, and she even travelled to parts in London that were bombed.
For this she was loved by the population, and I guess it was the power she had over them, resulting from her strenght, that made Hilter worry.
I can only assume, that the quote 'he was afraid of the queen mother' comes from a later time, and Hitler did not call her that himself.