Anywhere apart from the Ritz ! Never understood the attraction of this place for tea. Brown's is good though. It used to be a place where you just turned up unannounced; still is , if you don't mind not sitting in their tea room; but they prefer you to book. The link above offers 25 per cent off there now. Claridge's is ignored by Londoners; it is not a place which you'd instantly think of for lunch or tea; but is certain to be good for tea, and you probably get their tame harpist playing, as at lunch!
With the Olympics on, you'll find that lots of good hotels and restaurants are quiet. There's nobody around in central London.