Halibut has always been thought of as one of the most expensive fish available in the UK (and it's certainly a nice fish to eat) but, if I was going an expensive fish which has long been available here I'd prefer Dover sole.
However fish which have only become widely available in supermarkets (etc) over the last few decades often have more flavour. Consider swordfish, if you can get it (look for white flesh, rather than grey), or red snapper.
Good fish doesn't have to be expensive though. Really fresh sardines, simply grilled and served with some nice bread, can be just as tasty.