Wear a fascinator!! I only learnt that word when my son got married a couple of years ago- it's one of those feathery things you clip into your hair- they look fab + don't give you flat "hat hair" x
I went to a wedding reception and some of the ladies kept their hats on. I cant remember if I left mine on or off but like you say it's an expensive item if you are not going to get VFM for it so I would wear it for as long a poss.