If you buy goods that are simply the wrong size (unless they were 'wrongly described' through incorrect labelling) you have absolutely no right whatsoever to either a refund or a replacement. A shopkeeper can simply tell you to 'get stuffed' and order you out of his shop. (It's up to you to make sure you buy the right size in the first place).
If a shop allows you to exchange goods that are neither defective nor wrongly described, they're doing you a favour and they have every right to impose their own conditions. For example the store could have said "There's £2.50 extra to pay, plus a £10 exchange fee".
One of the many reasons that I refuse to shop in Mark and Spencer is because they stupidly allow customers to exchange unwanted goods. In my view any customer who tries to exchange anything which is neither faulty nor wrongly described should very quickly be shown the door (preferably with a firm boot up his @rse) and told to read up on contract law.
Chris