There appears to be nothing wrong with the phone itself so I would rule out a claim under Sale of Goods Act 1979. But clearly what you thought you were agreeing to was a phone that was capable of using a service provided by Orange. Therefore you have a legal remedy under common law since Orange are in breach of contract.
You should exercise your right to rescind the contract by refusing to continue to pay. But you also have a right to damages for your losses incurred by Orange failing to meet their contractual obligation to provide you with a service.
In addition, or alternatively, (supposing Orange refute the breach), the contract is voidable on the grounds that there was a lack of consensus when the contract was entered into - i.e. you thought you were getting a phone that was capable of providing you with 'X', and Orange intended to only give you a phone that may give you certain services but that was not material from their point of view. A bizarre defence should they attempt it.
Have a check of your terms and conditions but don't worry too much if Orange have put in an exemption clause - because it is likely to be struck down by the provisions in the Unfair Contract Terms regulations.
So, write to Orange (and record it), tell tem that they are in breach of contract and, as a result of their breach, you are exercising your rights in law to rescind the contract. Request that they return all costs incurred as a result of that breach - including the cost of your stationary, postage, phone calls etc - anything directly connected to their failure to provide the service that was promised or implied. See what they do next. They may continue to harass you for the 'debt' and make threats. You can either let them, and see if they take you to court for non-payment, and you then defend the action and counterclaim...and most likely win. Or, you pursue them through the small claims court, present your case.