The way to handle this is tricky. Firstly, despite what you have said, you will need to contact your builder. Inform him of a deadline for completion of the work that you require (E.g needs to be completed by [00th Dec] because [Blah]...make sure you can reach a (Preferably written) agreement with him on a reasonable time scale (Reasonable depending on what work you need finishing...the larger it is, the longer is reasonable). Also, make sure he isn't intoxicated when he signs the agreement - or is at least sober enough to comprehend what is being signed.) Then, as he can "Be a handfull", leave a phone message saying that his failure to keep to this time scale will allow you to treat the contract as terminated unlawfully on his part, as the time is now a condition of the contract. (There is case law authority on this but I won't bore you.) If he fails to complete...inform him that you are terminating the contract and find a new builder to finish the work. You will have to pay this new builder again...but now you can sue builder 'A' for damages. You sue for the money already paid to builder A in full owing to his unlawful repudiation of the contract.
If A does complete the work in time, and you consider that the extra money paid is a substantial enough sum worth recovering, then you could attempt to sue A under the principle of having paid under economic duress (This may be shaky, depending on your circumstances.) For this, consult a local solicitor with the full facts, and good luck.