You can only dismiss him without notice in cases of gross misconduct, which doesn't seem justified here.
If one dismisses someone with notice, it must be done fairly. It seems what you did might be seen as unfair within employment legislation. Also the notice period would appear to be 5 weeks unless something longer was agreed between you - plus he is due outstanding holiday pay.
He has every right to apply to an employment tribunal for the outstanding monies, and also to lodge a claim for unfair dismissal.
Whether he does so, is his call - you might be able to dissuade him by contacting him tomorrow to offer that you appreciate he is due some money, and work out what he is owed under contract. He might then not bother with the unfair dismissal element.