You need to re-itterate to that jerk of an ex that he can pick up his son at a nominated and pre-arranged time and that there will be no change in this arrangement unless it is done say 2 days previously. You should then tell him that it will be his own fault if he tries to change things at the last minute and gets annoyed because you have said no. This you must drive into his thick, self centred, childish, immature head.
As for the courts....if you keep calm, serene and to the point and focus on 'your son and how he is affected' then I think you will be ok. If you lose your rag and start bringing in irrelevant personal issues, then they may see it as you just trying to get at him through his girlfriend.
Good luck beverleymot...He sounds like a to$$er and it is impossible to have a reasonable discussion with one of those because they only ever think of themselves and how life affects them - so you need to keep things short and to the point. (I used to be married to one too!!! I tell him how things are going to be now, rather than enter into discussion , so we don't argue.)