I was in the same situation not so long back, the big difference being that my ex-husband finally contacted me to see our now 7 year old son on his birthday. The CSA are pointless, he wanted to me contact them several years ago (he's gone through periods of seeing him, then not) and when I did he appealed against the decision as he thought it was too much (he asked me to go through them!) now I receive practically nothing as they won't do an attachment of earnings on him ('too much work' according to the CSA) . In the last three months I've received �35. He's on an extremely good wage, has his own house, no other children, extra income from his partner etc. So the CSA is a definite no in my opinion!
As for his father I agree with DottyH, until your son starts asking questions about him and wondering why he's not around, I'd leave it. The blokes obviously not grown up yet by the sounds of it.
You say you're still in touch with your ex's mother, does she see your son?