When you were claiming, after each tax year end you would have been sent a form to do an annual declaration. When you had done that they should have issued you with a final award notice for the tax year concerned. If there was an overpayment, it should have been shown on that award & - if your claim was continuous - the overpayment should have been deducted from ongoing payments by reducing them (usually by 25%).
That is what you say happened on two occasions. It is just possible - but very unlikely for the 2006 overpayment - that they never recovered the full overpayment before your claim ended & that what they are asking for now is the rest of it.
You need to do a Data Protection Act request for a copy of all the paper & computer records they have for you, & a CD of all calls. Make sure you ask specifically for a copy of all the award notices they sent you & that you specify that what you need must cover the 2005-06, 2006-07, 2009-10 & 2010-11 years as well as all others. You may find information on where to write to on HMRC website. If not, contact your local CAB who will be able to tell you the address to write to.
I would not be at all surprised if you find they no longer have any records for 2006 (I believe they only keep them for 5 or 6 years after the year end). If that is the case it is very likely to be impossible for them to provide any proper evidence that you do owe them any money for that year so you should argue very strongly that you have no knowledge of any debt for that year.
But do the Data Protection request first (it takes up to 40 working days to get the reply). Their various departments do not communicate well so, as well as submitting the request, reply to the letter you have received (which is probably from their debt collecting branch) telling them you have no knowledge of owing them anything & are waiting for the answer to your request before doing anything more.