I think it's a good idea to keep your P60 (annual summary of tax and insurance paid) back to the year you first started working. (which for most people will only amount to about 40 pieces of paper)
Reason: I don't trust these darned government computers and their records. Every national database that's every been introduced has has gone wrong or had errors - Child Support, Child Tax Credit, the Passport system, DVLA etc. etc.
One day everybody is going to want to draw a State Pension based on how many years they've been paying N.I. contributions. I'd rather trust my filing system than a government computer if it comes to an argument.