Here's the way I always look at it Beds. The "magic" number is three years. Yes, the PP is for an initial five years, but that is for rather different reasons.
Within three years, I would have submitted a Building Regs application. That's the one that matters. Not only submitted, but, most importantly, notified them of a "Commencement".
As soon as they have that notification, then the PP clock effectively stops running.
To clarify Atheist's point... that is a common misconception. Although it makes sense to actually "get on with it" after notification of commencement, the actual nature of the work is up to you. Perhaps you may only clear the ground and do a bit of preparation, but you've still started, and paid your fee. That keeps Building Regs happy ;o)