It's guaranteed that this question crops up at least once a month, and it's also guaranteed that a few well-meaning but mis-informed people trot out with the same mis-information. So I'll have yet one more attempt at education.
The legal boundary is shown on the LR title deed plan - typically anotated with a red line showing it. It's called a 'General Boundary' by the LR and they specifically say that you cannot use it to scale from or to work out EXACTLY where a physical boundary should be on the ground. Yes it is true that more recent housing estates often show fence ownership by the letter 'T' that demarcates ownership when the developer carves up the land into little plots separated by fencing he erects. These new plots definitely start life with the fence (physical boundary) aligned to the legal boundary.
The physical boundary is where the fence is on the ground. Ideally both parties will agree that the legal boundary and the physical boundary co-incide. When they don't, disputes arise.
Over time, physical boundaries can (and do) move. Not by much, perhaps, but if the owner of a fence takes it down and puts another back up in a different place, over time the physical boundary will default to a new position - unless the landowner takes steps to make it clear where the legal boundary is - in writing to the adjoining landowner. (I'm not going to propose the wording of such a letter, though I have personally been involved in such a dispute).
So if you've put up a second fence inside the line of the original one (and we must be talking just a few centimetres here?), over time (probably >12 years, but I'm not sure about that), you'd have remarkable trouble getting back to the original boundary back again, if you went to court to try to prove it. If the original fence posts are still there and you wish to maintain your physical land ownership to the original place, you should talk to him next-door explaining this position then follow it up with a letter that states that you have erected a new fence on your land but that you consider that the ownership of your land to be the original posts. Though what you can do with this strip of land seems unclear. None of the above shift in position changes the legal General Boundary between the two plots, as recorded by the LR.
For those needing authenticity to the above principles, see here:
http://www.landregistry.gov.uk/public/faqs/how-do-i-find-out-where-the-boundary-of-my-property-is
http://www.landregistry.gov.uk/public/faqs/how-do-i-find-out-who-owns-andor-has-the-responsibility-for-the-boundary-fencewallhedge-of-a-property