He probably can. Between my house and my neighbour's there is a narrow passage which belongs to them, but on which we have a right of passage (an easement). (This passage leads into the gardens - we are two semis not attached to each other). If you have any sort of right of passage, then your neighbour should not have put up that fence. (If you have been getting out of the car onto the front of his house for the past 20 years, you would have acquired such a right; the right could also be registered, but the solicitor acting for you when you bought your house should have pointed that out to you).