If you're at home when he's doing this, then surely at least one (or maybe both) of your parking spaces are occupied, so how is he using your drive ??
If you're not at home when he's using your drive, what's the problem, as you're not there to see it happen, so it can't wind you up that much ...... can it ??
If you haven't got a car and he's using your drive to reach his parking spot, what's the problem ??
Like previously asked, is it just winding you up in general, or are we missing something else ??