This may change in the not too distant future.
There has recently been aprivate members bill in parliament which would classify gardens as greenfield rather than brownfield which would make it much more difficult to get planning permission.
http://www.gardengrabbers.co.uk/news/Garden%20 Grabbers6716.ink
There was also some discussion about a year ago to make the rise in value of a plot due to planning permission taxable, the money being used to invest in infrastructure where populations are growing.
I don't think anything is imminent but it's definately on the political agenda