re-read your house deeds - normally there is a time period attached to any stipulations in the deeds (10 years is normal so it's quite likely the stipulation as expired). Failing that then i'm pretty sure only the builders who created the deed stipulations in the first place, are the only ones who can object to your intended 'violation' of the deeds, if no planning permission is required. . Ring a solicitor who will confirm this.
If they still exist you can contact them to obtain permission if you require to do this completely legally - normally these things are done by common consent (i.e. they don't annoy a neighbour who then contacts the builder) as the builders don't give a monkey's once they've sold off all the houses.