All 6 of you entered into the agreement when you bought. Legally, it's part of the deal. No one has the option to opt out of it. Any new owner will be made aware that they are also entering into this agreement. Next time the drive is re-surfaced, they should contribute.
To exclude that particular house from the arrangement, it would need the agreement of all 6 to vary the conditions of the existing contract. Easily done, but costs of (potentially) 6 solicitors might make it costly.