First off, your husband made the agreement with them so s
he should take responsibility for withdrawing it.
This is just a suggestion (not ideal but it may be better than nothing), would you consider e-mail them and explaining the situation but offer them the use of (i am assuming) the spare room for their guests, purely for sleeping arrangements? Obviously not ideal but it would maintain better relations with your neighbours, who are as you say good people.
If not, then they should be notified sooner rather than later.
good luck.