I would say invite both couples to begin with and then again whether the first couple are on holiday or not.
I used to be in a Dinner Club. There were 7 of us and this would involve cooking dinner for 7 once every 7 weeks if you will. The other 6 weeks you would go to the others house. Sometimes one of the 7 couldnt make it but it didnt really make much difference.
It was great fun and something to really look forward to.
Enjoy and dont make apologies, I am sure you are all adult enough to know it wouldnt be friend stealing so to speak and if your first friends think that then they shouldnt be playground behaving.
Katie. x