Society; a large part of managing a business is to organise workloads according to staff levels and to plan ahead for any holidays, days off, etc, when temps may be required or 'juggling' of duties to accomplish what is required to be done.
Your boss should really be able to do this properly without resorting to calling staff in when they are enjoying their rest day(s). At the very least he/she should enquire the previous day whether staff are prepared to work it. This puts the boss as the one who should be feeling guilty, not you!