Just to check. Do you have an employment contract? If so, have you checked it to see if it says anything about it being compulsory to attend staff meetings?
If it does, the yes, you do have to go, but it should also say something about additional pay or time off in lieu for attending during non working hours/days.
If it isnt in your contract, then technically, they cant force you to attend, but as you have found, they caould make life harder for you.
Instead of just saying no, if you want to try an appear preofessional and positive, you could discuss with your manager the reasons for your reluctance, and include things like the cost of the journey in etc, and ask if there is a way of working it out, i.e could they pay the cost of your journey, give you lieu time, or ask if you can arrange a briefing before the start of one of your shifts so you dont miss out.
Whilst one doesnt want to be taken adantage of, it is always best to appear professional and amenable to finding solutions, that way, if they dont work with you to find a way, and cause problems for you later for not coming, you have the recourse of being able to show you tried to work something out.