Sounds more like water hammer, Starry. When a tap is open, the pressure is released. When you close it, that pressure makes the whole system shudder.
Either the mains pressure could do with reducing, or you may have to have an "arrestor" fitted. Simple job, it would take a plumber 10 minutes to fit one somewhere.
Amazon.co.uk User Recommendation