It's all about whether or not the radiation is ionising. What does this mean?
Imagine you have an atom. It has a nuclear, with electrons spinning round it (a very crude model, but it'll suffice). An ion is one that has lost or gained an electron (or a few).
Electromagnetic radiation, which means light, microwaves, radiowaves, X-rays, gamma rays, etc., is made up of little particles called photons. The electrons in atoms are being held to the atom with some force. If the photons of the radiation hit the atom with more force than the electrons are being held onto by, then the photons can knock the electrons out of the atom, ionising them. This is called ionising radiation.
Note that it's the energy that each individual photon has that gives it its ability to ionise atoms, and not the number of photons (called intensity).
Now, if the atoms of your skin for example are ionised, then, if enough of them are in a small region, it could cause damage to the cell that these atoms make up. If a damage cell is in your body, your body will try and remove it. But sometimes (and I'm not sure it's 100% understood, but I'm a physicist and not a biologist), enough bad cells group together to become cancerous cells.
Clearly, this can damage your health.