Depends really. If is just once in a while, it may be due to a voltage reduction or 'switch changing' on the national grid. (No I'm not a scientist...my dad worked in the electicity industry for 45 years!!!)
If its more of less constant I'd be inclined to have the wiring checked out, especially if you don't know when it was last done or its very old.
if it is constant it sounds like a loose connection at one of your lights. when it does it again listen if you can hear a sparking sound a any of them.