I agree with Lan ~ your eyes aren't getting used to them because of infrequent use. I am sure if you wore them regularly this will stop..however as Lan also says, if you find it persists even after longer term use a trip to the Optician or maybe your doctor may be for the best.
My daughter wears contact lenses. If she changes them to her specs she has similar symptoms, even though the prescription is the same! she also gets this when she changes her contacts to a new batch.