Your question employs a phrase within which is the answer... "Why do I feel..." relates to 'wind chill'... Temperature is absolute, whether measured in still air or in air that is moving... it's just that the moving air on damp skin will feel colder than in still air. The evaporation of moisture on ones skin makes the person "feel" colder than the actual temperature, therefore a 'damp day' will produce a colder 'feeling'...
By the way, to feel the effects of wind chill doesn't neccessarily require a windy day... just walking or moving through the damp air will produce a smaller result of the phenomena...