If you were allergic to tomatoes you would have had the symptoms all year round, not just during the hay fever season.
However there might be something in tomatoes which could worsen the effects of hay fever. For example, when I was teaching, the dust from blackboard chalk didn't bother me for most of the year but as soon as the hay fever season started I had to stop writing on the blackboard, because the chalk dust made my hay fever symptoms (which might otherwise have gone almost unnoticed) extremely severe. Spicy foods (especially chilli) had a similar effect. Perhaps tomatoes acted as a catalyst to make your barely noticeable hay fever symptoms far worse?
Equally, though, it might just be coincidence that your hay fever disappeared around the time when you stopped eating tomatoes. Most people find that their hay fever goes away after a while. (That might be because they've simply 'grown out of it' or because, for example, a local farmer has changed the crops which he grows, so that there's different pollen in the air).
Chris