I think an age gap of 5 years might be a big difference to a 21 year old but not to a 30 year old. By the time we get to 30 maybe we already have it in our heads that age differences are a problem.
Another thing is that the "baby" of the relationship may be seen as inexperienced and prone to being taken advantage of. This way, it may be that if you're seeing an older partner it is OK but not a younger one.
Then again, maybe we (as humans) just look for differences between "us" and "them" in general - in this case, "them" is anyone of a different age group. Isn't that why prejudice & bigotry exists - because so many of us see problems and differences rather than enjoying the variety of ages, races, religions, cultures etc?
That was a bit deep for me. I'm off for a lie down.