There is no mystery as to why men have nipples. Until about the 14th week of human gestation, when the various hormones kick into action, the foetus is not gender-related. However, by that time, the tissue that will become nipples has already started forming. As a result, men as well as women have them.
And, despite our having been clothed "for some time", that time-span is but the blink of an eye in terms of evolution, so it's no surprise that many traces of our ancient heritage - such as pubic hair - still remain with us.