I've always had downy arms...and it's never really bothered me. As a redhead, my arm hair was never dark,and bleached somewhat in the sun.
It's only recently with society's obsession with the removal of hair in odd places,that I've wondered if maybe I should have been removing it all these years...