Not saying he's doing it, but there are some men who pretend to be gay so that women will be their friends.
And the fact that he talks effeminately doesn't mean anything. You don't say how old he is, but I've known a few older men who behave like this (because it's often to do with how they were brought up) and whilst it might be creepy, they don't mean any harm - they just don't understand how uncomfortable they make people feel.
Whether he's straight, gay, bisexual or otherwise, if you really don't like the way he touches you, then just say so quietly and politely. Something like, "Please don't take this personally, but I'm really not comfortable with people putting their hand on my shoulders like that. I'd prefer if you didn't do it." Try to get other people to respond in the same way.
Then if he does it again, you have good reason to speak to your manager about him.