Quite right. Its the skin reacting not the hair. But this is because we used to be hairy beasts.
When it's hot and you need to cool down, little muscles at the base of each hair relax. Your hair becomes relaxed. Your sweat glands pump out body heat in sweat. Your blood vessels get big to take more heat to the skin to get rid of it. When it's cold, the arrector muscle pulls the hair up. The duct to the sweat glands gets small to conserve heat. Our blood vessels also get small to save heat.
Hair standing up doesn't make very good insulation - we don't have enough fur for that. Humans don't have very much hair on their bodies anymore. Millions of years ago, humans probably did. And that hair standing on end helped keep people warmer. Those little muscles we have on the end of each hair still work. They still make goose bumps. You will notice similarly that birds in cold weather �ruffle� up their feathers to keep warm.