If by ‘special clothes’ you’re referring to the prayer shawl worn by men, here is the low-down:
Called a Tallit, made of cotton or wool (sometimes silk) and usually decorated with stripes of either black (mostly worn by Orthodox, or traditional Jews) or blue. Along the two shortest edges are knotted threads called fringes. At each corner are long stings called Tsitsit which are tied into knots in a particular way, which is designed to remind us of the 613 laws in the bible.
Surprisingly, there isn’t actually an instruction in the Torah telling Jewish men to wear these shawls. However there is an instruction from G_d (HaShem) that we attach ‘fringes’ to the corners of our garments:
Numbers 15:37-41
"And HaShem said to Moses saying: Speak to the Children of Israel and say to them that they are to make themselves tzitzis on the corners of their garments, throughout their generations. And they are to place upon the tzitzis of each corner a thread of techeiles. And it shall constitute tzitzis for you, that you may see it and remember all the mitzvahs of HaShem and perform them"
Now, at the time (around three and a half thousand years ago, give or take) the Israelites wore robes, so the commandment was simple to apply. When the Jews adopted western dress however, they had to come up with a way to honour the commandment. They decided to wear a symbolic robe, with the fringes attached.
Why are they only worn by men? Because The shawl is worn only at times of prayer, and only for morning prayer (excluding Kol Nidre), and women are excused all time sensitive commandments (mitzvot) because they have the more important role of raising children and tending to the home to do.....