The earth's surface has changed a huge amount over the billions of years it has been here.
Due to movements of land surfaces and volcanoes and so on the earths surface has been bent and buckled over millions of years.
This has left all sorts of different type rock in the ground, some of it very hard, some of it very soft.
Millions of years ago the earths surface was higher than it is today.
This has gradually been worn away due to the effect of wind and rain, AND the effect of millions of tons of snow during the ice age.
Of course the wind and rain and snow will wear away the SOFT rock quicker than the HARD rock, so the HARD rock still sticks out of the ground where the SOFT rock has gone.
Here, for example, is a valley created by the movement of snow over many years that wore away the soft rock leaving only the hard rock.
http://en.wikipedia.org/wiki/File:Glacier_park 1.jpg
You can see examples in the UK in the Lake District and Scotland.
So basically most large lumps of rock sticking out the ground are a HARD rock where the soft rock around it has been worn away over the years.