Music tends to take up around 1mb of memory per minute of music, so 1gb will hold around 1000 minutes or about 20 albums (very roughly speaking).
Flash memory (or built-in memory) is more reliable because there are no moving parts, but only goes up to around 8gb I think (such as the ipod nano). Hard drives are bigger but are more susceptible to bumps and knocks because they do have moving parts (like the video ipod).
Ipods or other MP3 players - up to you really, you do pay a bit more for ipods but they are very easy to use and are well made.