If you use the email provided by your internet provider, then this is all stored in their remote servers. Reliability varies enormously between compnaies and packages. So if you have very important emails / attachments its worth saving these yourself and backing them up on your doohickey thing.
One easy way to do this is to highlight and copy the email, paste it into a word doc then save that word doc on your own computer or backup device. That's one reason why i say you wouldn't want to do that with lots of them - so check what email storage your email has and how long its stored for.