Two things.
1) You mention Outlook, but there are two versions of Outlook. The "free" one that comes with Windows called Outlook Express, and the one that comes with Office called just "Outlook".
I believe (but willing to be proved wrong) that they keep their email and other files in different places. I believe what ACtheTroll says about Outlook.pst MAY not be true for Outlook Express.
More here:
http://www.insideoe.com/files/store.htm
2) I use Virgin as my ISP, and one of the options I set in Outlook Express is to "keep my mail on the server". This means that even when I download my mail to my PC, a copy of it is still kept on the server.
It is only deleted from the server when I delete it from my computer. All the rest of the emails are kept on the server.
This is great if I want to do my mail on another PC as I just need to set it up on the other PC and all my mails are there on the server.
This is the window in Outlook I use to set it to keep the files on the server
http://www.dansfishinhole.com/temp/eudora/OE_S ettings_2.jpg
(I know this may not be useful for you right now, but may be useful in the future)