I'm guessing that Windows Live Mail is using the RSS 'Common Feeds List' which is fed by Internet Explorer.
Open Internet Explorer, click on Favorites and then on Feeds. Right-click on any feed you see there but don't want and select 'Delete'.
Then switch to a
decent browser (which is basically anything except Internet Explorer but Firefox would be top of my list:
http://www.mozilla.org/en-US/firefox/all/ ). If you're not using IE it can't collect feeds for Windows Live Mail!