I've been using a Vista O/S and have not come across many problems. This was on a new pc. Would'nt upgrade an old pc to Vista, because I think thats when you start to get problems.
If you 'Google' your motherboard you get quite a bit of info on its max configuration.
Generally on your setup think you need to upgrade your GPU to the 8800 series with the top card being an 'Ultra', although you will need deep pockets for that one!
Your Ram's ok but you can go to 4Gb. Worthwhile upgrade is to 1.5Gb Corsair Ram