The best advice I can give you about the MTU setting is to take the same question to the xbox.com support forums where you'll get more specialist knowledge about it.
My 360 is hard wired straight into the router with an Ethernet cable for simplicity, if you have the option to do this, then it;s much easier and a more reliable connection. If you're going wireless on the 360 I assume you have the Microsoft wireless adaptor as that's the only thing that will work - gotta love Uncle Bill :)
You shouldn't need to install any software for the router as they will run without it, the USB wireless dongle does need software though which you've done. Is it your router that wants a password to connect to it, or is it your ISP wanting a password? If your router is wanting a password to connect to it properly, then you should find a wireless key somewhere in your documentation which is probably what you're missing