you don't say which printer - or which model of router.
you can fit some printers with network cards ... and connect them directly to the network
(some come with them built in)
you have a number of other options
connect the printer to one machine ... and you can share it with the other.
you can get a do-dah called a print server
http://www.google.co.uk/products?sourceid=navc lient&ie=UTF-8&rls=GGLD,GGLD:2004-30,GGLD:en&q =print%20servers&oe=UTF-8&um=1&sa=N&tab=wf
and some routers have a usb port which will support a printer (ie bt home hub)
the shared option is simplest .... but it means you can't switch the pc off while the other machine might need to print