Zacs, you've really got to start hanging around more upmarket toilet blocks. ;o)
PVC or acrylic, there are plenty of smart panels around now. I've used them as splashbacks in my kitchen.
Preferably straighten up the walls by fixing battens to them. Then simply glue the panels to the battens.
They make matching jointing connectors and end trims as well.
As for cost comparisons... it depends on the complexity of the job. Tiles can be easier to cut around fiddly bits, but panels are quicker to cover a whole wall.
As always, there's no single answer to your question. quotes are the only way to go.