easyJet (who fly this route) don't require that you have a passport but they do require that you produce photo ID. Annoyingly, their website fails to specify exactly what type of ID they'll accept.
You should be OK with a photo driving licence but, if you're planning to use something else, it would be wise to contact the airline first to check if it's acceptable: 0871 244 2366 (0800 - 2200 daily).
If you're travelling by ferry, this is from the NorfolkLine website:
"16.1 Norfolkline reserves the right to refuse to carry any Passenger that does not have the necessary travel, identification and visa documents.
16.2 Irish Sea only. All Passengers aged 18 or over must produce photographic identification in the form of a valid passport or valid photographic EU national identity card or valid photographic driving license or valid police warrant card/badge or Citizen Card or a valid Government-issued identity card at Norfolkline check-in."
Chris