Everything is expensive, but that's just they way it is...they have to import a lot (obviously).
I'd recommend the blue lagoon, whale watching, going to Geysir (to see the geysers), skidooing on a glacier (I think we did it on the Vatnesjokull (sp?!!!) glacier), horse riding along the flats by coast (try the weird gait that Icelandic horses have!), boat tours (we did one where you can try fishing, dolphin and seal spot and land on a small inhabited island for lunch ). And just going to the local swimming pool is fun, they're all so clean, and there's jacuzzi's everywhere. And, just all around the island are geographically interesting things, caves, massive faults in the ground, steam vents, and some natural small pools/lakes that are warm enough to swim in.
There's so many outdoor activities available. Have fun!