Im not sure though I recall watching a chef the other day either Jamie Oliver or that guy from River Cottage Hugh Fearnley-Whittingstall's talking to a major grower here in the uk, so it does grow here.
In addition to Hawk's answer, the saffron crocus flowers in the autumn so whatever you have in your garden at the moment won't be it, he said from Saffron Walden.
you can buy saffron crocusses and harvest your own. I tried it once but it was a poor summer and the bulbs didn't like my heavy clay soil.
http://www.suttons.co...cJ_y6Juc1tG5PH5MCtkli
You need a whole garden of crocuses, though, to produce the weeniest bit of saffron once it's dried - I can't remember the proportion but it's something like a whole field of crocuses to produce a small bag of saffron. That's one reason why it's so expensive.
PS. If you go to Spain on your holidays always pick up some saffron from any supermarket, it's about an eighth of the price in the UK even with the duff exchange rate, I always bring some back for foodie friends, and me :-)