Get some Sensodyne toothpaste - preferably the blue gel if you can find it.
Put a blob of the toothpaste around the gumline of the tooth, and try to leave it there. It may hurt at first, but will start sealing up the points at which the nerve is getting exposed.
Leave for a short while in case it's discomfort from something that's healing. Senstive toothpaste is useful. But if it's still around in a few days, see the dentist. It may be a chip or small cavity or something else minor, best caught early.
I have had my fair share of toothache in the past and I find Ibuprofen is really effective. Its an anti-inflammatory as well as a pain killer. Ultimately though, it sounds like a visit to the dentist is on the cards. Sorry.