Do be careful about tasting berries you don't know -- some could make you seriously ill if you ate enough to taste.
However, I agree with Apricot -- sounds like a black mulberry which is edible. The berries look like slightly prickly blackberries (which are also called mulberries in some areas, and are probably the ones in the nursery rhyme). The leaves are heart-shaped, with regular teeth along the edges. Usually a fairly small, domed tree.
Have a look at
http://waynesword.palomar.edu/fruitid6.htmfruitid6.htm and scroll down to mulberry.
If it is that, you're very lucky -- I love them -- I think you can make jam etc with them. They do stain though.
I think it's not the leaves of this one but those of the white mulberry which are the food of the silkworm -- the wrong kind was planted by mistake in enormous numbers in England in the 17th or 18th century.