Not so. I've answered quite a few questions which have subsequently been removed but it's never affected my total of removed answers.
If you're trying to find the question by looking through the section where it was posted, you should remember that the editor sometimes moves questions to a more appropriate category. (e.g. now that we've got Chatterbank back, the 'chat'-type questions which used to be posted in B&S are now, thankfully, being moved to CB).
A much better way to see if the question still exists, irrespective of whether it's been moved or not, is to go to 'My Profile' > ''Questions I've Answered'.
Chris