Bad idea. The whole point of writing the number on the counterfoil (not the actual ballot paper) is to prevent possible fraud. On the event that someone walks in, claims to be you, and is issued with a vote on that understanding, then their vote would obviously be recorded as your vote. And then you walk in later, claiming to be you -- but of course your number is struck from the list! You've already voted. But you protest, of course, and should be able eventually to confirm your identity, so it is accepted that the previous vote was fraudulent.
But now comes the point. In order to remove the false vote from the box and replace it with your own, the false vote has to be traceable. This is impossible if there is no link between the vote you cast and the "person"/ "roll number" who cast it. This list can be consulted only if there is a suspicion of fraud. During the election it can be handled fairly swiftly; afterwards all counterfoils matching the voter number to the ballot paper number are locked away, and can only be opened with a court order and are anyway destroyed after twelve months.
Your vote remains secret despite this procedure; and you are protected from fraud because of it. Rubbing out the number is self-defeating (unless you were the fraudster, of course).