Here is a sitting on the fence answer.
It should be scrapped at General elections, but retained at by elections.
The problem of long list of daft candidates only occurs at by elections. This is chiefy because they are used by people to buy themselves onto the telly. Most of those are not interested in the political process, they just want their 15 minutes of fame. OK, they can have it, but it will cost them £500.
The wannabes are not really a problem when all constituencies are being contested at the same time. The deposit could be scrapped or negligable then.