Most keys on the keyboards have at least two uses. The further back you go, the more operating systems you use, the more uses you will dig up.
The fact is that, once you start using a keyboard for EVERYTHING you quickly run out of keys and so have to start doubling them up.
Overwriting is useful for authors, for example to obliterate an entire line and replace it with similar text, for data entry operators etc.
A lot of the time you would say "why not just select it, delete it and type instead" and the answer is speed. Someone who's used the keyboard all their life and grew up without a mouse will find any mouse operation incredibly slow compared to the keyboard shortcut equivalent.
I can press Home, Shift-End and type (to do the same job as selecting all the text and typing over it) at least twice as fast as I could do it with a mouse. If I happen to have the cursor where I want it already, I can press insert, type, press insert just as fast.
People who use computers these days (god that makes me sound old) use the mouse more than the keyboard and that slows them down. Admittedly, you have to learn what every key does and how to do things like run Windows when no mouse is connected (which is getting harder and harder as newer versions of Windows and applications remove keyboard shortcuts) but it can save hours of time in the average day.
Hence, those weird keys which you don't know what they do will likely stay on our keyboards for a long time and you'll be hard pushed to find a keyboard without them (some laptops omit certain keys as do certain mini-keyboards) but what you're doing is removing a one-button way to perform some action that may take minutes any other way.
If you want a mystery, try to find out what you'd use the Pause/Break key or the Scroll Lock for in this day and age (on Windows, that is).