I'm pretty much in favour of people being allowed to engage in any dangerous sport they like, but would make an exception for boxing.
The number of deaths may be relatively small, but it's not just deaths which have to be considered - the risk of long-term brain damage is considerable.
"80 per cent of professional boxers have serious brain scarring on MRI scan"
"Brain tissue, once damaged, remains damaged. The symptoms of such brain damage - commonly known as being 'punch drunk' - include slurred speech, slow reactions and even occasional blackouts (‘chronic traumatic encephalopathy’). These symptoms may take years to appear – on average about 16 years after taking up boxing, but sometimes as later as 40 years, long after the boxer has retired from the sport."
Admittedly the situation is less clear for amateur boxers.