What is wrong with it is that it holds you back whilst others undertake and op in, in front of them and you. Regardless of how much overall difference it makes to the journey, that is frustrating it's like you are being deliberately messed with for the amusement of the person deliberately choosing to hold up everything behind them for the benefit of those in the other lane. Inconsiderate of others and their peace of mind.
A similar thing can happen at multiple lanes at the traffic light junction. You get someone who need a bus length space between them and the vehicle (or white line) in front, so you approach and chose the shortest lane, except these jokers have got you to join the long lane, and your chance of getting through next time is reduced because, their lane happened to consist of fewer vehicles after all, and they get the amusement of getting through knowing you may well end up waiting a second time.
People should be thoughtful and more considerate. Other drivers can be a stress.