The imagined potential is only rarely matched by the cold reality, especially when the circumstances have changed.
What may clearly have worked when the decision to follow the current path was taken may not be so easy now. Much like how hard it can be when you suddenly realise you missed your freeway exit. You can't go back and the next exit might be on the other side of a river.
It is never too late to change but there is always the risk that loosing what you have will be more painful than gaining what you imagine life could have been. More so if the new path doesn't work out like you imagine.
It really depends on how prepared you are to take risks.