Baby pidgeons grow very quickerly and if they leave the nest they will stay on the building where the nest is.You proberly have seen plenty but don,t know it.
I read somewhere that baby pigeons do not leave the 'coop' until they are fully grown - they have too many predators and could not survive if they left earlier - which also explains why we have so many of them :-)