As the two babies are the same generation, they will be second cousins.
Once removed refers to when there is a generation difference (eg the other baby to your boyfriend will be first cousins once removed)
If you look at this picture:
http://upload.wikimedia.org/wikipedia/commons/ thumb/5/5d/Cousin_tree.png/400px-Cousin_tree.p ng
And assume your child is the "ME" in orange, then you and your bf are the mother and father above. If you go straight across to the right from the "ME" you will see there is a box saying "2nd cousin", this is the cousins baby. Notice also that its parents (your bf's cousin) is your baby's first cousin once removed.
Bit confusing but if you trace through this diagram it will tell you what relation everyone is to your child!