Not necessarily. Your neighbour could be a woman and she has the daughters, but their father has now gone on to pair up with someone else, and he has a son with someone else. This boy is their brother, but not your neighbours child!
Surely the question is ambiguous as each duaghter could have more than one brother.
If the question had stated that each daughter had ONE brother then then answer is 5.
However, the question only says that each daughter had A brother , so they could actually have more than one!