The only obligation your neighbour has it to make sure any fence or wall is safe.
Is the damaged panel on your land? Could it injure you?
The owner is under no obligation to repair or replace the fence. He could take it down entirely if he wished to.
You can put a whole fence up as long as it within your boundary.