I am sorry to hear that. I keep hens, and I did have problems with a dog at one time. ('Course, the owners always say ' My dog? Can't be, he's so placid')
You do have some responsibility for having a proper boundary; an 'open plan' garden, with no walls/fences is asking for trouble. Assuming you have, you have no reason to keep them in a separate pen. It's the owner's responsibility to keep their animal under control.
Taking action: if a dog is caught 'red-handed' I believe farmers still have the right to kill any dog worrying their stock. With a neighbour, I don't suppose you'd want to go that far! But you could perhaps point that out to them. You certainly also have the right to make a claim for compensation against them providing you have some evidence to prove whose dog it was and that the dog did kill the birds. Without evidence, or an admission, I don't think you'll get far as the owner can simply say 'not my dog, your honour'.