Woofy is right. I own all the fencing around my garden. Not sure whether that's a good or bad thing.
fazy if you put up the brackets and the weight of the baskets damages the fence or causes in to lean, then if it were my fence I would expect you to pay to put it right,whether or not you asked permission. And I would feel the same if I were to damage your fence.