I don't think anyone worries over nothing; you either have grounds to worry or you don't. Only you will ever know that so the first thing is ask yourself a couple of things to start off with.
Does his flirting bother you? You say you have 'an understanding relationship' and I wouldn't really know what that means. But if it means that you would both flirt and be content with each other, then fine. But if that's the case this is proving too much for you.
Do you consider flirting to be acceptable? Consider that 'things happen' when either party get into flirting. With all the best intent in the world ... sometimes, things happen.
Ask him about her, in a none-threatening way, and see what he comes up with. If neccessary look for confidence in him that nothing will lead from this. If you don't feel confident, then it's something you both should really sit down and discusee. Just my take.
~FrankiE~