The big unknown, Janz, is the quality of the timber and the treatment given to it before you get your hands on it.
It's very difficult with things like fence panels. "You get what you pay for" still holds. Panels are sold quite cheaply, and I've no idea what level of preservation they get.
Assuming it's been well treated, I can tell you that a fence post of good quality, planted in the ground can still be rot-free after 40 or even 50 years.
Unless you suspect your panels are really cheap and nasty, I wouldn't worry about it.
I agree with Karl. Fence staining is mostly "cosmetic."