I have Jacobs ladder & have never had a problem & from what you describe I'm wondering if your plants are Solomon's seal
here?
what you describe sounds like sawfly lavae (not caterpillars) which can only be controlled by picking them off & killing them as they become visible (ugh! I know!) or using a contact insecticide.
I don't use chemicals but I usually manage to hold on to my solomon's seal for a short while & enjoy it until the larvae start going faster than I do (not difficult).
If I'm wrong about your plant please ignore this.... :o)