I used to suffer from terrible mouth ulcers, both in my mouth and also down my throat. I too tried everything to stop them combing back, and in the end I found a really cheap toothpaste, one which did not have all the added ingredients, and after using that for a short while, the ulcers did not return. When I tried other fancy toothpastes again, back they came, so I just stuck with the cheap basic one, and was ok.
I know how painful they can be, so it may be worth a try for you too.