'Line' is often used to mean an imaginary boundary of one sort or another, as in "Don't step over the line or you'll be punished." In the case of your mother's use of the word, I imagine she meant that she had got as far just being annoyed as she was prepared to go. If there were to be any further annoyance, she would be obliged to step over the 'line' and take drastic action! So far, she is just 'on' it.
You're not a Midlander by any chance, are you? I'd forgotten that my grandmother used that expression frequently, as did many of our neighbours in inner Birmingham in the 50s and 60s. I don't think I've heard it for 40 years or more.