What does it mean for a mathematical concept to exist? I don't think it means the same as something more physical existing, by a long stretch. The concepts are very abstract indeed. With a bit of care, you can in principle construct an entire Universe using mathematics, but that Universe has no reality beyond the scribbles on a piece of paper or on a blackboard. So to talk of, say, non-Euclidean geometry -- or even Euclidean geometry for that matter -- as existing "before" the Universe doesn't seem to make any sense. At least, not in a physical way, although it would perhaps be equally wrong to say that Euclidean geometry only started existing once Euclid wrote it down.
I don't know if it even makes sense to ask the question about whether Maths existed before the Universe did. There's no evidence that there was even a before. And there's this confusing though running through my head that maths has an existence independent of the Universe (because it describes things that are inherently non-real), but also depends on a Universe and creatures within it existing to discover/ invent the Maths. I don't know about anyone else reading this, but I've certainly managed to confuse myself.
The only thing I'm "certain" of is that Maths, in and of itself, has no bearing on the existence of a Creator.