The basic stumbling block here is that you're thinking that our universe and infinity are the same thing....while they may be (if it can ever be proved) or may not bet the best way to think about it is that infinity is like the biggest number you can think of plus one, then plus one, then plus one and so on forever. The edge of infinity is unobtainable...but if there are ever proven to be other universes then it would be interesting to see which form they take.
Our universe isn't infinite but by definition is everything we can ever possibly see or measure - so we will never know whats outside. We can of course guess - maybe lots of other universes.
Perhaps space is a relationship between objects, rather than something in its own right. If that's true, then if there are no objects there's no space.