I am no poetry fan, but have a soft spot for John Betjeman.
Reading it on a surface level it seems to me he is lamenting the break up of a relationship with another person (a girl I assume).
When you are "in love" with a person the whole world seems to revolve around what you do together, places you visit together have a different meaning.
So, for example, he says they discovered and visited many churches together as in "The vast suburban churches, Together we have found:
The ones which smelt of gaslight, The ones in incense drown'd;
But now they have broken up and he has no one to share these events and memories with the churches are just that, churches, and "I'll use them now for praying in"
From my own point of view I can see where he is coming from.
A few years ago my wife were in Norfolk and we wanted to take a boat trip but we arrived 2 hours too early as the tide was out.
So we just sat on a wall at the edge of the beach and for 2 hours watched the tide slowly come in, going round rocks, filling in rock pools etc.
I have great memories of that day, but if I had been on my own it may have been boring. Only because I has someone to share it with did the day mean something to me.
I think that is the sentiment Betjemen is trying to make.