To add - further googling reveals it was one of the songs that competed to represent the UK at Eurovision in 1963. Looks like it was Vince Hill but the version on youtube is a re-recording, so sounds different [to me].
You'll find the lyric you mentioned just after half-way through.