Most stars are assured of lifespans measured in the billions of years. Our own Sun for example has an estimated 10 billion year life span, and we are thought to be around half way through that period. One of my favourite large numbers is the estimated 600 million tons of hydrogen burned per second, with a lifespan of 10 billion years! My brain cannot process that number :)
The overwhelming majority of stars you can see with the naked eye in the sky are actually 10s ,maybe hundreds of light years away.
Given that age of the light arriving to us now,(10s - 1000s of light years) and the likely life span of the stars (billions of years) it is very likely that the more easily observed stars in the sky are still there, merrily burning away.
It is true to infer that many stars will have died out, those estimated to be millions of light years away from us and whose light is only just reaching us now - but you would need something like the Hubble Telescope to see their light. It is also important to remember that star formation and death is a process, and that whilst stars are going out, new ones are being born.
So, the idea that stars whose light is only just reaching us have already burned out is a reasonable supposition. What I do not quite understand is how you get to the universe ceasing to exist from that supposition......