Are the agents proper managing agents, it sounds like they are arranging the check out/leaving process? Do they hold your deposit too?
I think you should try to put the responsibility with them and stay out of it. Obviously, you wont want to be in a flat without fridge, carpets etc, so he cant have access for that till after you go anyway.
So arrange the date for your departure with the agents, get them to do the checkout that says you left it as you took it. Ensure you have something written to say they are happy with the state, and you will get your deposit back. Then give the agents the keys. If the owner wants to get it, he can do it after you go.
Just make sure you get your deposit back, as if he is being repossesed, and he holds the deposit, you could have problems.