Sorry to disagree, but thats wrong
leepylee. Even if it says in your agreement that the landlord or agent can make visits, with or without notice, they are not allowed to unless you agree. You can refuse to let anyone in whether you are there or not.
You, as the tenant, are entitled by law to 'quiet enjoyment'. If the landlord breaches this then you would be entitled to change the locks on the doors to prevent it happening again (you would have to change them back again at the end of the tenancy). The landlord could also be charged with harassment for persistent offences. Visit this landlords site for more info, its a subject that comes up fairly often
http://www.landlordzone.co.uk/forums/forumdisp lay.php?f=3
Of course, its always better to act reasonably and allow visits when its convenient for you. I dont know how you'd differentiate between investors and other buyers though.
Talk to the landlord or agent about who you would allow to visit and when.