TripAdvisor has a filter which pulls together reviews that include the word 'quiet' in them. That might only be 'quiet room' or 'quiet hotel', rather than 'quiet resort' or 'quiet location', so you do need to read through the reviews carefully, but it could be a useful starting point for you:
http://www.tripadvisor.co.uk/HotelsList-Rhodes-Quiet-Resorts-zfp9378.html
Avoid Faliraki - this is a clubbing/ bars and Very Loud town (went there in my youth.
A couple of my friends have literally just come back from an area further south called 'Afantou' - they stayed with friends so I can't recommend a hotel, but as they are walkers and love peace and quiet, they found loads of really nice/ historical places off the beaten track in that area and had a wonderful quiet holiday.