I wear a casual, zip-up jacket, leather shoes as normal. If I go into town, especially if I go with my wife, I also wear a tie. [I'm 75, by the way, if that makes a difference.] I've always been accustomed to dressing 'respectably/neatly', so it's a habit that I've kept. As regards people looking much scruffier these days, well, when I was younger there wasn't the wide choice of fabrics and styles that's available nowadays, so perhaps it wasn't as easy, then, to look scruffy.