No, it doesn't matter how much you have, over a certain basic level. Be like Sherlock Holmes and retain only what is useful to you. But I'm no Sherlock Holmes, unfortunately, and can't help remembering curious facts and odd gobbets of information, useless unless I need to converse with someone who's only interest is that subject (a bore, in other words), when I can feign interest and fuel the conversation with such gobbets.
Of course, like everyone else, I had to learn vast amounts of stuff in school, and also for my profession. Virtually none of it, including the professional exam stuff, have I retained !
And serious quizzers are something else entirely. Not so much intelligent and educated but people who have devoted their time to memorising subjects which are the subject of quizzes. .