The problem is that deciding on an issue, and deciding on a person to decide about issues, are two very different decisions anyway and aren't comparable. There's no contradiction, or irony, or anything like that. For example, if you wanted to make a scientific decision, you could have no understanding about the science itself whilst being perfectly capable of recognising that you might want to ask a scientist to look at it. It seems to me to be essentially the same thing in politics. We implicitly trust the MPs to do all the hard work of researching, listening to competing opinions and expert advice, and being aware of the wider issues, either because we don't have the time or the expertise to do that properly ourselves.
If you want to undermine the criticism of Sir David's point, you can do no better than asking yourself the question: "why are you fine with a referendum on the question about leaving the EU or not, but not fine at all with a referendum on the details of that decision?" Or, should we not hold a further set of referenda to check whether or not we want to stay in or leave the single market; what (if any) level of contribution is acceptable in future if we'd rather try to keep access to that single market; whether we want to emphasise immigration levels as a whole over all else; whether or not there should be an exemption for eg foreign university students; a desire to maintain tariff-free trade where possible... and so on, and so forth. Presumably, on each of these most of you will have a particular preference, but they are all questions that deserve a considered answer. Putting them to the people in further referenda, on the other hand, is just utterly stupid, in part because it's a waste of money but mostly because that's clearly not what referenda should be for.
Where Sir David goes too far perhaps is in arguing that there shouldn't have been a referendum at all, but that is more about where you draw the line between questions that are appropriate (or not) for referendum, rather than whether there should be a line at all.