How Can The Pollsters Improve?

[ToraToraTora]

http://www.bbc.co.uk/news/uk-politics-35151201
Badly wrong this year, even that septic Nate Silver got it wrong and he called every US state at the last college so what now for pollsters? any ideas for a new approach?

[jim360]

One thing I'm not sure has been given too much consideration is how much the polls themselves became part of the story. The message that they sent in the build-up to the election is that Labour wasn't in a position to win, facing serious trouble in Scotland, so that with the Tories unlikely to win either we were looking at a hung Parliament. Despite repeated denials, a lot of people suspected that this would lead to a Lab-SNP coalition, and I'm not sure that even many Labour supporters would have wanted that. This (so my hypothesis goes) helped drive a few wavering voters to the "safe" Tory option -- delivering a Tory majority, avoiding the dreaded hung parliament and thereby upsetting all pollsters.

In the event they were right in Scotland, and apparently in a few other key areas, so I don't think the polling science itself is under threat. My claim is that they became a part of the story, a factor that influenced some people's decisions -- enough to swing the result. I don't see how pollsters can account for how people will react to them. One possible response would be to restrict polling in, say, the final two weeks or so before an election.

[ToraToraTora]

yes but they base their predictions on percentages and the election is not based on percentages so it's not surprising they get in wrong. Time for a rethink?

[jim360]

It depends rather a lot on the model, actually. Quite a few out there are sophisticated enough to cope with the constituency system and correct for that to allow for local variations. One rather obvious way to see this is that in exactly none of the predictions ever did UKIP's ~10% or so of the vote translate into a predicted 65 seats, but rather somewhere between 0 and 5. Which is hardly a percentage-based prediction, clearly.

I do think the key factor is that polls can only work effectively if they are decoupled from the system they are trying to measure. Anything else you can correct for or try to account for, with some margin of error of course. Indeed, much of it already has been! As an example, it's been observed that typically Labour support ends up a little overestimated, for various reasons -- as a result, I'm fairly sure that the raw data gives Labour more support than is actually quoted in the published result (by about two or three points, say).

This is anyway going to take a more complicated fix than anything you or I can come up with, and isn't as basic as suddenly realising that we don't have PR in this country. I mean, duh.

[ichkeria]

Maybe the polls weren't that wrong? You can only go by what people tell you in advance, and it seems that for whatever reason Tory votes are often hard to predict. The exit polls, and the polls in Scotland, were largely right.
In some countries indeed opinion polls are restricted, and that seems a good thing to me

[ToraToraTora]

the exit polls were bang on much to the embarrassment of Paddy Pantsdown! Pity the non exit polls seem to struggle!

[Peter Pedant]

what is the question - and what does septic mean in 3T speak this week ?

why arent pollsters more accurate considering their pay is dependent on their getting the results right

ask them - people lying seems to be a leading cause previously

Silver I think got it right previously by using medians rather than means for prediction ( not percentages ) but I think the point is a bit scientific for this thread ....

[Peter Pedant]

Nte Silver has written a book by the way
penguin
The signal and the noise