Should we believe opinion polls?

On Facebook, activists regularly suggest that opinion polls should be ignored, because they are politically biased, or because they have proved inaccurate in the past. This is particularly important because a series of polls have now tried to understand the views of Labour members and voters on Brexit issues and policy.

Firstly, opinion polls are a much more systematic measurement of public attitudes than anything else – certainly there is nothing representative about those who answer to door to canvassers, or attend political meetings. This does not mean the polls are absolutely accurate, nor that bias is not be introduced by journalists reporting the results.

Secondly, there is no evidence that poll results are biased by the views of their owners (many of whom are conservatives). They earn their living by telling the truth as far as they can, no commercial advertiser or political party will pay serious money just to have its views or products flattered. For this reason, polling companies have a professional code, they publish their questionnaires and sampling methods and their data. Anyone can look for bias: if you find it, call it out!

Thirdly, most polls are accurate within their stated limits. All publish their margins of error, which are rarely reported in the press. In the case of the EU referendum YouGov consistently predicted a result within this margin throughout the campaign, and the final result matched this. Of 41 UK election and referendum polls conducted by YouGov up to 2016, their predictions were only twice out by more than 3%, and in 29 cases they were below 2%.

When an election or referendum is tight, the margin can reverse the result. This is particularly the case where elections are by constituency, and local variations between constituencies can overturn a prediction (thus Hillary Clinton won the popular vote in the US Presidential Election, but lost because the vote was distributed unevenly between States).

However, in most polls, the numbers of respondents in each constituency it too small for prediction. For this reason YouGov developed their MRP methodology, which combines the known attitudes of particular groups (drawn from large national samples), with the demography of individual constituencies, to predict at Constituency level. Thus we have a fairly clear idea of the proportion of Remain voters among white, 35 year old, university educated, women in high earning groups. We also know the proportion of the electorate in each constituency who fall into that group. By applying this to all the groups in each constituency it is possible to predict a result for each constituency. In the 2017 general election this predicted a national Conservative lead of 3.5%, against the result on the day of 2.4%.

All this suggests that one should believe polls within their advertised margins of error, but beware of how the results may be spun by journalists. When they suggest that a very high proportion of Labour voters now support remaining in the EU, and that there has been a significant swing to remain in the most leave voting Labour seats, they should not be dismissed.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.