You don’t need us to tell you that polls are enormously influential. One week before polling day The Times reported that fears about the Union precipitated the sale of £17 billion of UK shares, bonds and other financial assets – the largest such event since the collapse of Lehman Brothers in 2008. While the No side won relatively comfortably, with 55% to 45% for Yes, it was clear that the Yes side had made considerable gains over the course of the campaign.
Though some of the polls did pick up the shift, it was also clear that polling data early on in the campaign could only have been so reliable. People change their mind and not everyone turns out. It’s very much a moving target.
Thinking now to the imminent referendum, it is inevitable that there will be some movement as we get closer to polling day, in particular as the defining issues of the campaign come into greater focus among a wider proportion of the electorate.
So, the question really is: “How reliable are polls as predictors of the outcome when they are considerably farther out from the election?”
Ireland provides the perfect petri dish for such a question. Across the water, referendums are very frequent; 25 have occurred in the past 30 years. The demography and culture of Ireland and the UK are relatively similar and the polling houses (most of which are based in the UK) are governed by the same rules – the rules of the British Polling Council. So we’re comparing apples with apples here.
Using the past 25 referendums held in Ireland we can model the relationship between the predictive accuracy of polls and the number of days until polling day. The predictive accuracy of the poll is measured in terms of the absolute percentage error between the poll and the actual result. The model accounts for turnout and other extraneous factors.
As Figure 1 shows, the polls only become marginally more accurate at predicting outcomes in the months leading up to the referendum, while in the last month, and in particular the last week, error plummets; the polls become considerably better predictors.
Other features that will be important to consider include turnout estimation and methodology.
Turnout – Expected turnout has a highly significant impact on polls as estimates for referendum outcomes, with polls that cover referendums with higher levels of turnout tending to be considerably more accurate than polls that cover referendums with lower turnout.
Methodology – This of course relates to a third feature; polling methodology, for which some polling houses can be considerably more accurate than others. This ranges from data collection mode effects through to question design and data adjustments. Evidence suggests that some approaches may be better than others for different types of election.
We will cover both these aspects in future editions of BMG’s #EURef Report.
Thinking allowed…what if issues are a proxy for how people may vote?
Given the divergence in current polling, an alternative view may be to take people’s concerns and views on issues they feel affect society more widely.
Often overlooked, a simple snap-shot of the main concerns of the three major groups in this campaign, the ‘remainers’, the ‘leavers’ and the ‘undecideds’ shows some interesting results.
Results from the latest BMG Research poll show that leavers are relatively united in their views. Half of leavers (49%) rated immigration as the most important issue facing society today, demonstrating a very clear and salient area for the leave campaigns to rally around. Conversely, remainers tend to be more evenly distributed in their concerns. For remainers, only a fifth selected their top issue healthcare (20%) and this only nudges immigration (16%), the national economy (11%) and the cost of living.
The relatively diffuse nature of remainers’ concerns could have two major implications on the ‘in’ campaign.
First, we know from previous referendums (Scotland 2014, AV 2011) that clear issues are easier to rally around. Developing an argument takes time to bed in, so running on a few clear and powerful arguments is better than many complex and dry ones. Think about the pound in the Scottish Independence Referendum for instance, and then more widely national economic security. Remain will need their ‘pound’ for this contest.
Second, in terms of the issues, though the undecideds appear to be somewhere in between those on either side of the debate, there is no doubt that they seem to lean more towards the leavers than remainers might like, with immigration the clear first concern (27%) and healthcare (16%), cost of living (12%) and the national economy (9%)
Undoubtedly both camps will be focusing on the undecideds in the coming months as the key constituents open to persuasion, but with immigration a key concern for those yet to make up their mind, those on the remain side might need to confront the immigration debate if they are to thwart a potential break towards ‘out’ later in the contest.
Readers should note that polling averages suggest that, at this time, the undecideds are easily capable of deciding the outcome of this contest and that a close loss for leave could easily be spun as a win, as demonstrated by the Scottish independence referendum.
For further details about our #EURef reporting series, or any of the results from our polling series, please feel free to get in touch by email or phone.
0121 333 6006
Dr Kevin Cunningham – Consultant
Dr Michael Turner – Research Director