Track records and outliers

It appears to be a volatile time. In the last week, one poll (Newshub) had National at 47% and able to govern alone, and another (Newsroom) had National down to 30% with Labour at 45%. And a third (TVNZ News) had Labour at 44% holding a 4-point lead over National and able to govern with the Greens on 7%. But in these wildly diverging results, is there an underlying signal?

Just before last year’s election, I wrote on the accuracy of the election eve polls from the (then) five main polling companies in 2011. The One News-Colmar Brunton poll came out on top, followed closely by Roy Morgan.

This time around, there are only three still currently polling who were around in 2014: 1 News-Colmar Brunton, Newshub-Reid (then it was 3 News), and Roy Morgan.

The table below shows the eve of election poll results published before the 2014 election, and the actual election results. The green cells show which were closest for each party.

polls & election results 2014.JPG

The next table shows the variation of each poll from the election result. The blue cells show over-estimation from the final result, and the red cells under-estimation. The yellow cells show the average variation, using absolute values, for each party and for each poll overall (I’ve used absolute values to get the degree of difference from the final result, which would otherwise have been nullified by the combination of positive and negative values).

variation 2014.JPG

Interestingly, all three underestimated National’s performance (after overestimating in 2011). All substantially overestimated the Greens – perhaps ominous given Greens current rating just above the thresholds – but were reasonably close to the remaining parties.

As with 2011, One News was closest overall to the final outcome in 2014, in terms of having the lowest average variation, but there wasn’t much in it between the three.

This time around, there’s been a bit of a shakeup. As well as the three established players, the NZ Listener and the Newsroom websites have introduced their own online polls into the debate. And the Newshub-Reid method has been overhauled to included one-quarter of its sample being drawn from an online panel, in an attempt to reach those without landline access.

There’s a lot of debate about the accuracy of online polls, partiuclarly for the representativeness of the samples, and the panels that they’re drawn from. As the New York Times observed recently, “the new Internet approaches represent high-profile, very public experiments in the next generation of opinion research. The problem is that we don’t yet know which ones will pan out.”

Recent posts have looked at the changing trends of the political parties since Jacinda Ardern took over the Labour leadership. I’ve tended to focus on data from the three publicly available polling companies, but with the above debate in mind, it’s worth looking at the full range of polls for overall patterns and possible outliers.

The two graphs below show the polling results for National and Labour since Ardern became Labour leader. The pattern for National has most polling clustered around the trendline, but with the Newshub and Newsroom polls of the past week looking well outside the overall trend. Interestingly, Newshub has National generally on or above the trendline, while TVNZ has National closer to the trendline.

nat trend 170914.png

And for Labour, the pattern is substantially in reverse, although the trendline is steeper – reflecting the votes that Labour is gathering from across the spectrum. TVNZ generally has Labour on or above the trendline, while Newshub has Labour voting more scattered either side of the trend. But the overall trend is clear.

lab trend 170914.png

In this hilarious exchange between Ardern and Patrick Gower from Newshub, Gower said “I think everybody’s right, I think the voters are very volatile.” There’s certainly change in the air, but the available data suggests some clear trends are coming through. This week’s Newshub poll, and the online poll from Newsroom, both appear to be outliers for opposite reasons, rather than volatility.

The trend so far in this campaign is away from the government, and maybe even towards change. And unless Colmar Brunton’s established approach has not kept pace with changes in telephone use, they’re still the one to watch ahead of election day.