The Dim-Post

November 19, 2012

Labour leadership and the polls

Filed under: Politics — danylmc @ 10:24 am

I’ve used Peter Green’s R code to generate an aggregated poll result from 01/01/2009 to today (without any bias correction) because it’s a useful way to think about the Labour leadership. (Always funny to look at the outliers, like that recent Roy Morgan that had Labour at 29%, and remember all the media hysteria about them.)

So we can see that yes, Shearer has gained in the polls relative to National, and hasn’t taken many – or any – voters off the Greens or New Zealand First, which is what you want to see. But his party is still only at the same point Goff was a year out from the election. All the way through Goff’s term as Labour leader ‘the bloggers’ kept pointing out that he was a walking disaster, and we kept hearing the same defenses we’re hearing today. ‘The polls will tighten. Helen Clark was unpopular at first. He’s a genuinely nice guy, etc.’ Then the election campaign kicked off, the general public started paying attention, they got a chance to see Goff, and him and his party tanked. To my mind that’s the risk of sticking with Shearer.

Cunliffe has his own problems – the egomania, the general weirdness – but he’s unlikely to fall to pieces in a debate against John Key.

About these ads

37 Comments »

  1. “….they got a chance to see Goff, and him and his party tanked.”
    Though at the tanking the left grew and National tanked as well. Not sure if NZF was leftish though?

    Comment by xianmac — November 19, 2012 @ 10:32 am

  2. My other worry is look at the people behind Cunliffe: Mahuta, Prasad, Jones. Not exactly dynamic.

    Only one time as well but Shearer looked a lot more commanding on Breakfast this mroning

    Comment by max — November 19, 2012 @ 10:39 am

  3. (Always funny to look at the outliers, like that recent Roy Morgan that had Labour at 29%, and remember all the media hysteria about them.)

    The biggest polling outliers for both National and New Zealand First appear to be the election :-)

    Comment by Graeme Edgeler — November 19, 2012 @ 11:14 am

  4. So – since Labour crashed to 27% in the election, their vote has now climbed 5.4 per cent.

    At the same time the National Party, with the most popular Prime Minister in history, has dropped 1.8 per cent.

    That has happened alongside the Green Party maintaining the highest level of support it has ever received.

    Shearer has been in charge for 9 months or so. The election wasn’t even one year ago, however these trends now seem established and the poll-of-polls gap between the two parties has narrowed 7.2 per cent.

    Clearly he is terrible and has to go.

    Comment by Bill Engrish — November 19, 2012 @ 11:15 am

  5. But thinking about useful pool aggregates. Even without the sort of turnout or bias correction Nate Silver does, or possibly even without aggregation, wouldn’t it be great if our political polls were reported with probabilities?

    e.g. based on this poll, there is a 15% chance that Labour + Greens + NZF will get a parliamentary majority, and a 30% chance they would more seats between them than National + ACT + United Future + Conservatives.

    Comment by Graeme Edgeler — November 19, 2012 @ 11:21 am

  6. No, the people behind Cunliffe don’t look commanding. But then again, the people backing Shearer are supporting a man who still appears to be unsuited to the job, in spite of one popular speech. From being weak and indecisive he is now appearing sort of forcedly decisive and domineering: “I am the leader, I will decide”.

    I hate the way Cunliffe is blamed for disrupting the conference when there has been a groundswell of horror amongst the left at Shearer’s weakness and no doubt a reaching out to Cunliffe as the party’s – and country’s – best hope. To my mind, it’s not about a power play but about the good of the country. Sounds OTT but I’m genuinely worried about the Nat’s policies this term. I want a Labour/Green alliance and with Shearer … Well, it isn’t going to happen.

    Comment by Mo — November 19, 2012 @ 11:21 am

  7. @ Graeme: “Even without the sort of turnout or bias correction Nate Silver does, or possibly even without aggregation, wouldn’t it be great if our political polls were reported with probabilities?”

    Ummm … what does this mean? Opinion polls already are reported with a probability attached … in 95 of 100 polls conducted using this method, the result will be within +/- 3.2% of these figures (or similar). To then want a further probability that the reported figure will actually translate into some real-world outcome requires data external to the individual poll – either some sort of historically-weighted analysis of the accuracy of the polling method, or assumptions about the likelihood that those polled will actually vote, or the like.

    In short, you can’t have the sort of prediction you want (“based on this poll, there is a 15% chance that Labour + Greens + NZF will get a parliamentary majority …etc, etc”) without some sort of Nate Silver magic being applied … and that magic certainly can’t be worked on the basis of “this poll”.

    Comment by Andrew Geddis — November 19, 2012 @ 11:47 am

  8. Danyl wrote: Cunliffe has his own problems – the egomania, the general weirdness

    If you think David Cunliffe is weird, you must be hanging out with some very normal people

    Comment by kahikatea — November 19, 2012 @ 11:52 am

  9. Graeme wrote “Even without the sort of turnout or bias correction Nate Silver does, or possibly even without aggregation, wouldn’t it be great if our political polls were reported with probabilities?”

    I heard somewhere that the Herald corrects for expected likelihood of voter turnout on demographic grounds. This could explain why the Herald polls tend to under-predict Green Party support, in that the Green Party has higher levels of support among young people, and young people as a group are demographically less likely to vote, but young people who tend to support the Green party do not necessarily have as low a turnout as young people as a whole.

    Comment by kahikatea — November 19, 2012 @ 11:57 am

  10. Ummm … what does this mean? Opinion polls already are reported with a probability attached … in 95 of 100 polls conducted using this method, the result will be within +/- 3.2% of these figures (or similar). To then want a further probability that the reported figure will actually translate into some real-world outcome requires data external to the individual poll – either some sort of historically-weighted analysis of the accuracy of the polling method, or assumptions about the likelihood that those polled will actually vote, or the like.

    That’s not what the the 95% confidence level means :-)

    Also, opinion polls do not have probabilities reported!
    For example, the latest Colmar Brunton poll (n=856 for this question) had New Zealand First on 4.9%. Wouldn’t it be helpful to tell us that this poll gives NZF a ~46% chance of being in the next Parliament?
    That information is there for anyone who knows how to work out, but no-one does.

    When Polls were showing National in the high 40s, would it have been great to know that some poll or other showed National having a 35% chance of being able to govern alone, or a 45% chance of having a legislative majority with just the ACT Party, etc.?

    Doing calculations like this for all parties, you could “simulate” an election multiple times, based on a single poll, and then advise the types of probabilities I’m talking about. You don’t need turnout information, or assessments of economic data. Just take the information we’ve already got, and do something else with the numbers.

    Comment by Graeme Edgeler — November 19, 2012 @ 12:31 pm

  11. Even taking the probabilities over to the seat results would be helpful.

    If the polling shows there’s a 95% chance of National having between 42% and 48% of the vote, then you’re 95% sure National will have between ~52 and ~60 seats in a 121 seat Parliament

    Comment by Graeme Edgeler — November 19, 2012 @ 12:53 pm

  12. @kahikatea: I don’t think Green support is necessarily underestimated by the Herald or anyone else. Most of the bias is shifting NZ First support to National, with Labour and Green sampled accurately. Even then, this could be a matter of National’s likely voters switching to NZF during the campaign (the next version of the tracking poll will need to account for faster movement close to an election).

    @Graeme, Andrew: That sort of percentage should be fairly simple if you want the equivalent of Nate Silver’s “now-cast”, even based on a single poll. The tricky bit is allowing for movement over the next two years to get a proper forecast probability. If you want to do that you’re going to need multiple polls to get an idea of how fast things can move.

    Comment by pete — November 19, 2012 @ 1:01 pm

  13. “I heard somewhere that the Herald corrects for expected likelihood of voter turnout on demographic grounds.”

    I’m pretty sure that all the polling firms do this. I’d prefer that they gave an “eligible voters” estimate first, with “likely voters” as a secondary result. “Likely voters” is useful for the horserace, but the “eligible voters” estimate tells you more about actual support for the government.

    Comment by pete — November 19, 2012 @ 1:05 pm

  14. I was thinking the same thing last week, Graeme. Nate’s mathemagic is not the least bit difficult to follow. Surprised no one has done it here. David Hood suggested on PAS that the horse trading for government thing would be hard to model, but that doesn’t mean it wouldn’t be interesting analysis.

    Comment by Ben Wilson — November 19, 2012 @ 1:33 pm

  15. pete wrote @kahikatea: I don’t think Green support is necessarily underestimated by the Herald or anyone else

    I haven’t studied this recently, but in the period 2005-2008, the Green Party consistently polled significantly lower on the Herald Digipoll than on the other polls. Of course, because there’s only one election every three years, and because voter preferences may dramatically change during the campaign period, it’s hard to know for certain whether that’s an underestimation by the Herald poll or an overestimation by the other polls, but I don’t think it’s unreasonable to interpret it as an underestimation by the Herald poll.

    Comment by kahikatea — November 19, 2012 @ 2:18 pm

  16. The thing that annoys me is that they only show decided voters. They never show ‘undecided’. Especially around elections, this is crucial.

    Comment by max — November 19, 2012 @ 2:43 pm

  17. They never show ‘undecided’…

    That would sometimes (often?) demonstrate the polls to be quite unrepresentative. So, of course, “they” are not going to do that.

    Comment by RJL — November 19, 2012 @ 2:53 pm

  18. @kahikatea: I had a look at 2005-2008, and there was a bigger spread in the polls for the Green Party back then. About two points between Herald/Fairfax/One at the low end and RMR at the high end. Based on the 2008 election results, it looks like the firms on the low end were more accurate. Based on 2009-now it looks like about a 1-1.5 percentage point gap from Herald/One to 3/RMR, and Herald and One are closer to the result for last year’s election.

    Comment by pete — November 19, 2012 @ 3:47 pm

  19. No Leader of Labour will be able to have a rational debate with John Key because John Key is not rational. According to him, everything is running smoothly.

    Comment by Dan — November 19, 2012 @ 5:36 pm

  20. Good on Labour, especially Chippie, for ensuring the day after the policy announcements that all of the discussion is about David Cunliffe and not the policy. Dicks.

    Comment by Karl — November 19, 2012 @ 8:16 pm

  21. Danyl you are sounding like Mat Hooton.

    I worry about you sometimes.

    Comment by peterlepaysan — November 19, 2012 @ 8:37 pm

  22. “Ummm … what does this mean? Opinion polls already are reported with a probability attached … in 95 of 100 polls conducted using this method, the result will be within +/- 3.2% of these figures (or similar)”

    That’s not what the the 95% confidence level means

    What do you mean? That’s how it’s been explained to me.

    Comment by Steve Parkes — November 19, 2012 @ 11:39 pm

  23. What do you mean? That’s how it’s been explained to me.

    It may just be ambiguous, and the use of the word “result” in which is appears has thrown me off.

    When the real value of support is X, 95% of poll results will be within the range of X-m to X+m (where m is the margin of error for the 95% confidence level).

    Comment by Graeme Edgeler — November 20, 2012 @ 12:33 am

  24. Everyone expected John Key would fall to pieces under a withering Clark onslaught in the ’08 debates. Didn’t happen.

    By 2014 voters will have tired of Key’s sneering condescension just as they tired of the same in Clark. Shearers bumbling stammerama will be an appealing alternative.

    Comment by Aztec — November 20, 2012 @ 3:04 am

  25. Useful NZ election probabilities based on polls would be hard because (i) there’s a lack of readily-available historical data (and even if you were to go through decades of old newspapers looking for old poll results, there’s the question of whether anything predating MMP is relevant) (ii) races with more than two parties are volatile in a way that’s hard to model, especially since the volatility increases as the election approaches.

    Honestly if somebody waved a bunch of Rutherfords in front of me to come up with a 2014 probability forecast right now I’d base it on the previous election and on economic variables, and ignore polls — not because they’re irrelevant but because they’re difficult to deal with and the value they add is small. If we were six months out from the election then yeah, we’d need to use the polls.

    Comment by bradluen — November 20, 2012 @ 6:40 am

  26. You appear to be overlooking one major issue about Cunliffe. He isn’t a leader. He never has been. Some people are just more secure on the periphery. I think Cunliffe is one of these.

    Comment by Lee C — November 20, 2012 @ 7:04 am

  27. I tend to agree with your analysis of Shearer and Cunliffe. I think Cunliffe would make a much more effective leader of the opposition and would run a much more effective campaign in 2014.

    Having said that I think Shearer’s style would make him a great PM – the irony of course is that if he doesn’t change his style there is a higher risk of NZ not having a Labour PM until 2017.

    If he does get dumped from the leadership then Shearer should definitely be kept on in a senior capacity – I can imagine him making a fantastic minister of Foreign Affairs. (not to mention that other countries who only know NZ for it’s sheep would love the fact that our highest representative to them was a Shearer)

    Comment by Richard29 — November 20, 2012 @ 7:19 am

  28. @Graeme: “When the real value of support is X, 95% of poll results will be within the range of X-m to X+m (where m is the margin of error for the 95% confidence level).”

    But, of course, we don’t know the “real value of support”. That’s what the poll is approximating (through a randomised sample of respondants, contacted in a particular way). And the process of sampling and contacting can introduce distortions to the data, which pollsters can try to correct for … but not may not be able to do so accurately.

    Hence, the most we can say about a given poll is that “using THIS method of polling to determine that a party’s level of support is X, 95 out of 100 other polls using this method will produce a result within the range of X-m to X+m (where m is the margin of error for the 95% confidence level)”. Which is why, as Graeme himself noted, “The biggest polling outliers for both National and New Zealand First appear to be the election” … which is the only poll that gives us “the real value of support” of each party.

    Comment by Andrew Geddis — November 20, 2012 @ 8:02 am

  29. No. We can’t say that. The margin of error quoted does not account for bias, only sampling error (i.e. the chance that in randomly selecting a bunch of people, you happen to choose an unrepresentative sample). Effects caused by, for example, the type of person who is more likely to refuse to take a call, or cell-phones not being included, or people who aren’t at home and aren’t called back enough, aren’t included in the quoted margin of error.

    Comment by Graeme Edgeler — November 20, 2012 @ 10:18 am

  30. You can’t say

    “using THIS method of polling to determine that a party’s level of support is X, 95 out of 100 other polls using this method will produce a result within the range of X-m to X+m (where m is the margin of error for the 95% confidence level)”.

    It’s more like “If we ran this poll 100 times, we expect the true value of this party’s support in the population we are sampling to fall in the interval [p-m -- p+m] 95 times”. Apparently this is a “Level 8″ (postgrad!) concept in maths

    As you say, you might be sampling a population that is different from the one that turns up on election day.

    I think the point that Graeme is making is that we shouldn’t place undue attention on the edges of confidence intervals, if you are willing to treat polls as estimates of the population proportion then the true values are more likely to be close to the point-estimate than far from it. So, for instance, if you had a poll in which National was on 52% you could say, “ah, well, the CI contains 50% so maybe they don’t really have a stand-alone majority”, or, you could say “if the true support for National was 50% then we’d expect to result this extreme or more so about ~12% of the time”. Though, this is different from saying there is 12% chance that National is below 50% – it’s not actually possible to talk about the probability that an unknown variable takes a given value in the most common approach to statistics.

    (Now Pete or Brad can tell me what I got wrong…)

    Comment by David Winter (@TheAtavism) — November 20, 2012 @ 11:33 am

  31. David is right. I note only that everyone who isn’t a statistician makes probability statements about parameters all the time and for the most part nothing horrible happens.

    Comment by bradluen — November 20, 2012 @ 12:23 pm

  32. I’d rather say “We can be 95% confident that the true value (in the population we’re sampling) lies in the interval (p-m, p+m)”. The best thing about tautologies is that they’re always true.

    Comment by pete — November 20, 2012 @ 12:24 pm

  33. “Apparently this is a “Level 8″ (postgrad!) concept in maths”

    I’m pretty sure that “Level 8″ in that context means Year 13. Personally I’d be happy to relegate p-values to postgrad and teach kids Bayesian credible intervals instead.

    Comment by pete — November 20, 2012 @ 1:14 pm

  34. “He hasn’t taken many – or any – voters off [...]New Zealand First, which is what you want to see.”

    You know, I’m not sure that that’s what I want to see. As many votes as possible taken off New Zealand First, please.

    Comment by Milla — November 20, 2012 @ 1:40 pm

  35. @Graeme: 46%? Are you using a Jeffrey’s prior? (Terrible idea if you want to know the probability of NZF making the threshold, exactly correct if you want to calculate “based only on this poll”.)

    Comment by pete — November 20, 2012 @ 1:42 pm

  36. @David: “It’s more like “If we ran this poll 100 times, we expect the true value of this party’s support in the population we are sampling to fall in the interval [p-m -- p+m] 95 times.”

    Right – that’s what I was attempting to say … if it really is different to how I expressed it, then I’m happy to adopt that formulation. I get that there’s a difference between bias and sampling error … my point is that (1) a given polling firm’s method of gaining information from the electorate is as likely to suffer from the former as a given poll is the latter, no matter how carefully pollsters try to correct for it; and so (2) as different polling firms have different sampling techniques, then there is greater uncertainty about the match-up between their approximations of the electorate’s voting intentions and the real-world voting intentions of the electorate than the “margin of error” suggests.

    Which is why I think trying to mine even more information out of single polls (which is what Graeme wanted to see happen, which is where we started this discussion) actually isn’t very helpful at all … because single polls conducted using one sampling method aren’t a very good source of information about the real world, and so trying to make them appear even more authoritative (“this poll shows there is only a 12% chance that National will fall short of a majority government!”) is a bad thing. Now, of course, a responsible news media would carefully explain to its readers/listeners all the caveats that need to be placed on such a bald statement. But that would require that we have a responsible news media.

    Which is why poll aggregations are gooder sources of information. And which is why Nate Silvers are gooder still.

    Comment by Andrew Geddis — November 20, 2012 @ 1:46 pm

  37. >Which is why I think trying to mine even more information out of single polls

    I’m not sure if Graeme was suggesting that would be the only information used, just that it could be analyzed in a way that conveys answers to more questions than just how much support each party would get, according to the poll. The tricky bit would be in working out the horse trading based on the numbers. You could work out the probability of each possible coalition getting enough numbers, but working out whether the coalition would be viable, whether the parties would work with each other, involves a lot more analysis.

    Comment by Ben Wilson — November 20, 2012 @ 8:15 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 349 other followers

%d bloggers like this: