The Dim-Post

August 6, 2012

Polls, August 2011 to August 2012

Filed under: polls — danylmc @ 2:19 pm

TVNZ and TV3 both had new polls out last night. It’s always bearing in mind that each poll is only a single, very error prone data point. Stepping back and looking at the average polls over the last year, adjusted for their bias compared to the election result (thanks again to Peter Green for the R coding), we see the broad trends emerge.

My take on this is that Labour’s small target strategy has entered the realm of diminishing returns. Shearer has kept a low profile ever since his ‘I don’t have a dream’ speeches bombed (presumably this year is now designated his training wheels year) and this resulted in temporary gains for Labour because it coincided with a truly horrible six months for the government. Now they’ve peeled off soft votes, they’ll have to work harder for the rest. The Greens remain pretty steady.

As usual there’s a narrative vacuum from both main opposition parties. There’s no coherent critique of the government or substantive alternate vision. So whenever there’s a scandal or policy failure it takes place in this vacuum. And, as usual, National is very adept at crafting and promoting its own narratives. Unless this changes I don’t see these polls moving much over the next six months.

About these ads

21 Comments »

  1. Your analysis seems sound enough to me. Maybe you can expand a wee bit in future on role public inattention plays. I sometimes wonder what it would take to make the 45% who support National actually pay attention. I’ve found that when you run through the list of things that are ‘wrong’….then they sit up and take notice. But left to their own devices, they remain blissfully unaware….even as their own jobs evaporate or their pay is left the same as last years despite the profits where they work doubling. They see no connection.

    Comment by Steve (@nza1) — August 6, 2012 @ 2:33 pm

  2. As usual there’s a narrative vacuum from both main opposition parties. There’s no coherent critique of the government or substantive alternate vision.

    This is the puzzling bit. Taking asset sales for example, while the profile of the opposition is high (definitely driven by the GP as opposed to NZLP), no traction is being gained in the ‘alternate vision’ space.

    Possibly the logic is “This is such a fucking stupid idea, that apart from pointing out the obvious, there is no need to discuss anything other than the status quo” but that’s hard to put into a pithy soundbite when English is promising unicorns and icecream for all funded by the sale.

    The pitch then predictably becomes “Why do you hate unicorns and icecream, Pinkos? WHY?!”

    Comment by Gregor W — August 6, 2012 @ 2:40 pm

  3. A credible and engaging alternative would be nice to back up the ‘we hates them we does’ media sound bites.

    Comment by merv — August 6, 2012 @ 4:28 pm

  4. Although I generally agree with your analysis the media analysis is dreadful. A poll movement barely outside margin of error moves it from being a disaster for National John Key to a disaster for David Shearer! Clearly it is going to be close for next couple of years, but left have a clear advantage as National are stuffed if they get under 45. Colin Craig may bail them out a little, but that prospect is just as likely to scare liberals away.
    It is difficult for parities to have a positive narrative at the moment as they get turned into jokes by the media. Brownlees diplomatically disruptive comments about Finland shut that issue down very well.

    Comment by Luke C — August 6, 2012 @ 8:46 pm

  5. “adjusted for their bias compared to the election result” Really?

    Please elaborate. Is this the same adjustment used by the climategate people when assessing temperature recordings?

    Does it happen to make Labour look better and National look worse?

    Is your black line NZ First? Funny then that you have them at better than six percent when both TV3 and TVNZ have them at just three percent.

    Comment by Rob Ripley — August 6, 2012 @ 9:34 pm

  6. It’s always bearing in mind that each poll is only a single, very error prone data point. Stepping back and looking at the average polls over the last year, adjusted for their bias compared to the election result, we see the broad trends emerge.
    Not according to non-partisan blogger David Farrar.
    I think the polls reflect that the Government has got through the various distractions and issues of the first six months, and is now concentrating on implementing its programme. The other factor is that Labour are not a convincing alternative yet, and in fact are going backwards.
    HEH.

    Comment by questlovenz — August 6, 2012 @ 9:36 pm

  7. Darn it I’m confused again!

    So in this chart the bias in each poll is measured with just a single data point, which is the last election? Calculating a bias where n=1? Also we need to be careful not assume that methodologies stay consistent, especially just after an election.

    “It’s always bearing in mind that each poll is only a single, very error prone data point.”

    The samples sizes are around 1000 for each of the last two polls, so isn’t each poll result the average of 1000 data points, not one point? I’m confused about what you are referring to. You can’t be claiming just one point in the prediction of an election that’s two and a half years away, surely? Polls can only measure public sentiment at the time they are conducted. Even polls carried out right before an election can’t really predict an election day result – they can only measure public sentiment the week prior. They use samples of 1000 to measure that sentiment, which is fairly robust as long as good fieldwork practices are consistently followed and the questions are consistently understood.

    If you look at the Electoral Commission research, 40% or so of non-voters decided not to vote on election day. How can the polls factor that in?

    My concern is that the chart looks pretty, and sounds logical to those without much knowledge of statistics and probability theory, but statistically speaking it doesn’t seem robust to me. Forgive me if I simply misunderstand.

    Comment by Andy — August 6, 2012 @ 10:15 pm

  8. @ Rob Ripley

    You might want to read the other posts under the “polls” tag.

    Comment by sammy 2.0 — August 6, 2012 @ 10:53 pm

  9. Even polls carried out right before an election can’t really predict an election day result – they can only measure public sentiment the week prior.

    Surely the entire population doesn’t enter the polling booth with a coin ready to flip? If they have a stated preference a week out, that’s highly likely to be the way they vote.

    Comment by Neil — August 7, 2012 @ 6:26 am

  10. Surely the entire population doesn’t enter the polling booth with a coin ready to flip? If they have a stated preference a week out, that’s highly likely to be the way they vote.

    Well, when they invent a poll that can accurately sample an entire population instead of whatever small group of people they can get hold of, that would be a valid objection.

    Comment by Flynn — August 7, 2012 @ 6:55 am

  11. After Labour’s crushing defeat there was a slight glimmer of hope (now all but extinguished) that members would get a chance to elect a leadership capable of winning an election. I was at the Dunedin meeting when the candidates were asked ( first question out of the blocks if I remember correctly and from an old seasoned grass roots campaigner) “If the polls show your leadership still languishing in a year’s time will you step down?” Cunliffe said “Yes” Shearer said “waffle waffle waffle”.

    Comment by Sunny — August 7, 2012 @ 6:57 am

  12. So in this chart the bias in each poll is measured with just a single data point, which is the last election? Calculating a bias where n=1?

    That’s not ideal, but sadly we don’t hold elections after each new poll, and an n of 1 is greater than an n of 0, which is the bias correction for each individual poll.

    To me the point of these charts is not to make a robust prediction of what the next election result will be but to point out that the polls are very noisy, and taken individually they’re basically meaningless.

    Comment by danylmc — August 7, 2012 @ 7:57 am

  13. What is it with the Labour Party and the Australian Labor Party where the MPs pick leaders that ordinary voters, especially wavering ordinary voters, can’t get behind?

    Comment by billbennettnz — August 7, 2012 @ 9:56 am

  14. Eugh, I’m still not convinced by the NZ-First result.

    There’s enough of a plausible narrative to suggest Winston really did ‘surge’ in the final days before the election. Winston lucked into the Key&Banks debacle, and took full advantage of it.

    My gut feeling is that the model’s holding on to old polling data just a touch too long, and would be better off showing a more aggressive pick up in the NZF trend line during November. I’d be pulling their trend down a touch, and handing some of that vote back to National for the same (vice-versa) reason.

    Comment by Phil — August 7, 2012 @ 11:04 am

  15. @Phil: that apparent drop in NZF support is because the 3 News Reid Research poll gets consistently lower poll results for NZF than the Roy Morgan Research poll. The main purpose of the bias correction is to stop the smoother picking up spurious movements like that.

    Comment by pete — August 7, 2012 @ 11:58 am

  16. Sorry – didn’t make it clear @14. I’m not talking about the current NZF result, I’m talking about the whole level of NZF vote from November 11 to now.

    What I’m getting at is: the model assumes polls taken today under-represent NZF votes, because the election result was higher than the model’s assumptions about polling trends in the lead up to Nov-11. However, if the model took a slightly different approach to weighting it might do a better job of picking up what I called Winston’s ‘surge’, and would therefore have come closer to picking NZF’s actual election result. A consequence of this would be, post-election, the NZF trend would look closer to the results we’re now seeing polling produce.

    Comment by Phil — August 7, 2012 @ 12:52 pm

  17. Apologies Phil, that would have been perfectly clear if I hadn’t somehow confused November and August.

    It’s hard to see on the static graph, but the evidence for a NZF surge was pretty weak. There’s an interactive version here, which lets you isolate each polling firm. A more aggressive fit would have improved the prediction for NZF, but it would have played havoc with the (pretty good) predictions for the three major parties.

    Pedantry: what you refer to as the model’s “assumptions” should more properly be called “estimates”. For example, the model assumes that each firm has a constant bias in its results for each party, but the sign and size of that bias is estimated from the data.

    Comment by pete — August 7, 2012 @ 2:04 pm

  18. @Andy:

    I’m not a fan of “N equals such-and-such” arguments. A poll of 1000 voters could be interpreted as n=1 binomial observation or n=1000 Bernoulli observations, but the inferences would be the same in either case. Similarly the election could be read as n=1, or n=2278989. The problem is not with a small sample size, it’s with generalising to elections other than the one considered.

    Unfortunately using multiple elections put too much strain on the “constant bias” assumption; methodologies are closer to approximately constant over a shorter period (about one election cycle seems to be the limit). I’ve got a few ideas for getting around this, but they’ll have to wait until I have enough time.

    Comment by pete — August 7, 2012 @ 2:13 pm

  19. Pollsters be polling.

    Comment by Mackey — August 7, 2012 @ 9:18 pm

  20. why would you measure the bias with respect to only the last election? surely you can use the last 3 elections to see how the opinion poll results compare to the actual election results they are supposed to be predicting?

    Comment by kahikatea — August 7, 2012 @ 9:47 pm

  21. @kahikatea: the model needs to assume constant bias to work. So you can estimate bias over the last three elections, but it will be inaccurate if polling methodology changes over that timeframe.

    Comment by pete — August 7, 2012 @ 9:59 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 336 other followers

%d bloggers like this: