The Dim-Post

May 16, 2014


Filed under: Politics — danylmc @ 3:04 pm

Last week I posted about the predictive accuracy of polls in the 2011 election. Gavin White of UMR wrote to inform me that I’d made a mistake in the formula, so my chart was wrong. Here’s an amended chart. My original point still stands: the Roy Morgan poll isn’t as meaningless as some political commentators and Cabinet Ministers seem to think it is.


I should note that the Fairfax poll is now conducted by Ipsos, not Research International. They still seem to measure National higher than most other pollsters, and New Zealand First quite a bit lower. Also I was intrigued to see the latest Fairfax/Ipsos poll polled 1,011 subjects over three nights. If they’re getting a representative demographic sample in that time then that’s pretty impressive . If they’re not then that’s a bit worrying.


  1. Heh… I did wonder. Your previous results for Colmar Brunton looked way too impressive, and DigiPoll really got the rough end of the stick.

    Comment by Andrew — May 16, 2014 @ 5:15 pm

  2. Regarding the Fairfax polls, see my comment on The Standard here (scroll down to swordfish at 6)…

    Also important to understand that National and Right-Bloc support has been consistently and significantly over-stated (month after month) in opinion polls throughout the 18 months leading up to the last 2 general elections. It doesn’t really matter if these polls were correct (with Nat/Right support falling heavily away at the subsequent election for various reasons) or if they were wrong in their measurement (due to methodological problems). The fact is the Nats and the Right received far less at the 08 and 11 elections than they were polling months out.

    Here are National’s monthly poll averages for 08 and 11. And (in parentheses) I’ve compared these with National’s actual Party-Vote result at the subsequent election later that year: (so, for example, the Nats averaged 52% in polls of March 2011 and that was 5 points higher than the proportion they in fact received at the 2011 election):

    National 2011

    March 52% (+5), April 54% (+7), May 52% (+5), June 53% (+6), July 53% (+6), August 54% (+7), September 55%(+8), October 54% (+7), November 52% (+5), 2011 Election 47%.

    National 2008

    March 49% (+4), April 51% (+6), May 52% (+7), June 54% (+9), July 51% (+6), August 49% (+4), September 49% (+4), October 48% (+ 3), 2008 Election 45%.

    And here are the Fairfax polls:

    National 2011

    JULY Nat 56%
    AUGUST Nat 57%
    SEPTEMBER Nat 54%
    OCTOBER Nat 53%
    Early NOV Nat 53%
    Mid NOV Nat 54%
    2011 Election Nat 47%

    National 2008

    FEBRUARY Nat 55%
    APRIL Nat 52%
    MAY Nat 56%
    JUNE Nat 54%
    JULY Nat 51%
    AUGUST Nat 54%
    SEPTEMBER Nat 52%
    OCTOBER Nat 51%
    NOVEMBER Nat 49%
    2008 Election Nat 45%

    And that’s pretty much all I’ve got to say for the time being. Call me old-fashioned, but there you are.

    Comment by swordfish — May 17, 2014 @ 7:47 am

  3. Well done swordfish. A bit tough when facts get in the way of National supporters “craft.” On the good side maybe it will be good to have complacency riddle Mr Key’s confidence.

    Comment by xianmac — May 17, 2014 @ 3:21 pm

  4. Not sure if James and Grumpollie is the same person but Great work. How significant do you think the undecided voters are in a poll. I think that the last Morgan poll the number was just 5+%. In some polls the undecided may be between 10-20%.

    Comment by xianmac — May 17, 2014 @ 3:29 pm

  5. @xianmac

    They aren’t the same person, but thanks. I work for one of the polling companies so I do have a bit of an advantage when it comes to guessing what the different polls do.

    In answer to your question, I think the undecideds are incredibly important in determining a final election result. As far as I know, most polls probe undecided voters to try to get an idea of their lean or preference. This may not make a big difference to a poll result, or mean much at all, quite far from an election – but I think it’s very important during the week prior to an election. During that week, I think some of the undecideds will be paying closer attention to politics and will be starting to lean one way or another.

    I wouldn’t pay too much attention to the % of undecides between different polling companies. It would depend on a lot of things, like how the survey is introduced, whether their % includes those unlikely to vote, and where the party preference question is placed in the questionnaire.

    Comment by Andrew — May 17, 2014 @ 6:05 pm

  6. What is the same data for Labour and Greens

    Comment by ghostwhowalksnz — May 18, 2014 @ 1:32 pm

  7. Not sure about NZ, but in Australia the polling companies are well aware that polls sufficiently in advance of the election cannot be verified. And that the final poll before the election is the one that will be used for this type of analysis. Therefore the final poll before the election has 2-3 times the sample size. In effect, it’s not the same quality of poll as those earlier in the campaign (much higher quality), and therefore the accuracy or otherwise of that final poll is not necessarily indicative of the accuracy of polls in the leadup to the election.

    Comment by PaulL — May 19, 2014 @ 4:48 am

  8. That’s interesting, given that to increase the precision of a sample of 1,000 by just +/- 1 percentage point at the 95% confidence level, you’d need to more than double the the sample size (and more than double the cost of fieldwork).

    Comment by Andrew — May 19, 2014 @ 1:15 pm

  9. I always thought it funny when news people claimed the “polls narrowed” at election time. I took it that whatever bias there was in the polls between elctions was torn away by the actual vote…..and this gave the appearance of a ‘narrowing”.

    Comment by Steve (@nza1) — May 22, 2014 @ 1:39 pm

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: