The Dim-Post

May 8, 2014

Roy Morgan Polls

Filed under: polls — danylmc @ 9:47 am

Via Andrea Vance’s bottom-feeding troll feed:

morgan This attitude towards the Morgan poll seems to be a thing around Parliament rather than just Joyce spinning a bad result. Lots of journalists and staffers have told me that they don’t trust the Morgan poll, that it’s ‘always bouncing around’ etc. And the Morgan poll is always ‘bouncing around’, because the Morgan poll is roughly every two weeks, while Fairfax, say, only poll three or four times a year. Polls are noisy datasets and they jump around a bit but if you poll frequently you can generally see trends in the data while if you have very long gaps between your polls any changes you see might just be random noise.

David Winter and Gavin White both looked at how the polls performed when predicting the actual outcome of the 2011 election. I aggregated Gavin White’s data for the different parties and graphed them to show overall predictive accuracy for each pollster. (A zero would mean your poll result was totally accurate predictor – so the larger the bar the worse you did).


The Morgan poll is about middle of the pack. Their latest one might not be accurate though. It’s only one poll. After all, the gallery and political pundits all warned us that the departure of universally beloved blue-collar bloke Shane Jones meant the utter annihilation of the Labour Party, and this Morgan Poll shows them gaining tens of thousands of center voters, even before the Williamson resignation and Collins melt-down – so maybe it is a rogue. I guess the other outlets are waiting until after the budget to conduct their polls, so we might have to wait a while before we get additional data.

(NB: As Pete points out in the comments there’s an element of luck here. You might have a really accurate poll that happens to get one bad result just before the election and so you’d do poorly on this analysis.)


  1. Here’s why regular polling matters – the noise takes quite a while to shake out of a representative random sample:

    (Takes one of your NYT articles, but it’s worth it for the incredible infographic.)

    Comment by George — May 8, 2014 @ 9:51 am

  2. Worth noting that both those analyses use only the final poll from each firm, so there’s a large element of luck in the relative rankings.

    Comment by pete — May 8, 2014 @ 10:07 am

  3. Ta Danyl.

    The comparative dearth of polling in NZ was really contrasted with the squillion data points 538 was able to draw on last year. We really are under-served by commentators over egging their interpretation of a couple of nights on the landlines. And ever more so by the way they ignore syndicated results from the opposition – ignoring the data because someone else did it just makes commentators look ill informed.

    Public service announcement: if you wish to browse sites like NYT, WaPo, The Onion, Politico etc that limit free access to N articles per month then simply use the ‘incognito’ or ‘private browsing’ option within your browser of choice. Or use a different browser if you’ve expended free reads within your main one.
    Obviously the most efficacious solution is to actually pay for access you cheap rotter.

    Comment by Oh Busby — May 8, 2014 @ 10:08 am

  4. Peter, absolutely.

    Some of your own work shows how they were all out, and on the high side for National. One News Colmar Brunton was particularly bad (10% above election results) until the very last two polls.

    Comment by George — May 8, 2014 @ 10:28 am

  5. @George: Thanks, I’d forgotten I’d made that one!

    Comment by pete — May 8, 2014 @ 10:38 am

  6. Vernon Small et al on the Roy Morgan poll: “A Labour-Greens alliance has overtaken National in the polls for the first time”

    Am I misreading this? Labour-Green were ahead of National just two RM polls ago, and fairly regularly last year.

    Comment by pete — May 8, 2014 @ 11:05 am

  7. Didn’t mortgage rates go up in this period? Anyone ever graphed government popularity vs. mortgage rates? Just curious.

    Comment by Sanctuary — May 8, 2014 @ 11:08 am

  8. Morgan polls may bounce around, but that’s closer to the truth than the constant shouts of “National can govern alone!” from our TV screens in 2011. Every poll on TV 1 and 3 delivered the same message, and it was false. Or rather, every interpretation of every poll, from over-excited politcal editors.

    In short, Morgan isn’t on telly or the front page, therefore it’s written off by the axe-grinders. That has nothing to do with its predictive value.

    Comment by sammy 2.0 — May 8, 2014 @ 11:18 am

  9. George @1. Thought of your link as I read “John Armstrong: Red hot PM hits back” in the Herald. Wow so John Key is red hot in Question Time. My heading would have been: “Key evasive and trite.”

    Read somewhere today that Morgan contacts cellphone users which none of the others do.
    And their undecideds were 6% where some of the other polls have 10-20% undecided.

    Comment by xianmac — May 8, 2014 @ 11:28 am

  10. Had just posted on this at Polity when I saw your post. Something to keep in mind is that poll volatility can be due to a range of things – in addition to polling frequency and sample size.

    For example, I’ve found there’s more volatility in responses to attitudinal questions (eg, what do you think?) than concrete behavioural questions (eg, what do/did you do?). I consider ‘who would you vote for’ questions to be attitudinal – they are very sensitive to priming/questionnaire order effects.

    I’ve worked on continuous surveys (eg, 50 to 200 interviews per week), and also very regular point-in-time surveys (1,000 every four to six weeks), and it’s interesting how little volatility there *can* be in the data.

    Comment by Andrew — May 8, 2014 @ 11:45 am

  11. Hey Danyl.

    Which results did you aggregate, exactly? Just asking because this chart makes the Colmar Brunton poll look way better than I thought it would, and DigiPoll way worse than I thought it would.

    Appreciated the point at the end of your post. Pollsters put a huge amount of effort into trying to get things right, but immediately prior to an election they’re praying that the stars are aligned, and the Gods of chance are on their side. 🙂

    Comment by Andrew — May 8, 2014 @ 12:01 pm

  12. “John Armstrong: Red hot PM hits back”

    Armstrong always puts me in mind of a comment Danyl left once on No Minister, a comment whose pungent metaphor has remained with me to this day:

    “If Key were to wrestle [John Armstrong] to the ground and shit in his mouth, [John Armstrong] would be praising his rich, firm-textured stools to the sky.”

    Comment by Psycho Milt — May 8, 2014 @ 1:07 pm

  13. Pundits are like polls. Their contact with reality is judged by how closely they match ones views.

    Comment by NeiiM — May 8, 2014 @ 3:56 pm

  14. The poll was done pre Collinsgate and during Jonesgate and appalling headlines for Labour so not sure why but it does confirm the Roy Morgan rogue label. The methodology that Horizon use gives some lively and interesting results that can be hugely beneficial for the client writing the cheque and buying the headline.
    I think Labour have once again let Mallard off his leash and completely stuffed up the advantage they had with Collins stuff up, now the party that brought us Taito Phillip Field etc etc is getting unecessarily being thrown out of the house by trying to label arguably the most improbable looking bribe taker on earth Michael Woodhouse as being dodgy.

    Comment by David — May 8, 2014 @ 5:49 pm

  15. Only recently I was hearing David Parker in the morning talking about finance, now it’s all Mallard improbably trying to score points against National on political ethics.

    The most prominent rich immigrant trying to buy political influence is Maybe Labour and the Greens could set their sights on him rather than the chinese community. But then it’s politically more expedient for them not to.

    Comment by NeilM — May 8, 2014 @ 6:42 pm

  16. Roy Morgan actually rang me, I guess for this poll but I didnt stay on the phone long enough to find out- it was during a V8 Supercars race a couple of Saturdays ago. Hung up in the end on the pushy woman from the third world call centre.
    So its maybe 0.25% lower for National because of that, but it might explain a rise in the Green vote if that’s when they did their calling.

    Comment by Grant — May 8, 2014 @ 8:11 pm

  17. The poll was done pre Collinsgate and during Jonesgate and appalling headlines for Labour so not sure why but it does confirm the Roy Morgan rogue label. The methodology that Horizon use gives some lively and interesting results that can be hugely beneficial for the client writing the cheque and buying the headline.

    Horizon is not Roy Morgan. Conflating the two is just silly.

    So its maybe 0.25% lower for National because of that, but it might explain a rise in the Green vote if that’s when they did their calling.

    There were 847 people surveyed. Your non-response (assuming that another National-supporting respondent wasn’t surveyed in your place) shifts things 1/847 or 0.12%.

    Also, the survey was carried out between April 21 – May 4, 2014. So unless there’s pretty much continuous Supercars racing over those two weeks, you’ll need to think again.

    Comment by Flashing Light — May 8, 2014 @ 8:26 pm

  18. the survey was carried out between April 21 – May 4,

    Just as interest rates were going up and Parker announced their Reserve Bank policy.

    Comment by NeilM — May 9, 2014 @ 8:17 am

  19. Kim Dotcom has been a god send in regards to showing us what a sleazy pack of underhanded bull-shitter’s the national party and their coalition partners are .

    National and their supporters want him gone ( hi NeilM ) because he’s lifted the lid on a pack of worms which John Key presides over.

    I expect the ‘surveillance state’ to cost the Nats votes in the coming election …..

    Which along with all the other people they’ve pissed off, ignored or made poorer …… will see them voted out quite decisively.

    Which is a good thing, as “NZ Inc” and the ‘brighter future’ is a mean nasty place where Judith Collins is Saint Teresa.

    Comment by reason — May 10, 2014 @ 1:08 pm

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at

%d bloggers like this: