The Dim-Post

September 30, 2010

Travels in hyper-reality

Filed under: blogging,Politics,polls — danylmc @ 5:55 am

All those media stories about the Christchurch mayoralty race insisting that Bob Parker has surged ahead of Jim Anderton in the polls will become more credible now that there has been an actual poll conducted in Christchurch:

A UMR Research Poll released today shows 55 per cent of decided voters now intend to vote for the incumbent Bob Parker, up 27 per cent since June.

However:

The survey was done online and questioned only a small sample of 361 Christchurch residents so had a relatively high margin of error of 5.2 per cent.

DPF is a professional pollster and he wrote about online polls a few days ago:

In some circumstances they can be very useful – especially when surveying the opinions of a discrete group.

But generally they are unreliable when it comes to being a fair sample of all New Zealanders. Because they are only representing those NZers who have joined that online panel. And even with weighting, this does not mean it is representative. A weighted sample can still be unrepresentative.

But my view is that at this stage in NZ, they are not a reliable indicator of New Zealand public opinion. I have seen many online panel polls like the above, which produce results massively different from those produced by phone polls. Normally I ignore them, but as some media reported this one, I thought it is worth making the point.

Naturally he blogged the results of this poll because it’s a poor result for a politician he doesn’t like but I think his earlier comments are useful.

About these ads

11 Comments »

  1. Easy there, dpf’s comments were made in relation to online panels rather than polls per se…
    And there you go, I’ve just gone and defended dpf twice in one week – let the conspiracies begin.

    Comment by Sam — September 30, 2010 @ 6:03 am

  2. To be fair, the poll that originally had Anderton way out in the lead was also a UMR online job.

    They’re both based on this panel:

    http://www.umr.co.nz/sayit/

    Where the pitch is:

    “Give your opinions on the topical issues of the day by taking part in SAYit online surveys and go into the draw to win exciting prizes.”

    Hmmm.

    Comment by Russell Brown — September 30, 2010 @ 7:05 am

  3. From the panel on the UMR sayit site:

    Russell of Auckland won a $100 Prezzy card for participating in the August online omni survey

    Congratulations!

    Comment by danylmc — September 30, 2010 @ 7:18 am

  4. Isn’t releasing a poll result while people are voting kind of weird?

    Comment by lyndon — September 30, 2010 @ 8:47 am

  5. Uh – sorry – missed that this was an online panel poll also. Whew…

    Comment by Sam — September 30, 2010 @ 8:51 am

  6. Isn’t releasing a poll result while people are voting kind of weird?

    No. Just in New Zealand. Mostly because for general elections, such a poll is illegal. Exit polls are pretty common elsewhere.

    Comment by Graeme Edgeler — September 30, 2010 @ 9:06 am

  7. I always thought that the NZBCSD surveys are highly suspect for this reason – “A weighted sample can still be unrepresentative.”

    My impression of who answers the NZBCSD polls are that it is all people like me who feel really strongly about most of the issues they poll on. And I am definitely not representative of the average NZer on climate change for example…

    Comment by LucyJH — September 30, 2010 @ 12:07 pm

  8. The NZBCSD polls have hugely bias questions too, particularly around things like “how enthusiastic are you to spend lots of taxpayers money to prop up scheme ‘X’ just so as to reduce carbon?” Really; really really; hugely; or somewhat?

    I try to give the most outrageous reactionary response to compensate but I am but one person.

    Comment by insider — September 30, 2010 @ 12:46 pm

  9. Russell B. Thank you for shedding some very necessary light on this topic.
    The Natrad mouthpieces keep raising it.

    There is only one poll that counts………..

    Comment by peterlepaysan — September 30, 2010 @ 6:44 pm

  10. “All those media stories” arose from the fact that people were prepared to put money on it:

    https://www.ipredict.co.nz/app.php?do=contract_detail&contract=MAYOR.PARKER

    But you can still tell yourself that an old-fashioned poll, where there’s nothing at stake for the respondents, is “more credible”.

    Comment by Miguel Sanchez — September 30, 2010 @ 9:36 pm

  11. I was one of New Zealand’s first market researchers (geez I am THAT old!) and I think I can claim to be one of the pioneers of the discipline here although I have long since left the industry.

    Online polling is in no way accurate as a way of measuring any opinion on anything. Back in the days before the public rightly got fed up with researchers bothering them at dinner time and told them to fuck off over the phone, we used to go door-to-door on a statistcally structured sample basis and whats more we used to do callbacks if the respondents weren’t available to ensure statistical viability as far as was possible. It was tedious, but it was an accurate way of sampling and it worked. I doubt if anybody does this now because frankly the public wouldn’t put up with it in today’s environment and the wages costs would be prohibitive.

    By comparion, an online poll can never claim to be representative of the population’s overall opinion because it only represents (and I summarise):

    – “people who are interested enough to click the button, out of a sample of readers of that particular website (and no other) who have just read an article in probably in emotive or biased language presenting one set of facts and given an incomplete set of choices as answers”.

    This is not research. Mainly it is entertainment or a flight of fancy, but it is not a valid way of researching anything.

    HOWEVER… in some circumstances you could use online polling to measure the DIFFERENCES or CHANGES in opinion over a period of time, as long as you structured the question(s) the same each time and provided the same choices to click. I would think this is also true of online panels – it is the differences and changes which are the most revealing rather than taking the results as absolutes.

    Comment by Dave Mann — October 1, 2010 @ 1:58 pm


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Rubric Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 347 other followers

%d bloggers like this: