Are Pollsters Wearing Blinders?

In the late 1970's, when the first home computers arrived, I would have loved to have heard what the typewriter salesmen were saying? Probably something like, "Oh those things are just toys. They're just a gimmick, a passing fad, they'll never last. What you need is this brand new $800 typewriter here ...."

About 3 years ago I was getting a film developed from my old Nikon camera. I went into London Drugs, which has a very popular photofinishing service in Western Canada. I asked the clerk if she was concerned about the growing emergence of digital cameras. "No", she exclaimed, "they're just a fad. Film cameras will always offer better quality and resolution."

In the past few years the sales of CDs in music stores has plummeted, with the majority of music buffs purchasing and sharing their music online. The same is now happening with videos. I was recently in a Blockbuster in Chicago. It was an empty, depressing place. I don't have the heart to ask people working in these stores if they're concerned about their future employment.

We can laugh about this now but how clearly blind some people are to huge disruptive changes, often involving one technology replacing another. And those most invested in using a certain technology or doing things a certain way are often the most blind of all.

As many of you know, I've invested a great deal of my time, money, and energy the past 2 years in developing Pocket Pollster, a brand new piece of software that I hope will be very disruptive to many traditional data collection processes. Initially it will operate only on a Pocket PC but eventually it'll work on cel phones and other devices as well.

One of the obvious uses for this product is in the realm of political polling. In the U.S. it seems a perfect fit for "exit polling". But I also believe it has a great applicability in general polling "out on the street".

A professional acquaintance of mine kindly put me in touch with a contact of his, a longtime Canadian pollster. My goal was just to demonstrate the software to her, even offering to give her several free copies as thanks for testing it.

When asked about its uses I made reference to my belief that it may very well be a good replacement for telephone polling, which clearly will be (or should be!) extinct. My product may or may not end up as its replacement but I was astonished to see her defend telephone polling as she did. Much like the typewriter salesman of old, her business has been deeply tied to telephone polling and she just can't let herself see beyond its deep flaws.

One has to ask how telephone polling can possibly represent a random sample of voters when:

• 80%+ of people refuse to answer the phone and agree to be polled
• A growing number of people only have a cel phone, which pollsters are not allowed to call

In the last federal Canadian election the weekly poll results from different companies were wildly different and well beyond the realm of +/- X%, 19 times out of 20 that is often cited. By definition, if two different polling companies ask the same question (e.g. Who are you going to vote for?) to two different groups that represent a random sample of the same overall group then the results should be identical within the margin of error. When they consistently aren't then the most likely culprit is that the sample is not random and thus not representative. The fact that journalists don't challenge the pollsters about these discrepancies is shocking. But perhaps not so much once you realize that the media companies that employ these people often hire these polling firms, which supply information that are used to fill air time. It's thus best not to shoot the news provider (the pollsters) if they're providing raw material that keeps you employed.

Getting back to my situation, what really irked me is how this pollster attacked me, citing the old "I have 20 years of polling experience behind me and you don't." Whenever someone gets super defensive by citing their academic or experience credentials rather than arguing the points at hand, you know you're wasting your time about having an intelligent discussion.

This could most acutely be observed with the Dan Rather memo scandal in the last U.S. election. It was very clear that the memos about George Bush had been forged but Rather and his gang at CBS tried to pull the "who the hell are you bloggers in your pajamas, we are the real journalists" nonsense. Fast forward a few years and you can see many employed by the mainstream media quaking in their boots about the tectonic shift that has already occurred. Many people no longer get the majority of their news from old media sources like TV, radio, and newspapers. They still may use them as one source, but no longer trust them as the only source. This democratization of where we get our information from is a huge advancement for mankind but those with blinders on are frightfully scared by it.

During the next election in your area do closely follow the polls, especially those that come from different polling firms in a relatively short time span. If you see the numbers from 2 polls outside their margin of error then remember what I said here today!