Editor’s Note: For another view, see Counterpoint: Polling Isn’t the Problem
Perhaps the only thing to which the press and politicians pay more attention than fundraising numbers is polling results for either their campaign or public policy opinion. So, a fair question to ask in today’s politically crowded and noisy arena is can the polls be trusted?
The answer is yes, but the devil is in the details.
Polling itself is a rather straightforward math equation that relies on some less-than-complicated statistical work. It has been proven time and time again to provide reliable results, but not all polls are created equal. Yes, pollsters can get it wrong, but it is normally in how they weight the polls more so than their survey method or sampling.
Put more simply, pollsters can alter the results of a poll to fit what they think turnout demographics in an election will be. Let’s take 2016 as a prime example of when pollsters did get it wrong. Almost every poll showed Donald Trump down, and in my home state of New Hampshire many polls showed Trump losing by eight points when, in fact, he went on to lose by barely one point.
So how does this happen? How did so many people get it wrong?
Simply put, pollsters were using the Obama-era turnout numbers to unintentionally skew the polls in favor of a larger Democrat turnout. This was not some grand conspiracy but rather a simple oversight of a shifting electorate. Remember, it is in a pollster’s best interest to be as predictive of the outcome as possible, as most polling is for-profit and very expensive. For a pollster to have clients, they must be trusted; and to be trusted, they must have a good and accurate product.
This is why polling methodology has evolved from simple automated landline surveys to a complex assortment of online, IVR, cell phone, and live land line sampling. As people find new different ways to communicate, so will pollsters have to find new ways to contact them to ask questions. As we continue to add new tools to our kit, we will continue to get better; but there will always be some problems with polling.
One of the most common misconceptions is that polling portrays a perfect result. This is unfortunately impossible, as polling only a sample of the whole population creates a margin of error. Margin of error is the leeway a poll has for its results. For most polls it is anywhere from 3 percent to 5 percent. This is a big deal because a 5 percent range in a heads-up race can mask the Election Day result by a wide margin. Polls with higher sample sizes can narrow it down to a plus or minus 3 percent, but this is still a significant range, considering many elections’ margin of victory. Sadly, too often the press reports only the topline results and never mentions the margin of error.
This understandably leads to distrust and a dismissal of polls that do not fit our personal narrative or worldview. As they say: “There are lies, and then there are statistics.” It would not be wise, however, to disregard polls completely. They do serve a legitimate purpose even though some may push them to promote their own agenda.
I have always looked at polls as tools for seeing the big picture. Polling is incredibly good at doing that. While pollsters cannot perfectly predict the future and will never be able to due to the limitations of how we collect and process data we are still able to give deep insight to what is happening to our clients and the public. There is no better way to truly grasp what is going than to ask people, which is exactly what polling does. If you want to know what is going on in your town you may talk to your neighbor while a pollster will talk to 400 of your neighbors.
Finally, there is a way for the public to take a shortcut in trusting polling, which is to look at the aggregates of polling. There are many websites out there that do this, but the most notable is 538 by Nate Silver. What is so great about this is when you take the aggregate of many polls you in effect boost the sampling size and reduce the impact of errors made by individual pollsters.
While bad polls and potential outliers can skew the results the good will almost always outweigh the bad and polls should be trusted more often than naught.
