Mark Twain divided falsehoods into three categories: Lies, damned lies, and statistics. Had he lived to see them, Twain might have added a fourth: Public opinion polls.
Atlanta has two political polling firms that routinely tell us who is ahead this race or that, whether for mayor of Atlanta or governor.
Front-running candidates often use numbers generated by these companies to raise campaign cash. Journalists sometimes use the same statistics to help determine which candidates are worth their time.
One of these survey companies, Republican-oriented Strategic Visions LLC, has just finished a very bad week.
The firm was censured by a national organization of pollsters. And Nate Silver, a prominent political statistician, publicly suggested, in a series of posts on his much-read blog, fivethirtyeight.com, that Strategic Vision might be making up its results.
A clearly exhausted David Johnson, CEO of seven-year-old Strategic Vision, on Friday called the intimations libelous. “We categorically deny it. And yes, we are going to be taking legal action,” he said.
Johnson’s troubles are rooted in the New Hampshire presidential primary of 2008, when Democrat Hillary Clinton surprised one and all with a victory. Nearly every pollster had predicted a win for Barack Obama, the winner of the Iowa caucuses five days earlier.
The statistical debacle bothered the 2000-member American Association for Public Opinion Research, which is worried that more and more people in this math-challenged society are coming to view polls with Twain-like suspicion.
The 62-year-old organization – George Gallup was an early president – embarked on a year-long investigation, requesting internal methodology from 21 polling outfits that had operated in New Hampshire and other states with primaries.
In a noticed posted on its Web site last Wednesday, AAPOR announced that Strategic Vision was the only firm that “repeatedly refused to release essential facts.”
The AAPOR said it was not passing judgment on the quality of Strategic Vision’s work. “We’re saying we can’t know anything about quality if we don’t know what you did,” said Peter Miller, president of the organization.
Johnson, 42, said he isn’t a member of AAPOR and shouldn’t be subject to its policing. Miller said his organization has an obligation to point out murkiness in the polling industry wherever it is found.
The economics of political polling have changed a great deal in the last several years. Newspapers and other media outlets were once a major source of funding. This is no longer the case.
Some polling firms, including those in Atlanta – Strategic Vision and InsiderAdvantage – have adapted by giving polls away, or charging only nominal fees. They consider the surveys to be loss-leaders, and use them to attract paying clients.
In a world where you get what you pay for, quality has become a concern.
One measure of quality is the demographic breakdown behind every poll. We call them “crosstabs.” Those with the proper math skills can use them to help gauge a poll’s validity.
Whenever it has released a statewide poll in Georgia, Strategic Vision has kept the crosstabs to itself, as proprietary information. “Because we were trying to sell them,” Johnson said. Other firms do the same, he noted.
But not all. InsiderAdvantage CEO Matt Towery says he now releases background data with every survey – or to any news organization that asks.
“You have to. If you can’t show crosstabs, you can’t prove that you’ve got a poll,” said Towery, a former state lawmaker.
InsiderAdvantage has partnered with WSB-TV to poll the Atlanta mayor’s race. The background data provided by Towery has led some to criticize the surveys as distorted. The polls have given too much weight to young voters, critics say.
But even if it increases the second-guessing, the added transparency boosts public confidence in the results, Towery said.
Events of the last week have caused Strategic Vision to come to the same conclusion about its future polling.
“We’re going to release all the crosstabs, and put an end to this right now,” Johnson said. “That will squelch anybody from saying anything.”
For instant updates, follow me on Twitter.