With Super Tuesday so close, I chose readings this week that dealt with the importance of accurate math in poll stories. Stories involving surveys, exit polls and presidential preferences can be ripe with misinformation resulting either from carelessness, lack of knowledge about the nature of polling data, or plain manipulation.
In order to have an accurate story, a journalist must first start with accurate and respectable data. Sheldon Gawiser and G. Evans Witt detail how to determine if a poll is “scientific” in “20 Questions a Journalist Should Ask About Poll Results.” The major distinguishing factor between a scientific poll and an unscientific survey has to do with how respondents are chosen. If respondents volunteer, the poll is generally unscientific and has little value in determining true public opinion.
Logically, this makes a lot of sense. People that volunteer to spend part of their day answering questions could easily be considered more impassioned about the issue at hand than the general public. The problem is compounded when the respondents are not approached or called, an online survey being a prime example. Obviously, the respondents’ answers in cases like these would not represent the opinions of a larger group of people. If unscientific polls are constantly covered, the views of a fringe minority may mistakenly be interpreted as mainstream by readers.
Meeting this scientific requirement doesn’t happen as often as one might think. Barry Sussman demonstrates how even a widely respected poll has faults that leave it with little news value. In “How not to conduct a presidential poll,” Sussman describes The Des Moines Register’s poll of likely caucus-goers as possibly inaccurate. The newspaper reported unlikely predictions, yet did not feel the need to explain the conclusions made. The filter used to determine who would be a likely voter was not reported, indicating it was poor at best. What’s worse, the poll did not focus on real issues, intentionally leaving out Iraq, which is a major concern for several Democrats.
The results of such a poll therefore say little about what the electorate truly intends to do. However, reporting the poll’s results is not without consequence. Some voters are influenced by the numbers they see. If these numbers are wrong, and even if only a few votes change as a result, the disservice that such a newspaper provides can change the entire outcome of a very close election.
The third reading has to do with confidence intervals and margin of error. Robert Niles explains why one week’s poll should never be relied on. The common 95 percent confidence interval means that one out of every 20 times a poll is repeated, a result outside the margin of error occurs. This means that one week something crazy might happen in the polls, and the next week it’s as if it never happened.
We learned this in science class. When experimenting, you have to repeat your trials and take the average, removing the outliers. Most scientific research can’t be conducted in one week.
Reporters and readers alike often come to false conclusions about a change in percentage that is within a poll’s margin of error. Since the reporter is the one relaying the information, it is up to that reporter to take into account a poll’s margin of error when providing conclusions about poll data. At the very least, the poll’s margin of error should be reported so that an informed reader can accurately analyze the poll’s data.