Eight Key Questions to Ask About Polls

contributed by Deb Halpern Wenger
Associate professor, Virginia Commonwealth University and former assistant news director, WFLA, Tampa, Florida

  Who?

Were the right people asked? For example, if its a political survey, was the poll limited to registered voters?

  How many?

What is the sample size? In general, the larger the sample, the more confidence you can have in the results, assuming everything else was done properly.

  Margin of error?

Most public opinion research experts agree that the minimum acceptable margin of error is 5%, and thats still a wide swing. So when can you say one candidate is really ahead of another in a horse-race poll? When the gap between the two candidates is more than twice the error margin, you can say with confidence that Candidate A is leading Candidate B.

One other note: Be particularly careful when you report on subgroups within the sample. For example, a poll of 400 registered voters may have a margin of error of ±4%. But if you pull out the numbers for African Americans and there were only 40 in the survey, the margin of error becomes huge!

  Confidence interval?

Its complicated, but important. A 95% confidence interval is standard for a reliable survey. Even at this level it means that one time in 20 the polls results will not be reliable. For the other 19 times the estimates are within 5% of the truth.

  Sponsor of survey?

What polling firm, research house, political campaign, corporation or other group conducted the poll? If the person providing poll results cant or wont tell you who did it, serious questions must be raised about reliability and truthfulness.

  When?

Events have a dramatic impact on poll results. If the poll is taken on Tuesday and the candidate announces on Wednesday that he once smoked marijuana, the poll may no longer be valid on Thursday.

  How conducted?

Most surveys are now done by telephone. If its a mail survey, be aware that they take weeks to get results, so the information is not as timely. And forget about surveys done in shopping malls, restaurants or on the street. They should never be treated as if they represent a public opinion poll.

  Questions asked and in what order?

Find out the exact wording of the poll questions and evaluate it. Do the questions seem fair and unbiased to you? And on sensitive questions (such as abortion) consider including the complete wording in your story. The question order is also important: e.g., in tough economic times, if people are asked what they think of the economy before they are asked their opinion of the president, the presidential popularity rating will probably be artificially lowered.

For more information on what to ask about polls and why, check out Public Agenda Online at www.publicagenda.org.