The Margin of Error and What it Really Means

August 18, 2020

Caution: Some Math Involved

The most misunderstood statistic in politics is the “Margin of Error.” As most people know, this statistic is based on the size of the survey sample and not the size of the population. It doesn’t matter, for instance, that you are polling voters only in Fort Lauderdale or the entire United States. The sampling error for each stays the same no matter the population size.

To illustrate how to understand the margin of error, let’s say Candidate A in our poll has 48% of the vote and the margin of error for this survey is +/-3%. That means the range for Candidate A’s percent of the vote is within the margin of error, when it is between 45% and 51%. What that tells us, is that if you ran that poll 100 times, the outcome for Candidate A would be within that range, with a confidence level of 95%.

But this does not tell you if a candidate’s lead over the opponent is outside the margin of error, which would indicate his lead is greater than we would expect from sampling error. (I told you that math was involved).

If we want to know if Candidate A’s lead over Candidate B is outside the margin of error (for a two person horse-race), we have to assign the same 3-point error to candidate B as well. That means the 3-percentage margin for each candidate now becomes a +/- 6-point margin of error for the difference.

We could reasonably expect their true position to lie somewhere between –1 and +11 percentage points. In other words, for a candidate’s lead to be outside the margin of error, it must be greater than 6 points. The larger margin of error is due to the fact that if the Republican share is too high by chance, it follows that the Democratic share is likely too low, and vice versa.

In this example, for Candidate A’s lead to be outside the margin of error, it must be greater than 6 points (2 x 3%). Candidate A has 48% and Candidate B has 43%, a five point lead which is not greater than 6%, so we cannot be certain that this difference is not due to sampling error. Easy, right?

Normally you would have that explained by the reporters covering the polls, but unfortunately most don’t understand it. Many just report the single candidate margin and assume it includes the candidate’s lead as well. It doesn’t.

A lead that is inside the margin of error does not necessarily mean it is not correct. It only means we can’t be sure that it is not caused by sampling error. Polling was never meant to be a precise measuring instrument. That would require interviewing every single voter. It is an estimate within certain parameters based on probability theory.

Occasionally polls predict the exact the outcome. When that happens, pollsters pat themselves on the back and tout their great survey techniques, when in most cases it’s just dumb luck (random). I know, I’ve done it… Be safe.

By Jim Kane

Jim Kane is a pollster and media advisor, and was for fifteen years an Adjunct Professor of Political Science at the University of Florida. Kane is founder of the polling firm USAPoll and served as the Director of the Florida Voter Poll. His political clients have included both Republican and Democratic candidates, including the Republican Party of Florida, and both the Sun-Sentinel and Orlando Sentinel newspapers. At the University of Florida, Professor Kane taught graduate level courses in political science on Survey Research, Lobbying and Special Interest Groups in America, Political Campaigning, and Political Behavior. In addition to his professional and academic career, Jim Kane has been actively involved in local and state policy decisions. He was elected to the Broward County Soil and Water Conservation Board (1978-1982) and the Port Everglades Authority (1988-1994). Kane also served as an appointed member of the Broward County Planning Council (1995-2003), Broward County Management Review Committee (Chair, 1990-1991), Broward County Consumer Protection Board (1976-1982), and the Broward County School Board Consultants Review Committee (1986-1990).

Leave a comment

Your email address will not be published. Required fields are marked *