August 18, 2020
Caution: Some Math Involved
The most misunderstood statistic in politics is the “Margin of Error.” As most people know, this statistic is based on the size of the survey sample and not the size of the population. It doesn’t matter, for instance, that you are polling voters only in Fort Lauderdale or the entire United States. The sampling error for each stays the same no matter the population size.
To illustrate how to understand the margin of error, let’s say Candidate A in our poll has 48% of the vote and the margin of error for this survey is +/-3%. That means the range for Candidate A’s percent of the vote is within the margin of error, when it is between 45% and 51%. What that tells us, is that if you ran that poll 100 times, the outcome for Candidate A would be within that range, with a confidence level of 95%.
But this does not tell you if a candidate’s lead over the opponent is outside the margin of error, which would indicate his lead is greater than we would expect from sampling error. (I told you that math was involved).
If we want to know if Candidate A’s lead over Candidate B is outside the margin of error (for a two person horse-race), we have to assign the same 3-point error to candidate B as well. That means the 3-percentage margin for each candidate now becomes a +/- 6-point margin of error for the difference.
We could reasonably expect their true position to lie somewhere between –1 and +11 percentage points. In other words, for a candidate’s lead to be outside the margin of error, it must be greater than 6 points. The larger margin of error is due to the fact that if the Republican share is too high by chance, it follows that the Democratic share is likely too low, and vice versa.
In this example, for Candidate A’s lead to be outside the margin of error, it must be greater than 6 points (2 x 3%). Candidate A has 48% and Candidate B has 43%, a five point lead which is not greater than 6%, so we cannot be certain that this difference is not due to sampling error. Easy, right?
Normally you would have that explained by the reporters covering the polls, but unfortunately most don’t understand it. Many just report the single candidate margin and assume it includes the candidate’s lead as well. It doesn’t.
A lead that is inside the margin of error does not necessarily mean it is not correct. It only means we can’t be sure that it is not caused by sampling error. Polling was never meant to be a precise measuring instrument. That would require interviewing every single voter. It is an estimate within certain parameters based on probability theory.
Occasionally polls predict the exact the outcome. When that happens, pollsters pat themselves on the back and tout their great survey techniques, when in most cases it’s just dumb luck (random). I know, I’ve done it… Be safe.
