July 22, 2020
The trend among most pollsters has always been to use a sample of all residents or registered voters early on in the election campaign and switch to likely voters as the election nears.
The reason for this protocol was explained to me by an early leader in modern campaign polling, Bill Hamilton, when we both taught at the University of Florida.
He explained that voters need some time to absorb the effects of the campaign and the competing candidates. Using a likely vote screen early on, leads many voters to fib about their likelihood of voting in an election still months away. And besides, he would say. “it’s cheaper.”
As we get closer to the November election, almost all polling firms will be utilizing likely voter screens.
But in the polling and campaign business, there are some who believe that the method of interviewing registered voters rather than likely voters early on, is an intentional method to under-report Trump’s real support in the polls.
This argument suggests that the more liberal public polling firms, by calling only registered voters is an intentional effort to increase Biden’s poll lead.
John McLaughlin, well known Republican pollster, commented how many pollsters are skewing the results by using registered voter samples.
“The refusal to screen for actual likely voters is creating an under-polling of Republicans and therefore Trump voters. It seems intentional. It’s exactly what the media did in 2016. Let’s prove them wrong again.” ( Elections. Polls & Surveys. Press Releases, June 8, 2020)
For me, this argument begs the question, what are the polling differences between a registered voter sample and a so-called likely voter one.
The idea that Republican voters are more likely to vote in elections and Democrats less likely, has been asserted for many years. The argument goes like this, Republicans are better educated, wealthier and older. These are attributes that contribute to voting.
In 2010, the website FiveThityEight concluded that registered voters surveys favored Democrats by 5% over the actual election results and likely polls favored Republican candidates by 1%.
Oddly, the academic studies have not focused directly on this issue. It is true, that the opinion of pollsters and campaign consultants are all in agreement that more likely voters identify with the Republican Party and, of course, likely to choose a Republican candidate over a Democrat.
To address this issue on whether likely voter polls do favor Trump and registered voters Biden, I devised a way to test these assertions statistically by comparing two series of recent national surveys conducted by public polling firms.
The first group includes 22 polls of registered voters only. The second group includes 22 likely voter polls only. Both survey groups were completed in June and July with similar end dates. The polls asked the identical question of choice for president.
When comparing Trump’s percent only in both registered and likely voter surveys, the results showed a small difference between the two groups. The chart shows how likely voters tended to support Trump more than the registered voter surveys.
As you can see the per survey differences weren’t substantial, with a mean difference of only 1.73%, in Trump’s favor.
|mean 41.5||mean 39.7|
But that’s only half of the story. This table and graph only shows the changes in Trump’s percentage without Biden. When you look at the percent difference between Biden and Trump in each poll, it reveals a completely different picture. After all, Trump isn’t running against himself (no comments, please).
The best way to measure the differences between two candidates is subtracting one percentage from another, creating a net difference variable. In this example, I’ve subtracted Trump’s results from Biden’s. The table below shows the net percentage difference between Biden and Trump by whether the sample was of registered (RV) or likely voters (LV). Positive numbers indicate a net percentage gain for Biden.
|RV % Difference||LV % Difference|
|RV mean = 9.2||LV mean = 7.6|
All the differences are positive except one likely voter survey with a minus 4%, which means that Trump had 4% lead over Biden in that one survey only. As the means for the two different modes show, Biden’s lead among registered voters is 9.2% and a 7.6% for the likely voter poll. In other words, Trump performs better in the likely voter sample by an average difference of 1.6%.
But is this modest difference real or just a random variation. To determine whether the differences are not random, I used a one sample T-test, which compares the mean of your sample data to a known value. In this case, is the mean of registered voters versus likely voters?
Without boring most of you with statistical data, the answer is yes, and the differences are statistically different at a P value of <.0001.
The individual differences between individual polls can sometimes be substantially larger than others. By comparing the results of two separate sets of polls, the differences are averaged out.
Notice that in this example, there are some polls showing Biden leading by 12% or even 14%. But when you combine those outliers with other polls, we average out those differences. (Remember the wisdom of the crowd.)
An example of that is an ABC News / Washington post poll released last Sunday. To their credit they used both registered and likely voters in the same survey. The registered voters had Biden at 55% and Trump at 40%, a 15 point differences. In the likely voter sample, Biden had 54% and Trump 44%, a 10% difference. The pattern is the same, Trump does better when the poll uses a likely voter sample, but the difference is based on only one survey.
This analysis confirms that Trump on average, does do better in likely voter polls than with traditional registered voter samples. It’s not at the level of the recent ABC / Washington Post Poll showing a 10% difference. This is not the case when we average a series of public polls in the same time period, using identical sampling methods.
With most recent public polls showing substantial leads for Biden, an average difference of 1.6% won’t help much. With almost four months and two Presidential debates to go, this campaign is just getting started. Stay tuned and stay safe…