This is why American Polls got it wrong again: During the 2020 election, we took particular care to track two set of polls, one that showed Biden ahead and the other that showed Trump ahead. We were unable to understand why the gap between various polls was so high, it did not make sense at all. Today, we show you, why the polls were so wrong this time as well. See this tweet on October 20th, 2020
To understand the problem, we do a very simple comparison between Polls that showed Trump ahead with those that showed Biden ahead and see if there are some trends. We analyse two battleground states - Pennsylvania and Michigan. We analyse 3 pollsters in total
Pennsylvania
We compare the Emerson Poll that showed Biden ahead by 4 with the Trafalgar Poll that showed Trump ahead by 2. Biden is currently leading by 1 point in the counting which means that both the polls missed the final result by about 3 points.
Key Findings
Emerson Polls follow a mixed mode data collection while Trafalgar used an IVR method. In terms of the age weights, each of the polls used different weights for those older than 64 year of age. It was 23% for Trafalgar and 25% for Emerson
Michigan
Biden is ahead by 2.7% at the moment in the counting. Trafalgar poll predicted that Trump will win by 2 points while an Insider advantage poll predicted that Biden will win by 2 points (Closer to the result)
The Findings
Interestingly, both Insider Advantage and Trafalgar reported very similar results in Pennsylvania and very different results in Michigan. Insider advantage also uses a mixed methodology for data collection, similar to Emerson. But there is a huge variance in the results amongst male voters with Trafalgar showing large leads while Mixed Method Pollsters showing lower leads. In terms of weights, in the Insider advantage poll, voters older than 64 made up for 21.8% of voters, in case of Trafalgar it was 23.1%
Consistent trends across the 3 pollsters are 1. Trafalgar (IVR Method) shows big leads amongst White voters and Men, 2. The Mixed Methodology Pollsters like Emerson and Insider Adavntage show much lower gaps amongst Men but it is not necessarily always producing accurate outcomes, 3. Age weights are not the same across all the pollsters
Looking at all the data, here are our major takeaways
Both these issues exist in every opinion poll around the world whether it is UK, Israel, India or even the recently concluded New Zealand Elections. It is therefore recommended that we prepare ourselves for wider margin of errors than stated by pollsters themselves. Summary numbers like the ones produced by RCP or Nate Silver are actually more reliable than individual polls.