Regardless of your political affiliation, the recent dust-up with the Fox News Poll on impeachment is a critical reminder that your research teams should be talking to you about how they manage accurate representation in any research you and your brand are running. In short, Fox News reported last Wednesday on a poll that indicated that 51% of Americans now support Trump's impeachment and removal from office. However, any researcher worth their salt would have noticed something fishy about these numbers - mainly that the reported split of party affiliation was way off-base. In the Fox News poll, respondents were represented by 40% Republicans, 48% Democrats, and 12% Independents. If you know anything about the political party breakdown in the US, you will notice that both Republicans and Democrats are significantly overrepresented here (Democrats even more so), with Independents significantly underrepresented.
Misrepresentation of Party Affiliation in Fox News Poll
This misrepresentation of party affiliation has a significant impact on the overall polling results, as was pointed out this weekend by the NY Post. At Flow Strategy, we've run the numbers ourselves, and if the polling data had been accurately reflective of the 29%/31%/38% party affiliation split we actually see in the U.S. according to Gallup, we would see 45.86% of Americans supporting Trump's impeachment and removal from office. Our numbers are slightly different than what was reported by the NY Post - we've reached out for comment and will update here when we hear back.
There are many takeaways here, but to summarize:
It's critical to ensure your data is representative of the audience you are seeking to research, whether you achieve that by implementing quotas or weighting.
It's irresponsible to report on data that misrepresents the audience. The 5 percentage point gap we see here between the captured results and the weighted results is HUGE. The inaccurate data was reported all over the place in the last week, even reaching SNL's Weekend Update. It is the responsibility of the research firm to accurately represent the U.S. in their polling, and they failed to do so.
Finally, this critical error has far-reaching implications beyond this one poll and this one news report. What's going to get lost in this reporting is the fact that this poll also indicates that support for impeachment is up significantly since July. But, trust in this poll (and potentially polling in general) is eroded when mistakes like these are uncovered.
Yes, this is a post about political polling, but it's also a reminder to you to challenge your research partners on how they ensure the research you are conducting is representative of your target audience. If you aren't researching the right people, how can you trust your research results?