A new report from the Civic Health and Institutions Project (CHIP50), a research collaborative supported by the National Science Foundation, offers insights into the aftermath of the recent presidential debate, specifically looking at voter preferences before and after the debate.
Matthew A. Baum, the Marvin Kalb Professor of Global Communications at Harvard Kennedy School, serves on the project along with colleagues from Northeastern University, University of Rochester, Rutgers University, and Harvard Medical School.
The years-long project surveys all 50 states and Washington, D.C., to collect representative samples across the country to understand public policy preferences.
As this report, the 105th from this project to date, noted, the recent presidential debate produced “a modest churn of voters’ preferences” with no real shift in the race between the two candidates. HKS spoke with Baum to understand the significance of the findings.
Q: What sets your work apart from the polling in the media since the debate?
I think our work adds a bit of perspective to the prevailing media narrative of a post-debate collapse in public support for Biden.
It felt to our team that, to some extent, after the debate a narrative developed—Biden is too old to run based on his debate performance—then journalists went out and found data that supported that narrative, using pre- and post-debate polling, sometimes extrapolating beyond what the data reliably showed.
There are various problems with comparing pre- and post-debate polls. One of the most important, which is what we really emphasized, is the fact that they involve different samples of people who may differ in important ways.
For instance, you might have more Democrats or more Republicans in one of the surveys. You may also, have had Biden supporters who were demoralized by watching his debate performance and just didn’t want to answer surveys about it for a few days, causing “phantom shifts”: a failure to control for systematic biases in non-responses that can produce seeming shifts in poll results because of changes in sample composition even if there is no change in individual voting intention.
Our survey was in the field in June for the ongoing CHIP50 work. A subset of participants in our April/May survey chose to participate again in June, giving us a convenience panel of respondents. The debate fell soon after our April/May survey, so we compared candidate choice before and after the June 27 debate using the same individuals. And while this is not a randomly generated panel, it did allow us to compare attitudes among one group of people prior to and following the debate.
Q: What did your research show?
When we looked at the same people, we ended up with over 1,200 repeat respondents. We saw a little bit of churn, but it adds up to really small changes from before to after the debate. Our data showed 94% of those who chose Biden and 86% of those who chose Trump in the May survey still preferred the same candidate after the debate.
I don’t want to characterize this as definitive, but comparing attitudes of the same people before and after the debate has some advantages over comparing results from two different samples of people. The reporting dominant in the media has drawn heavily on the latter sorts of comparisons. I think it’s important to add some context to the prevailing media narrative, which has at times tended to overstep what the data can tell us with confidence.
It takes a big change to register a difference in those comparative cross-sections the way some have been trying to do. The actual observed changes are in many cases just not big enough to detect or distinguish from random noise, which is something that all the media organizations should recognize.
“Our data showed 94% of those who chose Biden and 86% of those who chose Trump in the May survey still preferred the same candidate after the debate.”
Our surveys allow more direct comparisons over time. We can also look at responses from April/May and June, prior to the debate and find some shifts in voter preference. But any number of factors might account for those changes. Events in the Ukraine war? Events in the Israel-Gaza war? Changes in inflation? There are lots of things that could account for changes in support for Trump and Biden having nothing to do with the debate.
In fact, we don’t really see anything different when we compare the change over time among people we polled in April/May who we also polled during the pre-debate part of June, versus the change over time among people we polled in April/May who we polled again during the post-debate period (after June 27). This suggests that the debate doesn’t seem to have shaken up the race much.
Q: How does this report fit into your overall CHIP50 project?
We started polling in the immediate aftermath of COVID-19. It was called the COVID States Project. We were able to survey Americans in all 50 states and the District of Columbia every four to six weeks about all things related to COVID, both directly and indirectly, including social, economic, cultural, and political issues that arose during the pandemic.
Since COVID, we redefined the project under the CHIP50 banner. It has a broader mandate now. We have a portion of the survey that we reserve for the issues we think are important, such as attitudes on the Israeli-Gaza war, mental health among young adults, and of course the impending election. We also accept proposals from other researchers on a variety of topics. These could include public health, politics, social policies, and others.
Moving forward, we are going to do some projects related to social networking—how people’s social network affect attitudes and behavior. Then after that, we’re probably going to be focused on the presidential election.
—
Banner Image: People watch the debate between President Joe Biden and presumptive Republican nominee former President Donald Trump on June 27, 2024 in Chicago, Illinois. Photo by Scott Olson/Getty Images