Sandra Susan Smith is the Daniel and Florence Guggenheim Professor of Criminal Justice, the Carol K. Pforzheimer Professor at the Radcliffe Institute, the director of the Malcolm Wiener Center for Social Policy, and the faculty director of the Program in Criminal Justice Policy and Management (PCJ). In 2023, Smith announced a new research initiative at the Malcolm Wiener Center that would offer grants to faculty and students to support the development of qualitative research projects related to social policy issues. Qualitative research can add necessary context and explain social phenomena using data such as in-depth interviews, fieldnotes, focus groups, and questionnaires. The 2023 grant recipients were announced on Monday.
Q: Why is qualitative research important for addressing social policy questions?
Put simply, qualitative research allows us to answer a different set of questions than quantitative research, questions that are as important to the policy project even if undervalued in the current policy context. The field’s elevation of “big data” and randomized control trials (or equivalents) as gold standards of social policy research privileges a rather narrow set of questions and concerns (read efficiency and effectiveness), and this has effectively pushed other equally important questions critical to our understanding of policy fields to the margins of those fields. Good social policy, however, requires a deep understanding of the stakeholders—of who they are, what beliefs they hold, what motivates them, what resources they can marshal, and toward what ends. It requires an understanding of the historical context within which a problem or set of issues has emerged and how that context shapes the contemporary moment. It necessitates an understanding of the set of potential risks and uncertainties that might inform processes and outcomes. It demands an understanding of the processes by which programs or policies are implemented and what this means for outcomes. And it requires an understanding of what constitutes success. The list goes on. Arguably, all of these questions are best addressed with different types of qualitative data—including but not limited to fieldnotes based on ethnographic observations, in-depth interviews, surveys and questionnaires, focus groups, and archival/historical documents.
But qualitative research is important for addressing social policy questions for another reason. As others have argued brilliantly, “big data” research requires qualitative research to produce high quality results—to understand decision making by data producers and users, to understand the ins and outs of algorithms, to develop stronger theories, to distinguish real and important patterns from noise, and to better understand and evaluate the decisions that researchers make about the analytic approaches they take. Each of these is critical to doing quantitative research that produces findings about which we can have confidence. As such, qualitative research methods should be a critical component of training to become public policy analysts.
Q: Are there any misconceptions about qualitative work?
There are many. One is that qualitative research is basically descriptive, not analytical, and more akin to journalism than social science. This is simply not the case. There is certainly great value in descriptive research, and while qualitative research can certainly be descriptive—just as quantitative research can—it also enables us to explain relationships and identify mechanisms. In other words, through qualitative research, we can answer important questions about the hows and whys of social phenomena, in some ways perhaps better than quantitative research. Indeed, qualitative research is often the source of our strongest theories in the social sciences.
Another misconception is that qualitative research lacks rigor; it is easy to do, requiring little or no training, education, or skills. This is also false. As with quantitative research, doing qualitative research in a way that provides compelling insights about which we can have confidence requires years of training in data collection, management, and analysis. Unfortunately, however, social policy programs rarely if ever adequately invest in the types of courses that would support developing these important skillsets, despite the importance of qualitatively oriented research questions to the project of sound social policy development. Over the past 2-3 years, HKS has made real strides in this area, but much work remains to get our training programs where they need to be.
A third misconception is that one cannot generalize from qualitative research; findings are relevant only to the specific case or study but not beyond. This is also not true. While most qualitative studies do not allow researchers to make statistical inferences, logical inferences, rooted in patterns observed in the data are firmly in the realm of possibility with qualitative research and can generate good quality hypotheses for testing with appropriate data.
Q: Why did you decide to start a grant program to support this type of research and how do you think it will benefit both students and faculty?
I started this initiative for three reasons. First, my sense is that there are categories of questions that we are either failing to address or giving short shrift to because of our rather narrow focus in the policy arena on questions of efficiency and effectiveness. The small grant program is an effort to shift our focus to other questions that should be centered as well because they are vitally important.
Second, in conversations with a number of colleagues who identify as quantitative researchers, I have learned that they would appreciate having opportunities to expand their approach to include qualitative research because doing so would allow them to address other important questions and to improve the quality of their quantitative data collection and analysis processes. However, incentive structures do not often exist to support such expansions. This initiative is an effort to change the opportunity structure in some small way. I am excited that a number of this year’s awardees, including all three faculty awardees, identify as quantitative researchers and have exploited this opportunity to realize their expansive visions. For example, Daniel Schneider’s project on surveillance and automation in the service sector will now include in-depth interviews with service workers to contextualize and inform findings from their analysis of quantitative data.
Finally, with this initiative I want to encourage public policy graduate students to consider more seriously the merits of qualitative research. Almost certainly, more will be needed to shift graduate students’ orientations, especially graduate students from disciplines that traditionally do not deploy such methods. And, with the help of my colleagues, I will consider other approaches as well. But this is a solid start.