Leaving Theory Behind: Why Hypothesis Testing Has Become Bad for IR
John J. Mearsheimer, University of Chicago
For most of its history, the field of international relations has been rooted in theory: Theorists came up with big explanatory concepts that empiricists set out to test. When events, new information, or careful scholarship showed that cherished notions were wrong or incomplete, new theories were devised to take their place.
However, over the past decade or so, the traditional balance between the generation and testing of theories has shifted to the point where international relations is dominated by narrow "hypothesis testing." This style of research generally involves looking for statistical evidence of a causal relationship, following an approach that has only nominal ties to theory. Kennedy School Professor Stephen Walt and his University of Chicago colleague John Mearsheimer are calling attention to this trend, which they consider worrisome, in their forthcoming paper "Leaving Theory Behind."
Theory remains essential for making sense of a complex world, the authors assert, and diminishing its role is bad for international relations. New theories revolutionize our thinking, generate useful predictions, and guide policy choices and policy evaluation. Attention to theory is also critical for conducting valid empirical tests, whether one is relying on quantitative data or case studies.
Yet contemporary scholars often pay little regard to theory, say Walt and Mearsheimer, devoting most of their effort instead to "collecting data and testing empirical hypotheses." The latter exercise involves picking a phenomenon such as war — the so-called dependent variable — and seeing how it may be affected by different independent variables. A growing proportion of published articles employ quantitative methods tailored to this pursuit, while job postings in political science increasingly seek candidates with methodological expertise rather than a background in theory.
The premise underlying this shift, Walt speculates, is "the assumption that truth lies in the data, and we can understand how international politics works from the bottom up by collecting lots of data, analyzing it rigorously, and seeing which variables affect other variables." The problem is that this approach rarely works, "because if you haven’t thought carefully about the theoretical underpinnings, your data analysis or case studies are likely to be flawed."
Theory is essential for offering a big-picture perspective that more-limited inquiries cannot provide. Although Walt and Mearsheimer emphasize that testing hypotheses is a necessary part of social science, they maintain that "theories?.?.?.?help us interpret what we observe and tie different hypotheses together, making them more than just a piecemeal collection of findings."
The trend toward hypothesis testing may be on the upswing partly because of the greater availability of data and more-powerful computational techniques. Another factor, according to Walt and Mearsheimer, is that graduate programs are moving students through the system more quickly. Developing and refining theory is a time-consuming task that may not pan out, since the requisite creative spark may never occur. "By contrast, almost anyone with modest mathematical abilities can be taught the basic techniques of hypothesis testing and produce competent research," the authors contend. A further consequence is that papers in international relations are becoming more technical and virtually impenetrable to policymakers and others outside the discipline.
Reversing this trend — the "privileging" of empirical analysis at the expense of theory — will not be easy. The goal is not to try to force everyone to become a theorist, notes Walt, since no one knows how to train people to be creative and invent original theories. "But we can teach students to have a greater appreciation for theory," he adds, "by showing them how the neglect of theory can lead to unreliable and unconvincing results."
This change, which will require altering the criteria used to evaluate scholarly work and rank individual scholars, cannot be mandated by a few individuals or a few academic departments. Instead, "the field as a whole has to decide to reward a different style of work," Walt believes. "It’s possible that shift could occur, but it is sure to be a slow and laborious process."
— by Steve Nadis