Over the next weeks, I will give a chapter-by-chapter review of the new book Multi-Method Social Science: Combining Qualitative and Quantitative Tools by Jason Seawright. To my knowledge, it is the first book on nested analysis/multimethod research in the social sciences that is not an edited volume. A debate about mixed methods has been going on in adjacent disciplines for decades, but the political science development of nested analysis is largely running in parallel to the mixed-methods debate (and vice versa). Because it is the first book and there are many interesting and challenging arguments in it, I will briefly review each chapter on its own terms. (Maybe I will discuss some chapters in a single post; we will see because I have not finished reading the book yet.)
In the opening chapter, Seawright distinguishes between triangulation of qualitative and quantitative methods on the one hand, and the integration of both methods on the other. He opposes the idea of triangulating qualitative and quantitative methods for making a single causal inference and answering the same research question because both methods. In contrast, the integration of qualitative and quantitative methods allows one to draw on their complementary strengths and to use one method to check the assumptions underlying the other method (having read chapter 3, it looks like this is a one-way street where the qualitative part validates regression assumptions).
This sounds convincing, but it should not come as a big surprise for those who are familiar with nested analysis. The key distinction underlying nested analysis is one between a causal effect and a causal mechanism. The inference about a causal effect is different from an inference about a mechanism, meaning that triangulation as Seawright defines it is notpossible. It is true that the term ‘triangulation’ is used here and there in the multimethod literature, for example by Lieberman in his seminal APSR article (p. 440). However, it is not meant in the way Seawright defines triangulation. The idea of triangulation plays a bigger role in the mixed-methods literature, which is, as I said, largely detached from the nested analysis literature. We do not know, but I assume that researchers who might feel challenged by Seawright’s main claim are not among his readership, and that the political science readers of Multi-Method Social Science need not be convinced that integration is central to nested analysis.
Substantively, I think the issue is that the key selling point of multimethod research has been made in Lieberman’s pathbreaking article. This is not to say that there is no room for methodological work on nested analysis because some elements still have to be addressed and refined. However, the one reason why integrated research is valuable, the combined analysis of causal effects and mechanisms, can only be made once and has been made already. I should add that the formulation of a punchline is probably more difficult and less needed for methods books than substantive books because the value of methods books lies in taking one through the details of a method.
Stylistically, it is not obvious who is targeted with the rejection of triangulated causal inference because there is no recent reference cited in this context (1995 is the most reference that is given). This reinforces my impression that the central claim is endorsed by everyone because nobody in the nested analysis community argues for triangulation and against integration. More generally, my reading of chapter 1 (and the following chapters) is that it is a little bit too light on the referencing side. In some cases, a reference jumps to mind that is not cited, including Lieberman’s article at several places, and in some cases the literature is misrepresented (like the chain of references supposedly saying that multimethod research is not superior to single-method research, p. 2). I find this unfortunate because it limits the value of the first chapter for empirical researchers who want to use it as a guide to existing discussions of multimethod research and further readings (the chapters do not have a “Further Readings” section at the end).
Stay tuned for chapter 2 on the causal foundations of multimethod analysis.