Under the rubric of “Reader’s view”, I will post short reviews of books, mostly from the field of social science methods. The first post deals with Goertz and Mahoney’s (GM) A Tale of Two Cultures (Goertz, Gary and James Mahoney (2012): A Tale of Two Cultures: Contrasting Qualitative and Quantitative Paradigms. Princeton: Princeton University Press.) The following review is part of a contribution to a review symposium on multiple methods books. The symposium is forthcoming in the newsletter of the APSA Section on Qualitative Methods and Multi-method Research.
Two Cultures is an impressive extension of GM’s widely well-received Political Analysis article from 2006. The book extends the number of dimensions on which the quantitative and qualitative cultures are compared and discusses each dimension in much detail. It does not take much to argue that their book will leave a significant imprint on the field, and this rightly so because GM offer the first comprehensive and comparative discussion of quantitative and qualitative methods. However, I have a few concerns about GM’s message in Two Cultures and the meaning that readers might get from it. With regard to the latter point, one should understand that Two Cultures concerns the way in which GM perceive the common practice of qualitative and quantitative research. GM are clear about this, but it is worth emphasizing because the way qualitative and quantitative research can be done is much broader than the two allegedly existing cultures.
In order to understand why I say “allegedly existing cultures,” I have to elaborate my preferred view of Two Cultures: it is a descriptive empirical study of how social science methods are practiced. GM distinguish multiple dimensions on which one can describe an empirical study such as the breadth of concepts (thin vs. thick), the nature of utilized evidence and so on. They then develop the overarching hypothesis that all these dimensions load on a single qualitative-quantitative dimension, i.e., one dimension suffices to sort how social science methods are practiced. Given the diversity of ways in which quantitative and qualitative research can be practiced, this is an interesting and strong hypothesis. Unfortunately, Two Cultures stops at the formulation of the hypothesis and confines itself to the presentation of illustrative evidence. Although Two Cultures offers many insightful arguments and presents illuminating examples, the claim that a single dimension characterizes the field of social science methods remains untested. In terms of our knowledge about the landscape of social science methods, at the end of Two Cultures, we are more or less at the point where we started.
The next step one should take, either GM or anyone else, is to review selected parts of the empirical literature, gather the required data, and test whether the dimensions distinguished by GM indeed load on one dimension (the appendix in Two Cultures operates on a more abstract level and does not achieve this). Absent such a project, my hunch is that Two Cultures will be a mind-reinforcing book, but not a mind-changer. Readers who shared the perception of social science methods with GM will still do so after having read Two Cultures. People who already disagreed with the message of the Two Cultures article will also continue to do so and point to illustrative examples that are in discord with GM.
The absence of an empirical analysis of the methodological landscape also makes it difficult, in my eyes at least, to engage with the book on a substantive level. GM admit that their distinction of two cultures is very broad and that there are subcultures within each culture. For every argument that one makes against one of the claims in Two Cultures, proponents of the two-cultures argument can point to a subculture that does not exactly fit with the root culture, but nevertheless belongs to it.
Nevertheless, I have an opportunity here: I find it difficult to follow the subculture argument in every respect. For instance, it would be interesting to know whether researchers applying Bayesian statistics are really happy with being dubbed a subculture of frequentist research that GM mainly have in mind when talking about the quantitative culture. Similarly, I have serious concerns with subsuming QCA under the qualitative culture. QCA certainly shares some characteristics with the latter, e.g., the focus on set relations. However, if one takes QCA as a method as opposed to an approach and examines hundreds or thousands of cases , it is likely to share characteristics with the quantitative culture such as reliance on thin concepts. The fact that QCA as a method does not fit squarely is one reason for my believing that the large-n vs. small-n dimension (n being the number of cases) is also needed for properly mapping social science methods. Unless a thorough review of the practice of qualitative and quantitative research is done, we neither know if my hunch is true, nor whether there is a qualitative vs. quantitative dimension à la Two Cultures in the first place.