The current version (vol. 10, issue 1) of the newsletter of the APSA Section on Qualitative Methods and Multi-Methods Research includes a symposium on John Gerring’s 2nd edition of Social Science Methodology (SSM2, for short). The various contributors to the symposium deal with different aspects of SSM2 such as concept formation (Ariel Ahram) or the compatibility of set theory with the potential outcomes framework (by Carsten Schneider and me, available on SSRN). Some of the contributors provided very strong criticism of Gerring’s book. What I want to focus on here is Patrick T. Jackson’s discussion of Gerring’s take on philosophy of science. To a degree, I have to admit that it was a fun read for me. Jackson starts out by saying that he wholeheartedly agrees with the last paragraph of the book, but disagrees with most of what is written on all the previous pages. Why is that?
At the risk of oversimplification, Gerring argues that social science researchers should be able to communicate with each other. Communication is hampered by the pursuit of different ontologies and understandings of science (Gerring frequently speaks of methodology, but I think he mostly means ontology and sometimes epistemology). According to Gerring, this implies that the adoption of one methodology and an understanding of science is necessary for effective communication, and this should be the (neo-)positivist take on science. Science then should be “systematic, rigorous, evidence-based, falsifiable, replicable, generalizable, non-subjective, transparent, skeptical, rational, frequently causal, and cumulative” (p. 2). Jackson correctly asserts that, for one reason or another, many social scientists do not subscribe to this view and are likely to be offended at calling these the characteristics of true science. He criticizes Gerring’s plea for unity from various sides: the definition of causal mechanisms; the reference to falsifiability, an idea that one usually shies away; and the need for unity in the first place, just to mention three points.
However, there is a more fundamental problem with Gerring’s plea for unity that is not addressed by Jackson. As Gerring notes (and what is not criticized by Jackson), ontological disputes cannot be settled because they are beyond the reach of empirical analysis (see alsoHay, Colin (2006): Political Ontology. Goodin, Robert E. and Charles Tilly (eds.): The Oxford Handbook of Contextual Political Analysis. Oxford: Oxford University Press: 78-96). This is what Gerring identifies as a problem because exchange among scholars is hampered by different ontological and, as a consequence, methodological positions. That’s all fine, but Gerring then draws an unsustainable conclusion from this: that all researchers should follow one ontology and that this should be his pet ontology of the neo-positivist sort. I think the problem with this claim is apparent: if there is no right and wrong ontology and we cannot adjudicate between them, how do we justify all researchers settling on one of these ontologies? Regardless of whether one likes Gerring’s view of science and ontology or not (Jackson clearly is on the do-not-like side), one should eschew claiming that one has “found” an ontology that everyone should follow. For the same reason, there is little to be gained by criticizing Gerring’s view on science (or anyone’s view on science) by censuring it on substantive ground. One might disagree with Gerring’s conception of science and ontology, but these criticisms are not any more right or wrong than Gerring’s position itself.
Having said that, we are still left with Gerring’s correct diagnosis that the pursuit of different ontologies hampers communication (also partly because many scholars treat ontology as a matter that is decidable, therefore trying to convince the “disbelievers” of the “truth” that they own). What then is the basis for judging empirical insights gathered on the basis of a specific ontology? In my view, the answer is consistency, which has two elements. First, one’s personal take on ontology, methodology, and epistemology should form a coherent whole (see Petter Hall’s (2003: Aligning Ontology and Methodology in Comparative Research. Mahoney, James and Dietrich Rueschemeyer (eds.): Comparative Historical Analysis in the Social Sciences. Cambridge: Cambridge University Press: 373-404) intriguing piece on aligning ontology and methodology). Second, each of the three “-ologies” should be internally consistent. The rationale for consistency as the guiding principle is that if we cannot judge the adequacy of an ontology (and the other –ologies), we should focus on the extent to which a researcher does justice to his/her ontology and follows a methodology and epistemology that is consistent with it.
Here is a simple example of what consistency means. When you run a regression, you automatically make the assumption that the cases are causally homogenous (the assumption can be right or wrong, but it is the default assumption with which one starts). Now, suppose that you do post-regression process tracing in order to learn something about the underlying causal mechanisms. One sometimes finds the opinion that the generalization of insights derived from process tracing should follow the idea of contingent generalization (Nome, Martin Austvoll (2008): Getting the Pathway Case Off the Drawing Board. APSA Qualitative Methods Newsletter 6 (1): 20-23). Contingent generalization means that causal inferences are only generalized to cases that share certain conditions. For instance, when the regression analysis includes all countries in the world, contingent generalization would mean that process tracing insights gathered in a European country are only generalized to other countries from Europe. This practice is inconsistent because it is in discord with the causal homogeneity assumption embodied in the regression analysis. One either has to limit the regression analysis to European countries, or go for a more sweeping generalization of the process tracing inferences. One can think of more severe consequences of inconsistency than this one, but it illustrates the importance of thinking coherently when linking ontology, methodology, and epistemology to each other.
I acknowledge that it can be painful to take ontologies for granted, ontologies with which one wholeheartedly disagrees, and to limit oneself to an assessment of their internal coherence and consistency with methodology and epistemology. But ontology wars based on the flat rejection of any ontology other than the one we prefer yield little benefit. At the least, the engagement with consistency hopefully ensures that researchers produce the best research given their ontological commitments.