Don’t call it causal when it is an observation

In the case study literature, one new pair of terms has been established by the field that concerns the distinction between data set observations and causal process observations. In short, Collier, Brady, and Seawright (2004: Sources of Leverage in Causal Inference: Toward an Alternative View of Methodology. Brady, Henry E. and David Collier (eds.): Rethinking Social Inquiry. Diverse Tools, Shared Standards. Lanham: Rowman & Littlefield: 229-266) propose the following distinction: data set observations (DSOs) are comparable observations allowing one to apply quantitative methods. In an analysis of democratic peace, for example, data on the democratic quality of a country and on the state of foreign relations in terms of peace and war are comparable (hopefully, the data is comparable). DSOs for multiple pairs of democracies – democratic dyads – then allow one to determine whether democratic dyads are always at peace with each other.

Causal process observations (CPOs) are non-comparable observations related to the link between cause and outcome. In process tracing on democratic peace, one could select a case in which two democracies experienced a crisis with the potential for war. In a first step, one might find that the crisis leads to the mobilization of the military in both countries. In the second step, mobilization of the military leads to public protests against war and for peace. In step three, the protests lead to a reconsideration of the situation within the respective governments, etc. It is apparent that CPOs are not comparable within a case, but that they are valuable for explaining why two democracies do not fight each other (a pro-peace public would be the answer here). The idea behind the distinction between DSOs and CPOs is significant because it clarifies the different evidential basis for causal inference in cross-case and within-case analyses.

I also use this terminology in my book because the terms are established in the field and I did not consider it necessary to make the field more complex by inventing yet another pair of terms. With some distance, I now think that speaking of DSOs and CPOs is misleading in multiple respects and that there are better terms for expressing what they stand for.

Of course, there has been extensive debate about the term CPO in particular. Neal Beck (2006: Is Causal-Process Observation an Oxymoron? Political Analysis 14 (3): 347-352) considers causal process observation to be an oxymoron, which partly overlaps with my new view on the terminology.[1] It should be clear though, that I, in contrast to Beck, do not dispute the value of process tracing and qualitative methods for causal inference. My concerns are conceptual and terminological and are based on the risk that people may overstate what can be achieved with process tracing. Here are my points:

1. Process tracing serves to collect process-related evidence that is used for the generation of causal inferences.  The observations that we gather, for example from interviews or primary sources, are nothing but observations. Whether these observations permit the inference that the empirical process is a causal process is a separate matter (and one too complicated to be addressed in more detail here). For the democratic peace example, this means that process tracing would involve the reconstruction of the process leading from the crisis to the peaceful settlement of the process. At this stage, a case study researcher only knows what happened. Whether this is a causal process with a causal mechanism involved has to be assessed in a second step. When one believes that public protest is the causal mechanism, one would have to assess whether the absence of public protest in response to the mobilization of the military would have paved the way for war between the two democracies.Observing protest in process tracing thus does not convey anything causal at all.

The point now is that calling the occurrence of protest a causal process observation presumes that the process is causal before one has considered whether the absence of protest would have made a difference. My concern therefore is that case study researchers might believe that the fine-grained reconstruction of an empirical process is already about causal processes and suffices for causal inference. It does not (see e.g., Machamer, Peter (2004): Activities and Causation: The Metaphysics and Epistemology of Mechanisms. International Studies in the Philosophy of Science 18 (1): 27-39). One term that strikes me as superior at this time is simply process observation (see also below).

2. The problem of conflating evidence with inference also becomes apparent in Blatter and Haverland’s case study book. In short, they distinguish between causal process tracing and causal-process tracing. In the spirit of Heidegger, the hyphen makes a big difference here. I can live with the first term because it denotes that process tracing is interested in causation and causal inference as opposed to description. The term causal-process tracing is based on the same mistake as causal process observation because it presumes that what one is tracing is a causal process. If one understands process tracing as the reconstruction of an empirical process (which is my view on process tracing), one cannot know whether the empirical process one is tracing is indeed causal at the time of process tracing.

3. The term data set observation is misleading because (causal) process observations can also be arranged in a data set. Take programs like Nvivo, Atlas.ti and MAXQDA that allow you to create data sets based on interviews, videos, newspaper articles, etc. The defining feature of a data set therefore is not related to the comparability of data constituting the data set.

4. Conceptually, thus, the terms data set observations and causal process observations are not on the same level. The antonym of data set observation would be something like non-data set observation, which does not make sense in light of the previous point. A suitable antonym of causal process observation would be causal effect observation, but the term causal effect observation would run into the same problems as causal process observation.

For these reasons, I think that the best pair of terms simply is cross-case observation vs. within-case observation. This pair of terms is on the same level and denotes quite clearly the level on which an observation is located. Since cross-case observations are about inferring associations (“causal effects”) and within-case observations about causal processes and mechanisms, the two terms also highlight the part of the causal relationship in which one is interested (effect vs. mechanism).


[1] See also

Brady, Henry E., David Collier and Jason Seawright (2006): Toward a Pluralistic Vision of Methodology. Political Analysis 14 (3): 353-368.

Beck, Nathaniel (2010): Causal Process “Observation”: Oxymoron or (Fine) Old Wine. Political Analysis 18 (4): 499-505.

Collier, David, Henry E. Brady and Jason Seawright (2010): Outdated Views of Qualitative Methods: Time to Move On. Political Analysis 18 (4): 506-513.

Advertisements

About ingorohlfing

I am Professor for Methods of Comparative Political Research at the Cologne Center for Comparative Politics at the University of Cologne (http://cccp.uni-koeln.de). My research interests are social science methods with an emphasis on case studies, multi-method research, and philosophy of science concerned with causation and causal inference. Substantively, I am working on party competition and parties as organizations.
This entry was posted in case study, causal inference, causal mechanism, evidence, qualitative and tagged , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s