QCA is developing… apart?

At the end of last week, a two-day conference, Qualitative Comparative Analysis – Social Science Applications and Methodological Challenges, took place in Tilburg, the Netherlands. Needless to say, the recent and ongoing wave of criticism of QCA was a key topic on the agenda and in discussions among participants. Unfortunately, some central figures in the field of QCA could not make it to Tilburg, including Axel Marx, Charles Ragin, Carsten Schneider and Claudius Wagemann. Moreover, none of the critics of QCA were participating, which was not surprising because most are not based in Europe and would have had to travel a long distance.

In my perception, the conference showed what one could observe over the last two years or so: the QCA community at large is building sub-communities that differ in their views of what QCA is, how it should be studied and what QCA is good for. The key dimension seems to be the one distinguishing QCA as an approach from QCA as a method (see chap. 1 in this book). QCA as an approach refers to the original understanding of QCA as a case-based method for which close case knowledge is not only an asset, but constitutive. QCA as a method focuses more specifically on the algorithm that is used for the processing of data and how it performs in the face of challenges such as limited diversity. QCA as an approach implies QCA as a method, but not the other way around because one can run an algorithm on data without knowing any of the cases in detail.

Implicitly, the distinction already played out in the symposium on Lucas and Szatrowski’s “critical perspective” on QCA and centers on the question of whether simulations with hypothetical data are useful for evaluating the performance of an algorithm. Ragin and Olsen say “no” in their contributions to the symposium because you necessarily lack case knowledge when using hypothetical data and refuse to accept insights derived from simulations. Fiss, Marx and Rihoux, and Vaisey hold a different view because they run simulations themselves.

This is only one issue, although quite salient at present, and it is important because where one stands on the simulation-question determines what one thinks the best response is. If someone believes simulations are useless, they must argue against them, probably reaching an impasse quite quickly because neither side is likely to suddenly adopt the view of the other side. QCA researchers embracing the idea of simulations (like me) have to engage with the existing ones and might devise some themselves, committing an instrument that is disparaged by other QCA scholars.

It is not necessarily bad that QCA is building sub-communities and is probably unavoidable. It is a sign that the community has grown because the method spreads across disciplines and generations of social scientists; young researchers who had a different academic training from more senior ones enter the field of QCA and introduce new perspectives. Although QCA is often pitted against regression analysis, a somewhat closer look at the quantitative community shows that it too hosts many sub-communities having different opinions on the frequentism vs Bayesianism debate, on the sense and procedure of null-hypothesis testing, etc. Similarly, there are multiple camps in the case studies and process tracing domain that share a belief in the general value of the method, but disagree on many issues, such as case selection and the understanding of what a mechanism is. In this view, the development of circles within the broader QCA domain can be taken as a good sign: QCA has achieved a certain level of maturity and diffused across disciplines and generations, with the normal side effect of the community becoming more diverse. And QCA researchers value diversity, don’t they?


About ingorohlfing

I am Professor for Methods of Comparative Political Research at the Cologne Center for Comparative Politics at the University of Cologne (http://cccp.uni-koeln.de). My research interests are social science methods with an emphasis on case studies, multi-method research, and philosophy of science concerned with causation and causal inference. Substantively, I am working on party competition and parties as organizations.
This entry was posted in QCA, qualitative, set theory and tagged , , , , . Bookmark the permalink.

3 Responses to QCA is developing… apart?

  1. Alrik Thiem says:

    Hi Ingo, having myself been present at Tilburg, I generally share your impressions. If the community is “developing apart”, it only means that science is taking its course. Old ideas get challenged, new ones are born, and many will not survive the test of time. Every researcher in this community should thus welcome these signs as indications of progress.

    However, where there can be no doubt is that any method that claims to be able to draw causal inferences, and QCA clearly is such a method, must be subjected to minimal tests using simulated data that allow researchers to evaluate whether the method is working as it should. Of course, these tests must be properly designed and implemented, and almost all of the recent anti-QCA pieces have been wanting in this respect. But if someone carried out an adequate test with simulated data demonstrating that QCA does not work, then no attempt at immunization by rejecting simulations could invalidate the conclusion that QCA was unfit for real-life data analysis. And since QCA as a “technique” is embedded in QCA as an “approach”, the latter would be affected just as much.

    • ingorohlfing says:

      We are in absolute agreement. The “QCA as an approach” vs “QCA as a method” (i.e., the truth table analysis) reasoning once struck me as useful, but I am not convinced anymore because some now pit the aproach against the method. The problem lies with the argument that elements of the approach (case orientation) can compensate for problems with the method. I do not see how this can work in principle and do not know of an empirical example for this claim.

  2. Alrik Thiem says:

    Regardless of the way it is portrayed, either relation between QCA as an “approach” and a “technique” has self-defeating implications. The argument that QCA as an approach is distinct from QCA as a technique leaves the question as to what “the approach” then is other than some diffuse notion of case knowledge. And the argument that case knowledge can compensate for problems with the method raises the question as to what the method’s actual purpose then is at all. Stephen Vaisey (2014: 110) has summed it up nicely in his symposium reply to Lucas & Szatrowski (2014): “Knowledge of cases is ideal, but not necessary, for using QCA.”

    * Lucas, Samuel R., and Alisa Szatrowski. 2014. “Qualitative Comparative Analysis in Critical Perspective.” Sociological Methodology 44 (1):1-79.
    * Vaisey, Stephen. 2014. “Comment: QCA Works—When Used with Care.” Sociological Methodology 44 (1):108-12.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.