Qualitative Comparative Analysis (QCA) is the method of choice for the analysis of set relations and has changed considerably and improved over the years. The more one delves into the method, however, the more things you (I, at least) stumble upon that seem a little bit curious, sometimes also more than a bit (just as with any method, I guess). One issue I came across relates to the calculation of consistency scores in fuzzy-set QCA for assigning outcome values to truth table rows.
I used data from a study about parliamentary control of EU affairs, but the specific study and data are not important because it can happen with any data. I noticed that there is a truth table row, row 6 by the automatic counting of the QCAGUI package for R, which only has the Czech Republic as a member. The consistency value of the row is 0.78 and could be included in the minimization process because it is above the conventional minimum level of 0.75. The author of the study imposes a higher threshold than 0.78, but this is not relevant because it only matters that it legitimately could have been taken as consistent with the claim that the configuration is sufficient.
Now, when you take a look at the XY-plot for this truth table row, you see that the Czech Republic is a contradictory case in a qualitative view (or a “true logical contradiction”, as Schneider/Wagemann call it). This means the consistency level would allow you to include the row in the minimization process, but the only member of the configuration is not a member of the outcome. In other words, the row would be taken as sufficient, but you could not point to or study a single case representing the sufficient relationship.
The intricacies of consistency values in fuzzy-set QCA are well-known and are repeatedly a subject of debate (like here). To a lesser degree, there is also awareness of the fact that one should watch out for contradictions in fuzzy-set QCA. In principle, it then should not come as a surprise that a sufficiently high consistency value masks a contradiction, but I hadn’t linked those two issues until I had a look at this plot.
The plot conveys two points. First, it should become standard to look at the XY-plots of all truth table rows having consistency values above the preferred threshold. Second, in this example, the row should not be included in the minimization process because the consistency value is only driven upward by cases that are not members of the truth table row. In other instances, it will be less clear cut and, say, three cases are members of the row and consistent and one case is a contradiction. What do you do then? In a qualitative, crisp-set view, this would be a consistency value of 0.75 and be minimally acceptable.
I have no definite advice on how to proceed in such situations, but a two-fold standard would be one way to go: the fuzzy-set consistency value should be above a given threshold and the crisp-set consistency value that only effectively relates the members of the row to each other should be above a selected threshold. If anybody else has other ideas on how to address this problem, I would be happy to hear about them.