Social science is aimed at helping us understand the reality surrounding us so that we can improve it. Different fields of study have adopted a systems approach (Kast & Rosenzweig, 1972), which has become one of the predominant paradigms. A basic principle of Systems Theory is equifinality which generates the need to identify the various configurations of the system elements which lead to a desired state or outcome, while allowing understanding of the critical factors or necessary conditions explaining the presence or absence of that outcome or state.

However, social science, in an attempt to emulate its seniors, the pure sciences, has adopted increasingly sophisticated and powerful methodologies and algorithms. These are geared toward establishing and quantifying causal relationships between the variables, and underestimate the importance of analyzing the complex interactions produced in social science which allow for several alternatives to achieve the same end.

In recent years, the adoption of qualitative comparative analysis (QCA), which uses Boolean logic, has been growing, substituting traditional correlation methods to establish causal conditions related to a particular result (Ragin & Fiss, 2008; Ragin, 1987; Ragin, 2000; Ragin, 2008; Vis, 2012; Woodside, 2016). Aside from its application to case studies, QCA currently focuses on analysis of empirical data to generalize the analysis, taking account of possible replication in subsequent studies, and constructing logical propositions following the qualitative study of the phenomenon in question (Ragin, 1987; Ragin, 2000; Woodside & Zhang, 2012). This method is effectively and parsimoniously able to address the challenges described in the previous paragraph, and offers results which complement and enrich the state of the art.

This article provides a detailed description of the characteristics of published research that employs QCA. Specifically, the study aims to identify quantitative differences among its variants, to provide a global perspective of the state, scope, and impact of published research. The objective is to spread within the academic community, an analytical method that is able to complement and extend research programs. To achieve this, we provide a bibliometric analysis of the research contributions to date.

Based on this analysis, the present article provides information on the following aspects: (1) the number of studies per research field; (2) the number of articles published per year; (3) the number of articles that use each variant of QCA and the number of articles of each QCA variant published in JCR journals; (4) citations per field and average number of citations; most cited QCA articles; (5) the journals publishing the highest number of articles using QCA. Accompanying a description of the data, we explain the circumstances which possibly led to this evolution.

The study is structured as follows: ‘Qualitative Comparative Analysis’ section reviews the evolution of QCA; ‘Bibliometrics’ section provides a bibliometric analysis of the use, application, main authors, and impact of QCA; ‘Conclusions’ section discusses the main conclusions and suggests future directions of research on QCA.

QCA (Ragin, 1987; Ragin, 2000) is an analytic technique which combines quantitative and qualitative methodologies. The technique originally focused on small samples but further development has allowed its application to broader contexts.

QCA was developed from Mill's (1843) canons, to establish causal relationships through systematic comparisons. Particularly relevant are the methods of agreement and difference. According to the method of agreement, if two or more examples of the phenomenon under investigation have just one circumstance in common, the circumstance common to these examples is the cause or effect of the given phenomenon (Mill, 1843). According to the method of difference, if there is a circumstance when the phenomenon under investigation occurs, and another when it does not occur, this difference is the cause or effect or a necessary part of the phenomenon. Mill (1843) combined these methods in what came to be known as the “method of agreement and difference”. This method, although less consistent than previous methods, advanced application of these theories to real circumstances, and laid the foundation for the development of QCA (Vassinen, 2012).

Some of the disadvantages of Mill's (1843) principles are: (1) The complexity involved in identifying a common difference or circumstance, and (2) that this difference should be the determining cause of the phenomenon. In other words, Mill's canons do not allow for multicausality of the phenomenon. Furthermore, the method of the agreement is applicable only to phenomena with positive results.

Applications of QCA were initially focused on case studies. The drawback to these methods is the impossibility of generalizing the results to other similar cases. However, currently, in addition to being based on case studies QCA focuses on analysis of empirical data for the generalization of analyses taking into account possible replication in subsequent studies, and constructing logical propositions as a result of the qualitative study of the phenomenon in question (Ragin, 1987; Ragin, 2000; Woodside & Zhang, 2012).

Ragin (2008), which is founded in configurational analysis, allows comparison between cases, and at the same time, offers a detailed understanding of the complexity of each case, particularly in small or medium sized samples (Rioux & Ragin, 2009). Although historically QCA has been used for small- or medium-N, Emmenegger, Schraff, and Walter (2014) maintain that the method is applicable to research using survey data. To avoid the disadvantages of such samples, the authors recommend analyzing the strength of the results and propose a technique to achieve this. The most common procedure to ensure reliability of the results is the simultaneous implementation of the entire process by two researchers working together using the same data and selections. For their part, Fiss, Sharapov, and Cronqvist (2013) propose integrating the regression analysis framework with QCA to avoid eliminating possible drawbacks such as the omission of variables.

QCA uses Boolean logic instead of the traditional correlation methods to establish causal conditions strongly related to a particular outcome (Ragin & Fiss, 2008; Ragin, 1987, 2000, 2008). The basis of QCA's configurational approach is the analysis of sufficient and necessary causes to produce an outcome.

A condition is necessary if it is present in all instances of the outcome. One condition will suffice if a particular outcome emerges whenever the condition is present (Ragin, 2008; Schneider & Wagemann, 2012). However, there may be other conditions which lead to the same result, in other words, there may be multiple sufficient causes (Ragin, 2008).

There are two key concepts related to QCA: consistency and coverage. Consistency refers to the percentage of causal configurations of similar composition which result in the same outcome value. If the consistency of a configuration is low, it is not supported by empirical evidence. Therefore, it should be considered less relevant than other configurations with higher consistency. Coverage refers to the number of cases for which a configuration is valid. Unlike consistency, the fact that a configuration coverage is low does not imply less relevance. In cases where a result occurs through multiple causal configurations a single configuration can have low coverage but nevertheless be useful to explain a set which causes a particular outcome (Ragin, 1987, 2000; Woodside & Zhang, 2012). To organize the different causal configurations which lead to a particular outcome, truth tables are used (Ragin, 2009; Fiss, 2011). Every row in the table represents a configuration of the conditions that produce a particular outcome. The number of rows in a truth table is defined by 2k, where k is the number of causal conditions. Frequency thresholds can be set to determine the conditions that are part of the research and those that are not, according to their relevance, as we discuss later.

The two most common variants of QCA are presented below: fuzzy-set (fsQCA) and crisp-set (csQCA).

The concept of bibliometrics, or bibliometric analysis, was introduced by Alan Pritchard in 1969. However, the study of bibliography in a particular field dates back to the 19th century (Osareh, 1996). The field has grown exponentially since the advent of the Internet which facilitated communication among researchers around the world, and has enabled faster access to the contributions in a given area.

The present analysis collects data from publications on a given subject to identify factors of interest. These include authors or most frequently cited authors, scientific journals with the highest number of publications in the focal subject area, and the evolution of the subject area in terms of number of publications. Bibliometric analysis can also be focused on a particular journal to identify which subjects are most frequent, which authors publish most frequently in a certain journal, and which are the most cited articles (e.g., Merigó, Mas-Tur, Roig-Tierno, & Ribeiro-Soriano, 2015). Applications of bibliometric analysis vary as much as the factors that are analyzed with the result that the method has been applied in multiple subject areas including corporate social responsibility (de Bakker, Groenewegen, & Den Hond, 2005), international scientific cooperation (Glänzel, Schubert, & Czerwon, 1999), knowledge management (Gu, 2004), and medicine (Soteriades & Falagas, 2006). However, the analysis of topics or subject areas is not the only goal of bibliometric analysis. This study uses the technique to explore the relevance of an analytic method to identify cause and effect relationships: QCA. We exploit information from the Compasss database.

We found 4691 articles on the Compasss website which includes the QCA methodology in its three variants (fsQCA, csQCA, and mvQCA). COMPASSS (COMPArative Methods for Systematic cross-caSe analySis) is a global network which brings together researchers and professionals who share the same interest in theoretic, methodological, and practical advances related to a systematic comparative research approach to the use of configurational logic, the existence of multiple causality, and the importance of defining universes carefully. The project began in 2003, and its management was reorganized in 2008 and 2012 to accommodate the growing needs in the field.

The website's main objective is to advance development of systematic comparative analysis as an original and consolidated strategy to study diverse phenomena. It prioritizes the development and application of methods based on comparative configuration and theory sets (csQCA, mvQCA, fsQCA, and other related methods and techniques).

The platform provides a space for dialog and fruitful confrontations between qualitative (case-oriented) and quantitative (based on variables) methods, and a stage for the integration of these methods to progress in the development and application of approaches, methods, and original techniques, all aimed at developing a comprehensive approach to systematic comparative analysis of sets.

Alongside organization of this international resource platform, the main group in COMPASSS arranges seminars, manages a data file, publishes research, and participates in the development of software for the application of this methodology.

The objective is to make the platform a place of exchange among researchers from a variety of disciplines in the social sciences and psychology, among others.

Focusing on the bibliometric analysis, of the 469 articles based on the QCA methodology, 236 use csQCA, 222 use fsQCA, and 11 use mvQCA.

It is particularly striking that the articles based on fsQCA were mostly published between 2010 and 2015 while work exploiting the csQCA methodology started in 1991 and most was published in the 1990s.

In relation to the area of study, Compasss distinguishes the following applications: in urban and regional studies (10), in government and administration (29), in sociology (72), in business and economy (85), in environmental science (19), in education (17), in legal studies and criminology (11), in evaluation (11), in research and health policies (32), in management and organization (37), in comparative politics (100), in international relations (23), plus some applications in other fields (see Fig. 1).

Fig. 1.

Number of articles in accordance with the area of study.

(0.14MB).

Fig. 1 shows the huge differences in the number of articles using QCA in different areas of study. Comparative politics accounts for the largest number, followed by business and economics, and sociology. Currently, 54% of articles using the QCA methodology are published in these three areas. Other areas such as environmental science and education tend to prefer other methods. The popularity of QCA in comparative politics is logical since the method was designed primarily for this discipline. In areas such as education and legal studies, qualitative methods are common since these disciplines involve human characteristics in contrast to economics and sociology where statistical data weigh more heavily in the description of reality.

Regarding publication years, one of the articles was published in the 1980s, 44 in the 1990s (all utilizing csQCA), and the remaining 425 articles were published after 2000 (Fig. 2).

Fig. 2.

Number of articles published with QCA per year.

(0.09MB).

In 2003, the first article using the fsQCA variant was published. It was authored by Paul Pennings and appeared in the European Journal of Political Research. Note in particular that since 2010 more than 300 (of the 469) articles use QCA in both its csQCA and fsQCA versions.

In general, Fig. 2 shows that the number of published articles employing QCA as the method of analysis was relatively low between 1987 and 2001. During this period, the method was being progressively refined. However, since 2001, the number of articles published using this method has increased exponentially from 5 to almost 20 in three years, and to more than 70 in 2015, to a total of 134 articles published in (2014 and first half 2015).

This increase could be due to several factors. First, in 2000, Charles Ragin, author of the method, published a book on application of fuzzy sets in the social sciences. This method represented a huge advancement with respect to its predecessor, csQCA. Second, the development and expansion of use of the Internet and computers has favored research by allowing access to information previously not easily available to researchers.

CsQCA is the most method in these articles. However, use of fsQCA is not far behind (Fig. 3). What differs is the length of time since each method was introduced. CsQCA was introduced in 1987 when there was no similar method in existence. FsQCA was introduced in 2001 as an alternative to csQCA with fewer limitations. The limited popularity of mvQCA is based on its introduction as a modest alternative to csQCA which improved some aspects related to rigidity but did not resolve them completely.

Fig. 3.

Number of articles using the different QCA methodologies.

(0.04MB).

Comparison of the articles published shows that mvQCA accounts for a marginal proportion (3%), while fsQCA (47%) and csQCA (50%) are of similar proportions, although the latter was introduced much earlier.

Regarding the quality and impact of the journals in which the articles were published, of the 469 articles analyzed, approximately 80% were published in journals indexed in the Social Science Citation Index of the Journal Citation Index (JCR). Only 21% were published in non SCI journals. Thus, it can be concluded that the methodology is relevant in the scientific field, given the academic prestige of the journals publishing these contributions.

Fig. 4 depicts the number of articles published based on different versions of QCA, in SCI and non-SCI journals. It shows that although there is a high number of published articles using the csQCA method, the number using fsQCA published in indexed journals is even higher. In the case of mvQCA, the data are insignificant and only a little more than half of the articles published were in SCI journals.

Fig. 4.

Methodologies used and indexation of the journals.

(0.04MB).

Table 1 presents the total number of citations and articles published in each of the fields of research collected by Compasss. To determine the number of citations, we exploited the Google Scholar (GS) database. Although the number differs from the number recorded in the WoS (Web of Science), the proportions are sufficient to enable comparison among authors and institutions (Bornmann, Thor, Marx, & Schier, 2016). The last column in Table 1 presents the percentage of citations per article. It can be seen that the fields registering the highest ratios are sociology, environmental science, and management and organization.

These data reveal greater interest in the fsQCA method from researchers in these fields of knowledge. Other factors that might explain these results include a greater tendency to cite in these areas than in areas such as pure sciences, or less codependency among authors who may focus on a variety of aspects within the same area. This is common in areas with a high degrees of interdisciplinarity.

Regarding articles with the highest impact, Table 2 presents those that use QCA as the research methodology and have more than 100 citations in Google Scholar. The most frequently cited authors are Cress and Snow, with 511 citations, followed by Ebbinghaus and Visser (377), and Fiss (324). The majority of the articles with high numbers of citations were published between the end of the 1990s and 2011, and four of the top five are in sociology.

Finally, we analyze the journals which publish the most papers using this research method. Fig. 5 depicts the titles with the highest number of published articles using this methodology. Note that the Journal of Business Research an economics and business journal, stands out with 26 articles published in this area, the majority in 2014 and 2015.

Fig. 5.

Journals with the highest number of articles employing QCA.

(0.1MB).

The next two journals publish in sociology, reinforcing the relevance of QCA in this area, with four journals in the top 10 (American Journal of Sociology, American Sociology Review, and Social Forces). The difference in the number of publications in the next ranked journals is minimal.

This article has conducted a bibliometric analysis of the QCA method in its three versions: csQCA, fsQCA, and mvQCA. The use of bibliometrics with a method as a subject of study is novel and makes a contribution to the literature.

The results show an increasing trend in the use of fsQCA at the expense of csQCA due to the advantages offered by the former regarding the inclusion of conditions with a degree of membership in the set. The exponential growth since 2007, in the number of publications which apply this method allows the prediction that, as new applications are discovered for QCA in other fields, the amount of research and its impact will increase. It seems that this methodology has been accepted by the scientific field, evidenced by the number of published articles using this methodology in indexed and high impact journals.

In future research, bibliometric analysis could be applied to other methods in development to identify their evolution. In addition, the evolution of two methods used for the same purpose could be compared and their application discussed to forecast future trends. For example, it could be applied to regression methods and fsQCA. Another opportunity for future research would be to analyze the use of different methods in the field of philosophy of science; this would contribute by analyzing the degree of adequacy of each method based on a solid, rational logic which goes beyond periodic trends occurring within the scientific community.