The purpose of this methodological insight is to analyze the foundations of multilevel research, answering two main questions: why this methodological approach is important for management research and how to conduct a multilevel study. We examine why multilevel research is relevant, emphasizing its potential, opportunities and basic principles. Moreover, we point out the main theoretical, methodological and analytic aspects to be considered for an appropriate application of multilevel research. The paper refers throughout to the basic literature on multilevel research, reviewing conceptual, methodological and empirical works.
In this methodological insight we examine the main foundations of multilevel research. Through this methodological approach, researchers can analyze relationships between variables on at least two different levels of analysis. For example, we can analyze the importance of several determinants of firm performance (an important issue in strategic management), considering not only variables at the firm level but also at other levels of analysis (industries and even the territories in which firms are located). Another example is how certain variables at the individual level (for example, job satisfaction of employees) along with organizational variables (for example, human resource practices) influence individual employee performance and/or firm performance.
In this paper we use the term ‘multilevel research’, and not the more common expression, ‘multilevel analysis’, in order to offer a broader vision of this methodological approach. When authors refer to ‘multilevel analysis’ or ‘multilevel models’ (some books use these expressions in their titles, such as Goldstein (1995), Hox (2002), Kreft and de Leeuw (2002) and Snijders and Bosker (2004)), they focus mainly on statistical techniques for conducting analysis on several levels (for example, hierarchical linear models, as noted below). By using the term ‘multilevel research’, we want to emphasize that this approach goes beyond statistical techniques (Klein and Kozlowski, 2000; Hitt et al., 2007), and consider other important elements. Multilevel research includes the development of multilevel theory (for example, combining different theoretical approaches at different levels and establishing relationships between constructs at different levels), as well as the main elements of methods for empirical studies (sampling, data collection, variables and their measures, and analysis techniques, which may include quantitative and qualitative techniques).
The main purpose of this methodological insight is to answer two main questions. Firstly, why is the use of multilevel research important for management? Secondly, how should multilevel studies be conducted? Thus, we examine why this methodological approach is important for the advance of management research and business practice, examining its potential, advantages and the opportunities that multilevel research can offer in the management field. Although it is not possible to examine in depth all the practical aspects involved in conducting a multilevel study, we indicate the main foundations and basic principles of this type of research together with key ideas for its proper application.
The main contribution of this methodological insight is to provide information that may be useful for management scholars on multilevel research. In our opinion, insufficient attention has been paid to this methodological approach in our field (except in areas such as organizational behavior and organizational psychology, which have introduced multilevel research to the management field). Multilevel research can offer opportunities to improve and advance research through the development of new theory, the improvement and expansion of data collection and analytic techniques, and a closer approach to the real problems of companies. We emphasize that multilevel research can contribute to filling two important gaps in our field (the micro-macro gap and the research-practice gap), contributing to responsible research. The main seminal works on multilevel research will be indicated.
The focus and intent of this work is to introduce the main ideas of multilevel research. In management studies, more researchers use a single level of analysis than use multilevel research, and we consider that many scholars in our field do not know the foundations of this type of research. Our approach will be to see the ‘forest’ rather than some particular ‘trees’. We examine the big picture, indicating the main elements of multilevel research. An exhaustive analysis of all the elements of multilevel research goes beyond the purpose of this methodological insight, but we provide key references in the literature that could be used.
Why is multilevel research interesting for management studies?In this section we highlight opportunities and potential offered by multilevel research. Specifically, we indicate its role in bridging two important gaps in management: the micro-macro gap and the research-practice gap. We also examine the benefits of integrating theories at different levels to advance management research. Finally, we suggest that avoiding the fallacy of the wrong level also justifies the application of multilevel research.
Bridging the micro-macro gap in management and integration of other levelsOne characteristic of the development of management is the growing diversity and specialization. Several areas have emerged and consolidated in the field of management, for example, strategy, human resources, operations, organizational behavior, and environmental management. These specialized areas are reflected both in teaching (in specialized subjects in undergraduate and postgraduate degrees) and research (with specialized journals and conferences). One aspect of this growing specialization is the widely accepted distinction between macro and micro areas (Aguinis et al., 2011). Macro areas (for example, strategic management and organization theory) mainly focus research questions and analysis at the organizational level, while micro areas (organizational behavior, organizational psychology) are concerned with research questions at levels of analysis within the organization, mainly at the individual and group level. The existence of macro and micro areas in the management field is reflected not only in the topics and levels studied, but also in the creation of specific divisions within management associations (even specific associations linked to macro or micro areas), separate and independent sessions for each area within management conferences (even specific conferences for each macro or micro area), and specialized journals. This specialization has benefits that arise from the opportunities that occur when deepening specific topics. However, it also entails less positive aspects linked to a growing separation and fragmentation (Durand et al., 2017). The micro-macro gap is an important indication of this fragmentation.
The relevance of multilevel research is that it can help to bridge this gap by integrating disciplines and levels of analysis. Multilevel research emphasizes the joint analysis of variables located at different levels, examining relationships between them. Many multilevel works examine in the same study variables at the micro (individual or group) level and variables at the macro (organization) level, and then integrate these levels bridging the micro-macro gap.1
Multilevel research can also integrate other levels. For example, a multilevel study can integrate the organizational level and higher levels. In strategic management, several studies have examined the importance of determinants of firm performance, especially the firm, corporate and industry effects, but also other determinants such as the strategic group and the location effects (Hough, 2006; Misangyi et al., 2006; Short et al., 2007; Molina-Azorín et al., 2010; Pereira-Moliner et al., 2011a).2 In addition, levels below the organizational level can also be integrated, for example, individuals and work groups, as has been studied in the area of organizational behavior.3 As examined in the next section, the integration of several levels in the same study can help to address and bridge another traditional gap in management: the science-practice gap.
Bridging the science-practice gapThe increasing specialization can also lead to another important gap in management: the science-practice gap (research-practice gap, or academic rigor-practical relevance gap) (Bansal et al., 2012; Kieser et al., 2015). Specialization and fragmentation in the management field can lead to separate, independent and very specific studies that are very different from real and integrated business problems. Academic studies tend to be remote from issues and solutions, and from research questions and responses that may be of interest to companies and managers in their daily practice. In other words, the fragmentation and disintegration of the field of management into various disciplines and areas means that studies carried out in each area suppose an oversimplification of the complexity of managers’ tasks and practices (Pettigrew and Starkey, 2016).
The real problems faced by companies and managers usually involve a joint analysis of various disciplines. For example, the formulation and implementation of strategies at the corporate and business levels must take into account actions and decisions in the operations and human resources departments. These problems presuppose relationships between aspects located at different levels of analysis. For example, a human resources management system or the firm culture (variables at the organizational level) can influence employees’ behavior at the individual level (for example, their motivation and satisfaction), which in turn can also influence both firm and individual performance. As indicated by Hitt et al. (2007), most problems that managers face imply phenomena at multiple levels, but most research is conducted on a single level.
In activities carried out in business practice (analysis of information, decision making by managers, implementation of decisions, etc.), elements at different levels are involved (for example, a management system at the organizational level, characteristics of managers and their actions, and interactions between managers and employees). The joint analysis of several levels through multilevel studies can close this science-practice gap. As indicated above, the problems that managers must solve involve actions and variables at different levels, and from several disciplines within management. Therefore, multilevel research brings us closer to the reality of business practice.
This usefulness and practical relevance of research in management is one of the basic pillars of responsible research, along with credibility. Therefore, multilevel research can also contribute to promoting responsible research and its basic principles (the basic principles of responsible research can be found at https://www.rrbm.network/).
Integration of theories and improvement of theoretical development in managementRelated to the previous aspects, multilevel research promotes the integration of theories from different disciplines. This methodological approach facilitates theoretical development to improve knowledge of business phenomena, which can help the advance of management (Shaw et al., 2018). If a certain business phenomenon is multilevel in nature, theory and techniques of analysis should also be multilevel (Luke, 2004; Hitt et al., 2007; Mathieu and Chen, 2011). Therefore, multilevel research can promote the improvement of theoretical development by facilitating the understanding of business phenomena with antecedents and/or consequences in different contexts and at different levels. For example, we could analyze the influences that variables located at two or more levels have on a dependent variable located at one of these levels (direct effects). We could also analyze a cross-level interaction, examining how the linkage between two variables at the same level is influenced or moderated by a variable at a different level (Aguinis et al., 2013).
In addition, regarding the distinction between macro and micro areas in management, there are movements and initiatives in different areas to integrate the parts. In macro areas, where research questions and issues are examined at the organizational level, the role of individuals is being encouraged. For example, in strategic management, the microfoundations movement (Felin and Foss, 2005; Molina-Azorín, 2014; Felin et al., 2015) emphasizes the key role of individual actions and interactions to explain strategic phenomena, as well as the reciprocal influences between the individual and the organizational levels in the study of strategic issues. In micro areas, such as organizational behavior and organizational psychology, the integration of macro variables and relationships with macro approaches is also being encouraged (Wright and Nishii, 2007; Ployhart, 2015). Another example is the area of human resources management. Although this area usually includes micro-level work analyzing the effects of individual characteristics on individual variables, macro work has also been carried out, for example, in the specific area of strategic human resources management, examining, among other aspects, the influence of systems and organizational practices of human resources on organizational performance. Ostroff and Bowen (2000) defend the integration of micro (individual and group) and macro (organizational) aspects in research on human resources management, being this integration a key aspect for theory development in this area.
The key point is that multilevel research can play an important role in these initiatives where micro areas seek to integrate macro aspects and macro areas try to integrate micro issues. Multilevel research may also be relevant to the conduct of multidisciplinary studies that integrate theories, relationships, processes and variables from different micro and macro areas, and from different levels of analysis.
The fallacy of the wrong levelAs indicated by Gaviria and Castro (2005), most data in social science research come from phenomena where subjects form nested hierarchies. For example, in the field of education, students are grouped or nested in classrooms, which in turn are grouped into schools. In management, employees are nested in work groups or departments, which are grouped into firms, and these are nested in industries.
Before the use of multilevel research to analyze relationships between variables at different levels, there were two ways to study nested data: aggregation and disaggregation. But these two strategies may produce errors when conclusions are drawn at the wrong level, that is, when inferences and interpretations are made about relationships between variables at a certain level but the analysis has been carried out at a different level, or when an effect, variable or relationship are attributed to a level of analysis when they really refer to a different level (Rousseau, 1985; Hox, 2002; Snijders and Bosker, 2004; Dansereau et al., 2006; Hitt et al., 2007; Mathieu and Chen, 2011).
Aggregation consists of obtaining data at a lower level, and combining the values of those variables to the higher level. For example, if we analyze employees who work in different organizations and we have data from several employees for each organization, the average for each company may be calculated from their employees4. The analysis may then be carried out at that organizational level, where we could also have and use other variables at this level. If we are interested in the relationships between variables at this organizational level (because this is the theoretical level of hypotheses), then there is not any problem. But if the theoretical level of interest is the individual level (employees), it would be a mistake to interpret the results of the organizational relationships and generalize them at the level of employees. In other words, we cannot apply the results that refer to the companies to the individuals. This error is known as the ecological fallacy. The main issue is that we have eliminated variance within companies, and the relationships may be strong at the organizational level, but can be very different at the individual level. In short, we would have a statistical problem and a conceptual/theoretical problem (Hox, 2002). The statistical problem stems from the fact that the employees’ values have been combined to form a smaller number of company values, losing much information and power in the statistical analysis. We would have a conceptual problem as we interpret results that are valid at the company level as if they were also valid at the individual level. Given the importance of aggregation in multilevel research, we will discuss some aspects later.
In disaggregation, data from higher-level units are disaggregated into data on a larger number of lower-level units (Hox, 2002; Snijders and Bosker, 2004). Using the example of companies and employees, the values of the variables at the company level would be assigned to their corresponding employees. Each employee would also have individual values for other variables. If an analysis is carried out at the individual level, there would be a statistical problem derived from the lack of independence: the employees of two different companies (and their observations) are independent, but two employees working in the same company (and their observations) are not independent, as they receive common influences (which, possibly, will not be measured). Later we will emphasize this problem of lack of independence. Another problem is that we could not interpret or make inferences at the company level based on the analysis developed at the individual level. This is known as the atomistic fallacy (Hox, 2002).
In summary, the problem with aggregation and disaggregation is that we would reach irrelevant conclusions regarding a certain level when using data measured and analyzed at a different level. Multilevel research helps to deal with these problems or fallacies that result from ignoring the hierarchical and nested structure of the data. Thus, this methodological approach resolves the dilemma between aggregation and disaggregation, working with several levels simultaneously.
Fundamentals of multilevel researchAs well as knowing why multilevel research should be conducted, it is also important to know how to carry out a multilevel study. Describing in depth all aspects and steps to conduct multilevel research goes beyond this methodological insight, given the limitations of space and the complexity of the issues that multilevel research covers. In this section we indicate the main foundations of this methodological approach, its key aspects and some conceptual, methodological and empirical works that can help to understand the main elements of multilevel research. In particular, we indicate the pioneering and main works on multilevel research in the management field, then we examine its basic principles, and finally main ideas on multilevel theory, research design and analysis are studied.
Pioneering and basic works on multilevel research in managementThe main principles, methodological foundations and techniques linked to multilevel research have been developed in other fields, mainly in education and psychology. For example, in education an important issue is to analyze the causes of students’ performance (individual level), which could be determined by aspects and characteristics of students (individual level) as well as contextual elements of their classrooms and schools (higher levels). In the management field, authors from micro areas, such as organizational behavior and organizational psychology (e.g., Bliese, Dansereau, Hofmann, Klein, Kozlowski, Mathieu, Rousseau, Yammarino, etc.) have mainly carried out the pioneering works on multilevel research (conceptual, theoretical, methodological and empirical studies).
Rousseau (1985) can be considered one of the pioneering works that introduces key aspects of multilevel research in the field of management in general, and in organizational behavior in particular. This author refers to an important book (Roberts et al., 1978) which defends the need to conduct multidisciplinary research, highlighting the excessive specialization in the management field. Rousseau (1985) points out possibilities of collaboration by establishing some basic principles of multilevel research. Later, this author, together with other colleagues (House et al., 1995), would emphasize the need to develop a ‘meso’ paradigm, that is, the simultaneous study of at least two levels of analysis (micro and macro) and their reciprocal relationships, indicating the need for this perspective and some basic concepts and principles of multilevel research.
In the three decades of multilevel research in management since the publication of Rousseau (1985), a key publication is the book, published in 2000, titled Multilevel theory, research and methods in organizations, edited by Klein and Kozlowski (2000). This book contains several chapters by authors from the area of organizational behavior, and it has contributed to the progress and consolidation of multilevel research in management. Taking into account the work on multilevel research published from 1985 to 2000, the aim of this book is to clarify and establish the main foundations of multilevel theory and of multilevel research methods.
Other important works in the development and consolidation of multilevel research in management include a book series edited by Yammarino and Dansereau (Research in multi-level issues), with several volumes between 2002 (Yammarino and Dansereau, 2002) and 2009 (Yammarino and Dansereau, 2009). In addition, several journals have published special issues on multilevel research and the micro-macro divide: Journal of Management (Vecchio, 1997), Academy of Management Review (Klein et al., 1999), Academy of Management Journal (Hitt et al., 2007), Organizational Research Methods (Bliese et al., 2007), Journal of Management (Aguinis et al., 2011) and International Journal of Human Resource Management (Shen et al., 2018). Organizational Research Methods has made a call for papers for a future special issue on multilevel statistics and methods. Other relevant and interesting conceptual and methodological studies (some published in the previous special issues) are Klein et al. (1994), Hofmann (1997), Chan (1998), Morgeson and Hofmann (1999), Klein et al. (2001), Hackman (2003), Chen et al. (2004), Hofmann (2004), LeBreton and Senter (2008) and Mathieu and Chen (2011).
In addition, some literature reviews, articles and chapters have also been published highlighting the potential of multilevel research in specific management areas, such as human resources (Jiang et al., 2013; Shen, 2016; Peccei and Van de Voorde, 2019), organizational psychology and organizational behavior (González-Romá and Hernández, 2017), international business (Arregle et al., 2006; Peterson et al., 2012) and strategic management (Moliterno and Ployhart, 2016). These methodological works and literature reviews examine empirical studies published in these areas, emphasizing best practice recommendations for designing and conducting multilevel studies. Empirical works can also be of value, illustrating how other colleagues design and implement multilevel studies. In the next sections of this methodological insight, we indicate exemplars of multilevel studies when examining the basic principles of this methodological approach and some important aspects of multilevel theory, research design and analysis that must be taken into account for its proper application.
Basic principles of multilevel researchNested/hierarchical structure, main models and relationships, and applicationsAs noted above, a key idea of multilevel research is the existence of a hierarchical system of several levels, where some entities reside in nested structures (Hitt et al., 2007). For example, in management studies, employees are nested in groups or work teams, which in turn are nested in other larger organizational units (for example, departments), which in turn are nested in firms. Moreover, firms are members of industries, and industries are located in certain environments or territories.
Another important aspect is that variables at each level may influence variables at other levels. For example, in organizational behavior, the dependent variable that is usually analyzed is some outcome variable of employees (for example, productivity). This variable at the individual/employee level can depend on both the characteristics of the employee (lower level or level 1) (for example, job satisfaction and/or motivation) and variables of the group or team where each employee works (higher level or level 2) (for example, climate or cohesion). In strategic management, a key research question is why some firms are more profitable than others. We have a structure or hierarchical pattern of nested relationships: firm profitability depends on firm resources and capabilities (firms as the lower level) and on the industries membership (industries as the higher level).
Regarding relationships between levels, influences may be reciprocal between the levels considered. Thus, productivity of an employee is influenced by the characteristics of her/his work team, and the team productivity is also influenced by employees. Performance of a firm is influenced by the characteristics of its industry, but the actions by firms also impact on industry performance and its characteristics. Therefore, a phenomenon at any level may have antecedents and consequences at other levels, and it is of special interest to analyze the immediate levels, both at higher and lower levels (Hackman, 2003).
Fig. 1 represents multilevel antecedents and consequences for some business aspect. For example, a firm capability can have antecedents and consequences at the organizational level, as well as at a higher level (industry) and lower level (individuals). In multilevel research, we refer to the different levels with a number; the higher level is indicated with the greater number (level 1=individual, level 2=firm, level 3=industry). The level with more units (in this case, individuals, as there are more employees than firms and industries) is graphically represented at the bottom of the figure. This lower level (level 1) is usually called as the micro level, and the higher level/s would be the macro level/s.
Most multilevel studies examine two levels, and there are two main types of direct relationships and models (Fig. 2) (Aguinis and Molina-Azorin, 2015). In top-down models, the outcome variable is located at the lower level, and we analyze some antecedent at this lower level and other antecedents at a higher level. The traditional multilevel analysis techniques, such as hierarchical linear models (HLM), were developed for these top-down models. An example in the field of human resources would be to consider as a dependent variable the individual performance of each employee (level 1), using as explanatory antecedents variables at the individual level (level 1) (such as motivation) and variables at the organizational level (level 2) (such as the human resources management system or organizational practices). Another example in the field of strategy would be to analyze the profitability of firms as the dependent variable (level 1), using as explanatory variables some characteristics of these firms (level 1) and some characteristics of their industries (level 2). In bottom-up models, the outcome variable is located at the higher level and the antecedents are located at that level and some lower level. For example, we can examine how firm profitability depends on some firm characteristics and some individual features of employees.
In the previous examples there is a pure and strict hierarchical relationship. For example, when examining employees and firms (two levels), each employee works in a specific firm. In other words, each unit of the micro level (employee) is nested in one (and only one) unit of the macro level (firm). This pure hierarchical relationship can also occur when we analyze more than two levels. For example, employees (level 1, micro) are nested in firms (level 2, macro), and these firms are nested in industries (level 3, macro). Each employee works in a specific firm and each firm belong to a specific industry. Apart from these pure hierarchical relationships, there are other possible applications of multilevel research.
An extension is multilevel cross-classification. In this case, several macro levels are used and the units of the micro level are nested simultaneously in these macro levels, but there is not a pure and strict hierarchy between these macro levels. Pereira-Moliner et al. (2011a) used this cross-classification structure in their multilevel study of the influence of firms, strategic groups and location (regions) on firm profitability. The firms (micro level) are nested in strategic groups (macro level) and regions (macro level), that is, a firm belongs to a strategic group and a region. But there is not a pure hierarchical relationship between strategic groups and regions: a region can have firms from different strategic groups, and firms from the same strategic group can be located in different regions. Guo (2017) also uses a cross-classification scheme in a longitudinal multilevel study on the determinants of profitability of business units, considering as the higher levels corporations and industries.
Another application is multiple membership. Multilevel models usually refer to single membership: each unit of the micro level belongs to one (and only one) of the units of the macro level/levels considered. Thus, in the previous example of firms, strategic groups and regions, each firm belongs to only one strategic group and only one region. But multilevel analysis also allows multiple membership. Mollick (2012) conducted a multilevel study with multiple membership and cross-classification and analyzed the role of two types of employees (designers and managers) on the development of games. Each of the designers and managers (micro level) perform their work for the development of more than one game (macro level), making them members of multiple groups.
Multilevel research can be also used to analyze longitudinal data. In previous examples, the macro levels referred to context variables that could influence variables at the micro levels. For example, industry variables may influence the profitability of firms. However, the higher/macro level does not have to be a context for the lower/micro level. Longitudinal data (repeated measurements) can be seen as multilevel data, as those repeated measures correspond to the lower level (for example, data on firm profitability for several years) and firms would be the higher level. These repeated measures are nested in the firms, and are correlated within each firm. In the longitudinal multilevel study by Guo (2017), level 1 is repeated measures of profitability for several years of business units (level 2).
Existence of dependenceAnother key aspect of multilevel research is the existence of dependence (or lack of independence) among the observations/measurements considered, as a consequence of nested structures (students and classrooms, workers and work groups, companies and industries, repeated measurements in longitudinal data). For example, in the field of education, in studies about performance of students (dependent variable, level 1) considering as independent variables both individual characteristics of students (level 1) and classroom aspects (level 2), the fact that different students are grouped within the same classroom implies that they share the same context variables of the classroom (for example, teachers). In addition, people in the same group interact and this can also imply similarities in terms of performance obtained by students. In the case of longitudinal analysis, the repeated measures will be correlated (for example, profitability for several years by each company considered). In the field of strategy, when analyzing, for example, the determinants of firm profitability (level 1) considering both firm factors (level 1) and characteristics of their industries (level 2), it is important to consider dependence, as firms in the same industry are influenced by the same structural variables of that industry.
As the independence assumption of observations of the classical linear model is not fulfilled, there are important statistical implications. If a traditional regression analysis (which assumes independence) is applied with nested data, type I error will increase (greater probability of rejecting the null hypothesis when it is true) as the existence of dependence will lead to obtaining a p value lower than the correct one, indicating greater statistical significance. Therefore, with nested data, it is not be appropriate to use traditional regression analysis and other statistical techniques that require the assumption of independence. Multilevel statistical analysis takes into account this dependence of observations derived from the nested structure.
Levels of theory, data source, measurement and analysisAn important issue in multilevel research is the need to consider the levels of theory, data source, measurement and analysis (Rousseau, 1985; Klein et al., 1994; Hofmann, 2004; Hitt et al., 2007; Mathieu and Chen, 2011; Moliterno and Ployhart, 2016). The level of theory refers to the entity or focal unit on which it is intended to make generalizations. In other words, it is the level at which a particular construct of effect is predicted to exist. The focal unit determines a specific level (organization, group, individuals, etc.) but in multilevel research different levels are considered. Scholars must clearly determine these levels in order to study theoretical relationships between them. Some authors prefer to consider the level of construct (Kozlowski and Klein, 2000; Hofmann, 2004) as a specific theory may include constructs that reside at different levels. Therefore, researchers must determine the level of each construct.
There may be a mismatch between the level of some construct and the level of the data source. For example, in research with primary data it is usual that managers (individual level as data source) provide information about a construct at the organizational level (for example, firm competitive strategy). In this example, the level of measurement (the unit to which the data are directly attached) is the organization. The important point is that the level of construct is aligned with the level of measurement. If we are working with a construct at the organizational level and managers provide information about this construct, questions or items should refer to that organizational level (level of measurement).
The level of analysis is the unit to which data are assigned for hypothesis testing and statistical analysis. It is important that the levels of theory and measurement are aligned with the level of analysis. When these levels are misaligned, problems related to fallacies of the wrong level may occur (see above ideas about these fallacies). However, the level of measurement may differ from the level of analysis when an adequate process of aggregation is implemented. For example, several employees of a firm indicate their individual perceptions about the organizational culture (measurement at the individual level). If we want to analyze the influence of organizational culture on another organizational variable (for example, firm performance) and, therefore, the analysis is at the organizational level, we must aggregate the individual perceptions of culture to create a variable at the organizational level. But, as indicated above, this aggregation must be justified both theoretically (processes that relate the two levels) and statistically (examining whether there is some agreement in individual assessments).
Justification for the application of multilevel researchThe application of a multilevel design and analysis must be justified based on additional aspects over and above the existence of nested structures and dependence. A key requirement is to have sufficient data at the levels analyzed. Hitt et al. (2007) indicate that a multilevel theory may not necessitate a multilevel design, and they point out an interesting example. Upper echelons theory is multilevel in nature in that it incorporates features on individuals (mainly CEO), groups (top management team) and organizations. However, empirical research into this theory has been carried out at a single level (the organizational level) because there is only one CEO and one top management team per organization. Therefore, a multilevel design is not used as multiple lower-level entities must be nested within a higher-level entity (several top management teams and several CEOs would be needed in each firm).
Another requirement is to use sufficient data at the higher level. An example would be a study on behavior of companies in three different countries (several companies in each country). There is a nested structure and companies in each country could show similarities in their behavior due, for example, to the regulatory framework of each country. However, the fact that only three units (three countries) are available at the higher level (three countries) would violate one of the requirements of multilevel modeling as there are insufficient units at the micro level and the macro level.5 Later, in the section on multilevel sampling, we examine the issue of sample sizes.
In addition, even if there are enough units at the micro and macro level, a statistical justification is required to conduct a multilevel analysis. Specifically, there should be significant differences in the lower-level variables based on the higher level. In other words, contextual variables at the macro level should exert an influence on the micro-level variables. This statistical justification can be based on various indices, such as the intraclass correlation coefficient 1 -ICC(1)- , which determines which part of the variance of the dependent variable at the lower level is due to the higher level variability (Bliese, 1998, 2000). ICC(1) values different from zero are desirable for considering the higher level. Bliese (2000) indicated that ICC(1) values typically range from .05 to .20. A high intergroup variability would justify the search for higher-level predictors. When ICC(1) is zero or very low, it would not make sense to include a higher-level predictor. In this case, the results of a multilevel model will be similar to a classical linear model, and the use of a multilevel analysis would not produce benefits in terms of bias and efficiency of the estimated parameters, even if the data have a nested hierarchical structure.
Theoretical, methodological and analytical aspects in multilevel researchTheory and models in multilevel researchAs will be indicated later, methodological and analytical issues are important in multilevel research. However, the first and key steps in multilevel research are those related to theoretical and conceptual elements, establishing theoretical arguments that serve to specify the model to analyze, justifying the relationships between variables as well as the processes and mechanisms that connect the variables at different levels. It will also be important to establish the constructs and their definition, justifying their levels.
With regard to multilevel models and relationships, we previously referred to top-down and bottom-up models (Fig. 2). These models incorporate within-level direct effects and cross-level direct effects. We can extend the effects that can be examined, including cross-level moderation effects and aggregation/emergence processes (Moliterno and Ployhart, 2016). Cross-level mediation effects can also be considered and analyzed. Fig. 3 includes all these effects.
As noted above, theoretical arguments that justify the multilevel relationships among variables are important. Several theories that examine relationships at different levels may be integrated (for example, macro theories at the organizational level and micro theories at the individual level). Theories that link constructs at different levels can also be used.
Sampling and data collection in multilevel researchSampling in multilevel research is carried out in several stages, taking into account the levels that are considered. For example, in a multilevel study with data at the organizational level (level 2) and at the individual level (level 1), there must be enough organizations (first stage of sampling) and enough individuals in each organization (second stage) in order to reach appropriate statistical power. Therefore, there are requirements of sample sizes needed at each level to conduct multilevel analysis.
The 30/30 rule of thumb (at least 30 organizations and at least 30 individuals in each organization) is usually indicated as the minimum to reach enough power for cross-level direct effects and cross-level interactions (Hox (2002); Kreft and de Leeuw, 2002) to ensure sufficient variance between organizations and within organizations. However, this rule has been considered excessively demanding. In practice, studies consider fewer units in the lower level (in our example, the number of individuals within each organization). In fact, it is usually thought that it is more important to have a large number of elements in the higher level than in the lower level (Shen, 2016). Many published works include 5 or even fewer individuals in each organization, but try to have a large number of organizations. As there are several multilevel analysis techniques and multilevel effects (interactions, direct effects, …), different and specific requirements of sample sizes may apply (see Maas and Hox (2005), González-Romá and Hernández (2017), and González-Romá (2019)).
In terms of data collected, some multilevel works use primary data sources (Liao and Chuang, 2004; Pereira-Moliner et al., 2011b; Linuesa-Langreo, 2017) and other studies use secondary data (Short et al., 2007; Guo, 2017). A positive aspect of multilevel studies (especially works that use primary data through questionnaires and the levels considered are individuals and organizations) is the use of several informants for each organization. As indicated by Bou-Llusar et al. (2016), empirical studies with a single informant present several problems. Multilevel studies require multiple informants in each organization, and this positive characteristic of multilevel studies could be strengthened in two ways: first, it is possible to use informants from different hierarchical levels within the organization (for example, managers and employees); and second, employees could provide information about variables related to their managers and managers could evaluate certain aspects of their employees. This is an important point as researchers must determine the appropriate informants for specific constructs. In order to avoid social desirability bias, in a study about leadership in teams, leaders/managers/supervisors in these teams could provide biased information about their own leadership style, and then it would be important that employees provide information about leadership style of their managers (Bavik et al., 2017; Jiang and Chen, 2018; Linuesa-Langreo et al., 2018). As noted above, managers may also provide information about their employees. For example, in Zhu et al. (2018), managers assessed their subordinates’ creativity at work.
There are other important aspects to consider on data collection in multilevel studies. For example, in studies with primary data through questionnaires, a relevant issue will be the decision about the specific individuals in each organization who would provide information, depending on the objectives of the work. It is not only important to determine whether informants are managers and/or employees but also to determine the specific hierarchical levels, jobs, specific workers or core members in specific teams. Another important aspect is how to manage and collect that information from individuals (managers and employees): through a visit to the organization where the researcher interviews all the informants; or the researcher only interviews a manager in each organization, and then this manager distributes questionnaires to other individuals in this organization; or even the use of a telephone or online survey.
Aggregation and collective constructsFor bottom-up relationships, an important theoretical and methodological aspect of multilevel research is the distinction between influences between variables at different levels and aggregation or emergence of collective constructs. We indicate some aspects of aggregation and collective constructs in this section. Aggregation refers to the emergence of a variable at a higher level as a consequence of the aggregation of characteristics or perceptions of some units at a lower level. Examples would be organizational culture and organizational climate, which emerge as aggregation of the perceived values of individuals on these aspects.
As will be indicated in the next section, this aggregation must be justified from a statistical point of view. However, firstly, a theoretical justification is needed. González-Romá (2019) points out that this theoretical explanation is frequently disregarded in research manuscripts but understanding the nature of higher-level constructs and the processes involved in their emergence from lower-level properties is theoretically crucial. In this regard, two basic aggregation or emergence principles can be identified (Chan, 1998; Kozlowski and Klein, 2000; Hitt et al., 2007; Mathieu and Chen, 2011): composition and compilation.
In the composition process, each unit at the lower level (for example, each employee) contributes equally to the index that represents the variable at the higher level. Chan (1998) indicates different forms of composition models (e.g., direct consensus, referent-shift, etc.). Each model implies different theoretical assumptions about the nature of the higher level construct (Paruchuri et al., 2018). In the composition process, descriptive statistics (such as the sum of individual scores or mean) adequately represent the processes in which the lower level data (for example, individual perceptions of several employees about the values of organizational culture) are associated with a higher level collective construct (organizational culture). Another example is the use of the average of individual knowledge and skills of employees in an organization to represent organizational human capital.
In the compilation process, the higher-level phenomenon is a complex combination of the contributions of the lower-level units. Here measures of the units at the lower level (for example, individuals) are combined in a complex way, with some individuals being able to contribute more than others to the higher level variable. Therefore, simple descriptive statistics could not be used as in the composition process. An example of compilation process is the concept of climate uniformity (González-Romá and Hernández, 2014).
As noted above, composition and compilation processes are linked to collective constructs. These collective constructs are characteristics or properties of some group at some higher level that includes units at a lower level. For example, work teams, organizational departments and firms are groups of individuals (employees and managers). There are three main types of collective constructs: global, shared and configural constructs (Kozlowski and Klein, 2000; Hofmann, 2004).
Global constructs are descriptive characteristics of the group (for example, the size of a firm considering the number of employees). The main characteristic is that these constructs are objective attributes that do not depend on individual perceptions and behavior. Therefore, there is not any aggregation or emergence process. A global construct does not cross levels, and it is assumed only to operate at the group level. However, these global constructs can influence the characteristics of members working in the group.
Unlike global constructs, shared and configural constructs cross levels as these two collective constructs have their origin at the individual level. Shared constructs only come into existence and have validity when members of the specific group share similar perceptions. An example would be the organizational climate. Homogeneity is considered in the sense that all individuals are equally important. These shared constructs are linked to composition processes of emergence, and a structural equivalence between levels is usually established (isomorphism).
Configural constructs are similar to shared constructs in the sense that they arise from individual-level attributes and perceptions. However, in configural constructs, the individual actions, characteristics or perceptions combine in some complex and non-linear way. There is not homogeneity as some individuals may contribute more than other individuals. Therefore, configural constructs are associated with compilation processes. For example, regarding efficacy and performance of teams (e.g., a football team or an orchestra), the actions and interactions of individuals may compile in a complex way, with each player or musician performing different interdependent roles to produce the overall group performance. In an orchestra, each musician can play their instrument very well, but this does not mean that the orchestra as a whole plays well.
An important aspect of collective constructs is their validity and reliability. Chen et al. (2004) provide recommendations regarding indices of internal consistency, agreement/consensus among members and aggregate reliability for each specific type of construct. Another relevant issue is the wording of survey items used to measure group-level constructs (collective constructs) through individual-level data. Chan (1998) and Klein et al. (2001) provide important ideas and suggestions.
Analysis in multilevel researchIn the ‘introduction’ section, we indicated that we use the term ‘multilevel research’, and not ‘multilevel analysis’, to offer a broader vision of this methodological approach, as examined above. However, multilevel analysis, through the use of multilevel statistical techniques, is an important part of multilevel research. Over the years, great advances have been made in multilevel statistical analysis. Progress has focused not only on statistical justification of aggregation of lower level variables creating higher level, collective variables, but also on multilevel statistical techniques to analyze relationships between variables at different levels (cross-level direct, cross-level moderation and cross-level mediation effects). As noted below, hierarchical linear models (HLM) and multilevel structural equation modeling (MSEM) are some of the main advances. From a quantitative and statistical point of view, we can distinguish two groups of techniques: justification of aggregation and analysis of influences among variables.
With regard to aggregation, once the process of aggregation has been theoretically justified, statistical aggregation must be also supported through the use of several indices and statistical coefficients. There are two main types of indices: indices used to estimate inter-rater agreement/consensus and indices used to estimate inter-rater reliability (LeBreton and Senter, 2008). For some composition models and collective constructs (for example, shared constructs based on direct consensus models and referent-shift consensus models), to justify aggregating lower level data to the emergence of a higher-level construct, it is necessary to demonstrate that the lower level data are in agreement with one another (e.g., organizational climate). Within-group agreement and between-group variability are needed to justify data aggregation. Some indices to estimate inter-rater agreement are rWG indices and average deviation (AD) indices. It is also important to consider and estimate reliability. Reliability can be considered a measure of the consistency of responses among raters (Bliese, 2000), and it may be assessed through intraclass correlation coefficients. Bliese (2000) and LeBreton and Senter (2008) offer a conceptual and methodological description of some of these indices. Some empirical studies examine aggregation without analyzing influences between variables at different levels (cross-level effects) and therefore the statistical analyses are carried out at a single level (González-Romá and Hernández, 2016; Von Bonsdorff et al., 2018). Other empirical studies use aggregation to create some higher-level construct and next implement a multilevel analysis of cross-level effects between variables (Zhu et al., 2018).
Regarding the analysis of influences between variables at different levels in multilevel models (cross-level effects), several techniques and models can be used. These models can be classified in two main groups: conventional multilevel modeling (such as HLM) and more recent and advanced techniques (such as MSEM). González-Romá and Hernández (2017) study these multilevel models, examining their main characteristics, their performance, and how MSEM overcomes the limitations of conventional multilevel models. Next, we indicate some brief ideas about these two models.
Conventional multilevel models were mainly developed for top-down relationships. For example, through HLM (Hofmann, 1997), we can examine the impact of a lower-level variable (level 1) and a higher-level variable (level 2) on a dependent variable at the lower level (level 1). Pioneering areas that developed multilevel analysis were mainly interested in these top-down relationships. For example, in education, a key dependent variable is students’ performance (level 1) that is influenced not only by individual characteristics of students (level 1) but also by contextual variables, for example characteristics of schools (level 2). In organizational behavior, important research questions examine which variables (individual and group variables) influence employees’ performance, productivity, satisfaction and other characteristics of individuals. This top-down approach that characterizes conventional models is also based on the logic that context (higher level variables) exerts greater influence on variables at lower levels than the influence of lower level variables on context variables (Hitt et al., 2007). Some empirical works that have applied HLM are Liao and Chuang (2004) and Jansen et al. (2012).
Conventional multilevel modeling has several limitations. One important limitation is that it cannot model bottom-up effects (effect of a lower-level variable on a higher-level variable). Moreover, there are problems with cross-level mediation models. In recent years, new advanced multilevel analysis techniques are being developed to examine upward influences and improve the analysis of cross-level mediation. This progress is being made through MSEM (Preacher et al., 2010, 2011). For example, 1-2-2 models can be analyzed (influence of a lower-level variable on a higher-level variable through a mediating variable at the higher level). Several empirical studies that use MSEM are Yao and Chang (2017), Beltrán-Martín et al. (2017), Beltrán-Martín and Bou-Llusar (2018) and Distel (2017).
To carry out multilevel analysis, researchers can use generic software that includes a multilevel research module (for example, SPSS, MPlus, EQS) and specific multilevel software (for example, MLwiN and HLM).
Finally, it is important to note that, in addition to quantitative/statistical analysis, multilevel research can also be conducted using qualitative methodologies. Some examples of qualitative studies are Salvato (2009) and Huy (2011). Qualitative methods can be useful and appropriate to study the specific mechanisms of emergence processes that help build higher-level concepts (e.g., human capital) from lower-level units (e.g., individual employees’ knowledge, skills, abilities and other characteristics). Moreover, quantitative and qualitative methods can be combined and integrated in the same multilevel study, using a mixed methods approach (Bapuji et al., 2012).
ConclusionsIn this methodological insight we have indicated the main characteristics, advantages and basic principles of multilevel research, from a general and broad perspective. This methodological paper may be especially interesting for those researchers who have not conducted multilevel research but want to know its usefulness and foundations. We have also provided the main literature on specific aspects of multilevel studies. Our recommendation is that scholars should read both conceptual/methodological works on multilevel research and multilevel empirical studies. As noted above, one of the best ways to learn how to carry out a multilevel study is to examine how other colleagues have conducted these studies. Several special issues on multilevel research that have been published in some journals and literature reviews on this methodological approach include and analyze empirical studies, indicating specific suggestions and recommendations to carry out multilevel research (Jiang et al., 2013; Moliterno and Ployhart, 2016; Shen, 2016; González-Romá and Hernández, 2017; Peccei and Van de Voorde, 2019).
As examined in this paper, multilevel research offers opportunities to advance the management field, but this methodological approach is not a panacea. Multilevel research should not be considered superior to other research approaches. Research questions determine the appropriate methodological approach, and research at a single level will continue to be conducted and will be relevant to many research questions. Nonetheless, an important aspect of the relationship between research questions and methods is that research questions influence the methods we use, but methods may also influence the research questions we ask. Research questions shape and are shaped by methods. By extending our methodological skills and our repertoire of methods, we can improve the question-asking process, increase the rigor of our conceptual thinking, see new ways to answer research questions, and even identify questions that would not have occurred to us otherwise (Edwards, 2008). And we consider that knowledge of multilevel research can provide opportunities for identifying relevant research questions and answering these questions with theoretical and methodological rigor. Multilevel research may facilitate the identification and analysis of questions and problems that are relevant to practitioners as this methodological approach considers several levels of analysis. Moreover, multilevel research may promote collaboration between researchers from different management areas, both macro (e.g., strategy, organizational theory) and micro areas (organizational behavior, human resource management), conducting multidisciplinary studies closer to real problems of organizations.
Together with these advantages and opportunities, researchers must also consider that multilevel research has important challenges. Multilevel studies are usually more complicated and complex (e.g., combination and integration of micro and macro theories) and require more effort and resources (e.g., data collection from multiple informants; research skills at the theoretical, methodological and analytical level) than studies at a single level. An important aspect for addressing these challenges is training. Some doctoral training of novice researchers does not promote a broad understanding of micro and macro aspects, and knowledge of several methodologies. An obstacle for conducting multilevel research is the specialization in theoretical approaches (either macro or micro) and in research methodologies focused on a single level.
We encourage researchers in management to consider the advantages and opportunities of multilevel research, assessing the possibilities of this methodological approach for studying their research topics. In this methodological insight we have indicated the main foundations of multilevel research and some important works that may be useful to promote the use of this methodological approach.
This work has been funded by the Spanish Ministry of Economy and Competitiveness (Research Grant ECO2015-67310-P). The authors are grateful for and acknowledge the support received. An earlier version of this study was presented at two workshops organized by the Human Resource Management Division and the Strategy Division of the Spanish Academy of Management (ACEDE). We would like to thank Editor Xosé H. Vázquez and Associate Editor Alicia Rodríguez-Márquez for their suggestions. We are also grateful to Juan Carlos Bou-Llusar, Vicente González-Romá, Jorge Linuesa-Langreo and two anonymous reviewers for their constructive comments, examples of multilevel studies and feedback on earlier versions of this article.
In the case that only two levels are examined, for example the firm effect (the role of internal resources in explaining firm performance) and the industry effect (the industry membership of each firm), the lower level (firm) is also called micro level, and the higher level (industry) is also called macro level. Therefore, in multilevel research the use of the terms ‘micro’ and ‘macro’ levels is different from how they are used when we refer to micro and macro areas in the field of management.
With regard to the territory or location effect, we must emphasize that the possibility of applying multilevel research in management has benefited from the emergence of multiple international research projects that collect cross-country data systematically (for example, the Global Entrepreneurship Mónitor project – www.gemconsortium.org, which collects annual data on entrepreneurial activity in more than 100 countries). We thank one of the reviewers who indicated this example.
In this case, the individuals would be the lower or micro level, and the work groups the higher or macro level.
Below we examine the theoretical and statistical justification needed for aggregation.
We thank one of the reviewers for this example. In this case, when a multilevel analysis cannot be applied, a solution would be the use of dummy variables to examine the effect of context (country) on the behavior of companies, or the use of multigroup models in structural equation modeling.