Organizational performance evaluation in intangible criteria: a model based on knowledge management and innovation management

Abstract

Today innovation and knowledge management are determining factors for success and continuity of organizations. However, because they are considered intangibles, their measurement becomes a challenge. Therefore, this study aimed to develop a model to measure organizational performance with a focus on knowledge management and innovation management. To be able to do that, we used a quantitative research study, characterized as a multi-case study applied to three companies in the metal-mechanic sector in southern Brazil. The methodology uses the assumptions of well-known methods such as the Key Performance Indicators, the Swing Weighting and Simple Attribute Rating Technique. With the results, it could be seen that the proposed model can be an effective tool for assessing organizational performance and that, in its application, the surveyed organizations could already identify their main weaknesses and use the results reported to improve its management.

Introduction

In a world of constant change and where organizations compete with literally everyone in the global network, there are many studies on how to differentiate amid increasingly constant innovations, increasingly improved techniques and knowledge ever wider. The need for the organizations to adapt grows, given the discontinuities created by the globalization level, high volatility, hyper-competition, demographic changes and explosion of knowledge (Porter, 2009). The media, continuously faster, changes the business climate and every day it becomes more evident that organizational learning and knowledge management, as well as innovation, are prerequisites to face this kind of global trend (Easterby-Smith, Burgoyne, & Araujo, 2001; Nonaka & Takeuchi, 2008).

It is precisely this context that this paper seeks to explore. The era of knowledge as an important transformation of organizations, society and professionals, the management of that volume of knowledge following the changes and the importance of innovation as competitive advantage.

This paper adopts a broader definition of innovation in line with studies of Schumpeter (1984), focused not only on the product, but the phenomenon that goes beyond the dimension of technology. Moreover, it is emphasized that this article is geared to the firm, i.e., an internal dimension and not the systemic capacity of an economy\society to innovate.

It is evident that the ability to innovate is considered one of the most important features of competitive organizations. Because of this, the systematic search for radical innovations, i.e. those able to create new markets and provide rapid economic growth and production expansion and for incremental innovation, identified as continuous improvement processes, to “do better what was already being done”, is critical to the survival of businesses (Carnongia, Santos, Santos, & Zachiewicz, 2004; Machado, Carvalho, & Heinzmann, 2012).

However, how to evaluate whether an organization is or is not competitive and innovative? How to measure the results of the management of its knowledge? Innovation and knowledge management are now considered intangible assets and, therefore, their measurements become a big challenge for organizations. Thus, in view of the presented topic, the objective of this study was to develop a model to measure organizational performance with a focus on knowledge management and innovation management. Therefore, it was considered necessary to build a measurement tool; to apply the proposed tool to evaluate its effectiveness; to analyze the performance index obtained in the surveyed organizations; and to compare the results obtained from the companies surveyed to identify key areas for performance improvement.

This research is justified by the imminent growth on the issue of knowledge management and also the importance of the subject associated with innovation. In the same vein, with competition increasing and intensifying the race to get ahead, innovation becomes an important strategy for growth and even survival for organizations. This work is also justified by the contribution to the business world as it seeks to explain and solve, through the scientific method, phenomena that are part of the daily routine of companies. It is also important to mention the subjectivity involved in the constructor of innovation and knowledge management because they are intangible. In this line, it is a very big challenge to measure the performance of organizations in these respects. This study contributes to a tool that enables this measurement.

The work is divided into four sections besides this introduction. The second section offers a brief review of the literature on the concepts involving knowledge management and innovation management. In the third, there is a complete explanation of the methodology used for the study. In the fourth, the results achieved by applying the proposed method are reported. The fifth section seeks to make an overview of the work, ending it with the book references.

Systems for organizational performance measurement

The process of performance measurement is considered one of the key elements of strategic management, being able to identify the gap between the current situation of an organization and the level of excellence to be considered, by proposing goals that are aligned with strategic planning and the use of indicators (Hill & Jones, 2012; Kaplan & Norton, 2008). The proposal of using indicators is based on the fact that tangible and intangible factors, such as innovation, can always be measured, as long as they use well-defined metrics, routines that operationalize the data collection and standardized measurement scales, translating scattered data into useful information for managing production units (Hubbard, 2009; Olson & Slater, 2002).

Takashina and Flores (1996) say that the use of indicators plays essential role in planning and control activities, since they enable the establishment of quantifiable goals that help in anticipating future events and monitoring of current processes, assisting in decision-making and in the pursuit of operational excellence. Consequently, the provision of these tools contributes to both innovation and knowledge managements when promoting mechanisms that bring back robust information on their processes to the managers (Parmenter, 2012; Samsonowa, 2012).

Fernandes (2006) highlights an important topic about the performance evaluation, to clarify that the expected results may differ between the various stakeholders in the performance of an organization. Notably, the owners seek maximum return on investment (ROI), employees seek maximum payment and customers call for innovative high quality products at the lowest price, so the main goal is often a conflict between these groups. Thus, it is important to outline to whom the performance measurement system is destined and to which strategic vision it aligns.

Several models are available in the scientific literature related to performance measurement, each one with features that seek to track the rapidly changing global market. This concern was demonstrated by Neely (2002), which notes the growing expansion on researching this theme.

Amidst all these proposals, a compilation made by Neuenfeldt Júnior (2014) presents in a summary form some of the models considered most relevant to the performance measurement as well as their main features, as it can be seen in Table 1.

Table 1.

Methods for performance measurement.

Method	Key features
Management by objectives (APO)	Effort directed technique through planning and administrative control, in which the goals are set together between manager and his superior and responsibilities are specified for each position according the expected results.
KPI	Tool used to assess the status of certain activity, so that the levels of an organization understand how their jobs affect the business.
Balanced Scorecard (BSC)	Translates the organization's strategy into a set of measures that carry out the measurement of its performance in order to achieve the main strategic goals set.
Three Levels of Performance	Considers the establishment of three levels (organization, process, and performer) of performance, so that a company or system can be assessed from the implementation of these vertices.
Mckinsey 7-S	Management model developed to understand seven factors considered determining for effective change in an organization.
Baldrige	It aims to provide assistance to companies when it comes to stimulating the improvement of their quality and productivity by providing the necessary information to reach a high level of qualification of their processes
Quantum	Model proposed in order to associate mission, strategy, goals and processes within the organization, working with an array in three dimensions: quality, cost and time, seeking balance between these.
Performance Prism	It is a methodology that aims to integrate the processes in order to create value for the stakeholders in the system, starting from indicators that refer the status in which the management is.

Source: Adapted from Neuenfeldt Júnior (2014).

This list of possibilities, however, should not be understood as isolated models but as flexible options able to adjust the best possible way to the reality intended to be modeled, leaving to the user of the tool the responsibility to be sensible enough to do that, since even the scientific literature does not present a consensus of which method is most appropriate.

Adding to this, publications that are intended to identify desirable attributes in performance measurement systems such as the study by Figueiredo, Macedo-Soares, Fuks, and Figueiredo (2005) stand out, which identified the following nine characteristics based on the analysis of different bibliographic sources: organizational learning; critical analysis; balancing; clarity; dynamism; integration; alignment; participation; and causal relationship. The author also lectures on each of these attributes, in an attempt to guide the reader in the choice of an evaluation model. Accordingly, Simons (2009) argues on four points of view that should support the construction of a performance measurement system:

• (a)

  Its function should be to transmit basic information about the case either having economic focus or not;

• (b)

  It must contain routines and standard procedures;

• (c)

  It should promote cross-checks that allow the systemic view of the business, not the exact representation of processes’ data.

• (d)

  It should focus on improving the efficiency and effectiveness of processes, directed to the goals.

In light of the desirable characteristics for a performance measurement system, the Key Performance Indicators (KPIs) stand out. Parmenter (2012) states that there is a general misunderstanding about the tool because many organizations use measurements that, despite returning valuable information, cannot be considered KPIs. To define which are and which are not KPIs, the author frames the indicators under four groups:

• (a)

  Key Results Indicators (KRIs) express the performance achieved in a perspective of the Balanced Scorecard or critical success factors;

• (b)

  Result Indicators (RIs) express any result achieved;

• (c)

  Performance Indicators (PIs) express what should be done;

• (d)

  Key Performance Indicators (KPIs) express what should be done to boost performance dramatically.

As it can be seen, KPIs are a set of special indicators that reflect in a quantitative and condensed way the performance of a specific sector of the organization as a whole, affecting not just one but multiple perspectives of the BSC or critical success factors (CSF) (Dransfield, Fischer, & Vogel, 1999; Meyer, 2003; Parmenter, 2012; Samsonowa, 2012). Thus, the use of KPI assumes to establish a strategy with a target they want to reach (objective KPI), and through it unfold the CSF, where the correct identification of these corresponds mainly to the successful implementation of the methodology (Parmenter, 2012; Samsonowa, 2012).

Finally, the use of KPIs as a performance measurement system can be considered as an updated tool because of the theme recurrence in scientific works. Recent publications by Janes and Faganel (2013), Flipse et al. (2013), Dombrowski, Schmidtchen, and Ebentreich (2013), May et al. (2014), Sánchez (2014) and Galar, Berges, Sandborn, and Kumar (2014) are some of the latest examples that address KPIs and show this subject is being widely explored by the scientific community today.

Knowledge management

Although manufacturing still have fundamental importance for development, globalization has changed the concept of competitiveness of developed economies, moving away the standardized manufacturing activities from the knowledge-based services (Friedman, 2005). The world is moving from an industrial age, based on natural resources, to an era of knowledge, based on skills, education and research and development. Knowledge has emerged as a key source of jobs and economic growth in the global economy because it is the basis for innovation (Gulbranson & Audretsch, 2008).

Organizational knowledge provided improvements in the course of processes, activities, competitiveness and growth of organizations. The characterization of knowledge as the most important production asset of the organizations is the central aspect of the twenty-first century society, standing out from the traditional assets such as hand labor, capital and technology. Before, the central value was the mass production of goods, valued for its materiality. Today the central position is occupied by ideas, information and digital codes, valued in its materiality producer of innovation, creativity and service (Nicolás & Cerdán, 2012; Zabot & Silva, 2002).

Although there are numerous approaches to conceptualize knowledge management, there is a consensus that it is a structured approach to creation, codification, use, exchange and retention of knowledge to meet the organizational challenges and to create additional value (Rowlei, 2000; Tobin, 1998). Contemporary organizations are a result of the knowledge they and other individuals and groups have built in the past and continue to build through the experience they have and the changes that occur all the time (Rodan & Galunic, 2004; Zabot & Silva, 2002).

Knowledge can provide sustainable advantage, since over time competitors can usually even the price or quality of the products offered by a company. Meanwhile, the company that is rich in knowledge will be able to reach a new level of quality, creativity and efficiency. The advantage of knowledge is sustainable because it generates increasing returns by using tools that competition does not know (Chou, Wang, & Tang, 2015).

The relevance of knowledge as a base for innovation requires exploration and interaction of different sources for its achievement. With all the resources available now and the speed with how changes are happening, there is a growing demand for combination of sources, information and knowledge, facilitated by the proper management of these resources. This led to a substantial increase in the degree of interaction between organizations (Chen & Fong, 2015).

The current meaning of competitiveness covers not only performance excellence and technical efficiency of companies or products, it also covers the ability to develop systematic processes to search for new opportunities and overcome technical and organizational obstacles via production and application of knowledge. Innovation management seeks to bring together the mechanisms and instruments as well as the methodologies and forms of organization that can guarantee the ability to innovate in organizations based on knowledge acquired inside and outside the company (Carnongia et al., 2004).

However, there is still a lot of difficulty in the adoption of knowledge management practices by organizations and part of the reason for the possible failure of knowledge management initiatives is justified by skepticism because of the inability to develop metrics to measure the success of these practices. Knowledge management deals with intangible assets and for that reason it might be difficult to measure the benefits, but management needs to know in depth the benefits in terms of added value, derived from knowledge management initiatives (Liebowitz, 2013; Poyhonen & Hamalainen, 2001; Roper & Dundas, 2015).

Mills and Smith (2011) led a study that sought to find a way to measure knowledge management. The study provides evidence linking some knowledge resources to organizational performance and the results show that some knowledge resources (e.g., organizational structure and the application of knowledge) are directly related to organizational performance, while others (e.g., technology and conversion of knowledge), although important preconditions for knowledge management, are not directly related to organizational performance. Those insights can help companies to better target their investments and increase the success of their knowledge management initiatives.

Innovation

Industry has gone through periods of intense competitiveness in globalized economy, characterized by increasingly more efficient and qualified processes and technology, so that the adoption of innovative strategies becomes crucial in the management process (Hitt, Ireland, & Hoskisson, 2012; Porter, 2009). Thus, to ensure the survival of any business, it is necessary that the activities create value, not only within the organization's boundaries, but also in any competitive environment (Certo & Peter, 2005; Di Serio & Vasconcellos, 2009).

Innovation management is a contribution to the companies for making them more competitive in the market through the use of new concepts or improving existing concepts in the organizational context. Innovation is no longer a differential but became a determining factor for the continuity of businesses (Wang et al., 2008; Forsman, 2011).

There are still many studies about the innovativeness that aim to develop the concept of innovation itself, to try to identify the specific skills needed to make it possible to innovate (Wang et al., 2008; Yam et al., 2011; Forsman, 2011; Alves et al., 2011). However, other inputs are still needed to consolidate the concepts of innovation, since this mapping out is very complex and involves all areas of the organization.

The contribution of Schumpeter (1984) proved to be very rich in understanding the importance of innovation in organizations. The author points to a form of holistic innovation in order to determine dimensions for innovation, arguing that it can come in the dimension of a new product, a new process, in the search for new markets, developing new sources of raw materials or new market structures. He also says that innovation involves combining different types and parts of knowledge and turning them into new useful products and services to the market and society.

From that, other authors also advocate for innovation models that not only run from the concept of product innovation. The models by Utterback (1970), Pugh (1991), Thomas (1993) and Levy (1998) emphasize the market as a source of ideas for the development of new products and processes. Cooper (1993, 1994, 2008), Khurana and Rosenthal (1998), Goffin and Mitchell (2010), Rozenfeld et al. (2006) and Coral et al. (2008) highlight the organizational strategy as driving element to the beginning of the process, related to organizational strategy. Furthermore, models such as by Khurana and Rosenthal (1998), Goffin and Mitchell (2010) and Bessant et al. (2005) emphasize that the strategy must be the guiding principle of the whole process, giving a systemic meaning to the concept (Silva, Bagno, & Salerno, 2014).

In the current business world, innovation as discipline has not yet reached the stage of development able to satisfy the need to innovate. It appears that in many companies where innovation is considered important, the need exceeds the capacity (Bruce & Birchall, 2009; De Bes & Kotler, 2011; Sigala & Chalkiti, 2015). This is due to the innovation process being characterized as discontinuous and irregular, with concentration of innovation outbreaks, which will influence differently the various sectors of the economy in certain periods. In addition to not following a linear pattern, continuous and regular, innovations also have a considerable degree of uncertainty, since the solution of the problems and consequences of resolutions are a priori unknown. It reveals, however, a cumulative basis, given that the ability of a company to make changes and improvements within an established standard is strongly influenced by the characteristics of the technologies used and the experience accumulated in the past, which shows a strong influence of knowledge management in the process of innovating (Rowlei, 2000; Song, Zhu, & Rundquist, 2014).

Clark and Wheelwright (1993) proposed a classic model that seeks to understand the process of innovation as a key to the acquisition, development and application of technology for competitive advantage, the development funnel. The premise of this model is based on the theory of selectivity (Silva et al., 2014), in which many ideas go through phases of selecting and cutting, and only the most promising ones become products in the market.

In the same line of thought, aiming project selection, the model by Chesbroug (2006) arose, highlighting the idea of open organization. This author defended with his model the vision that innovation comes not only from people, knowledge, processes and internal capabilities of the organization, but also that the boundaries must be broken so that the knowledge and opportunities come to add to the internal process, in the same extent as it seeks new markets (Friesike, Widenmayer, Gassmann, & Schildhauer, 2014).

In the paradigm of Open Innovation, organizations become able to respond quickly and flexibly to changes in the environment, remain competitive and do not lose the market time of products and technologies’ life cycle. Cooperation with universities, research centers and new entrepreneurs is a great asset to improve and expand innovation strategies in a variety of organizations (Chesbroug, 2006).

More focused on the bias of the processes that culminate in innovations, Sawhney, Wolcott, and Arroniz (2006) presented his model named radar innovation, in which it is understood that the innovation of a business is not only the innovation of a product, but mainly of its values. Given that, the innovation of a business can be divided into four quadrants: product, customer, process and place.

Methodology

This research is classified as quantitative as to the nature, descriptive and exploratory about the objectives and, on the technical procedures, we opted for a multi-case study with three companies, all of these large organizations of the mechanical engineering sector of Rio Grande do Sul, located in the Serra Gaucha, Central and South Regions, now called only A, B and C.

For the development of the evaluation of knowledge management and innovation management factors and organizational performance determination, eight methodological steps were mapped, assessing internal aspects of business management and also external factors, as shown in Fig. 1.

Fig. 1.

Methodological research steps.

Source: Authors.

Then, we assigned variables that would be able to demonstrate, in the end, the degree of innovation performance of each of the companies, being structured in two depth levels: factors (F_i) and criteria (C_u), as shown in Fig. 2. To survey these factors and criteria, the principles defended by authors such as Rodan and Galunic (2004), Zogbi (2008), Freitas Filho (2013), Song et al. (2014) were used.

Fig. 2.

Hierarchical structure for evaluation of development.

Source: Authors.

To get the value that measures the organizational performance of each evaluated company, two factors will be used: Knowledge Management (F₁) and Innovation Management (F₂) that can be determining factors for the competitiveness of organizations (De Bes & Kotler, 2011; Terra, 2012). The criteria used are able to gather the key factors of performance (KPIs) to measure the results obtained in each of the companies separated by evaluation teams.

Knowledge management variables (F₁) measure the organization's ability to use internal and external expertise to improve its performance. The people criterion (C₁) addresses how the organization acts in people management to improve knowledge management, the structure criterion (C₂) measures how the company invests and uses the necessary infrastructure for knowledge management and the process criterion (C₃) measures if the company processes are focused on knowledge management.

In this same logic, the variables of innovation management (F₂) measure the organization's ability to turn their efforts toward innovation as well as the results it obtains focusing on that. Organizational alignment criteria were raised (C₄) in this factor, whose function is to measure if the organization has its strategy aligned with innovation objectives; support and organizational resources (C₅) seeks to measure whether the resources allocated in the organization are sufficient for the innovation development; innovation process (C₆) measures whether the innovation process is well defined and efficient; and the behavior and mental model (C₇) aims to understand how the company culture favors innovation.

Given the seven criteria for modeling, it is possible to determine KPIs capable to measure performance as explained in Table 2, using the principles espoused by Davenport and Prusak (2003), Prahalad (2008), Zogbi (2008), Santos, Basso, and Kinura (2012), Oliveira et al. (2012), and Ferreira (2012).

Table 2.

Description and purpose of KPIs.

KPIi	Definition	Purpose
KPI1	Incentives for knowledge generation	To measure how much incentive the organization provides for people to generate knowledge
KPI2	Roles centered in knowledge management	To measure whether the company uses specific roles to handle knowledge management
KPI3	Recruitment and selection based on skills	To represent how people's skills are taken into consideration when hiring
KPI4	Knowledge searching mechanisms	To show that the company has mechanisms to search the knowledge generated internally and externally
KPI5	Intranet	To check the use of tools such as intranet in the dissemination of knowledge
KPI6	Internal knowledge available for decision	To check that the internal knowledge is available and organized to assist managers in decision-making
KPI7	Knowledge documentation	To represent the formalization level of knowledge
KPI8	Performance measurement	To observe if the company has the habit to measure the results of its actions
KPI9	Knowledge transference between departments	To measure the level of knowledge transference between departments and areas
KPI10	Clear focus on innovation	To show if the organization has well-defined focus and if it is oriented to innovation
KPI11	Widespread organizational objectives	To check how widespread the organization's objectives are
KPI12	Financial resources for innovation	To measure if the financial resources for innovation are sufficient
KPI13	Adequate infrastructure to generate value	To measure if the infrastructure for innovation is suitable
KPI14	Clear process for innovation	To demonstrate the existence of a clear flow for innovation development
KPI15	Ideas from several departments	To check that the ideas of the different teams are moving forward
KPI16	Innovative organizational culture	To observe if the organizational culture is focused on innovation
KPI17	Partnership with universities	Represent the degree of involvement with universities and research centers

Source: Elaborated by the authors.

Regarding the development of modeling, developed mathematical equations based on multiple criteria methodology to support the decision, as explained by Gomes and Gomes (2012), will be presented to follow. From the approach to the only criterion of synthesis, it is first necessary to prepare the overall objective function (BOBJ) in order to be able to express the company's situation in relation to the context. The same was established from a mathematical model provided by Eq. (1),

which is necessary for both checking the condition of the two factors (knowledge management and innovation management), (F_i) considered for the measurement of context, as shown in Eq. (2)

where W_i is the relevance of each criterion in relation to the whole; NC_u is the total amount of characteristic criteria of i and C_u, which are the criteria disposed in the second level of the hierarchical structure, measured from the definitions proposed by Eq. (3)

Therefore, the determination of dos C_u is directly related to the result obtained by measuring the KPI_i, ∀i∈{1,2,…,n}, generated according to the metrics established at the time of definition of KPIs and following the math proposal described by Eq. (4), designed using the α range, based on the Likert, which used a 5-point Likert Scale, starting with a scale with a minimum of 1 and a maximum of 5 with intermediate values 2, 3 and 4 capable to transmit to the interviewee's opinion regarding the indicators, ∀e∈{1,2,…,n},

being the opinions expressed from the values assumptions in βe, according to the variation limits proposed by α. As the determination of weights W_y, according to Eq. (5), it was proposed to use standard techniques to faithfully represent the preference of the decision maker, such as Swing Weighting and Simple Attribute Rating Technique (SMART), based on the assumptions of Guitouni and Martel (1998), Clemen and Reilly (2001) and Poyhonen and Hamalainen (2001), so that the first, laid down by Edwards (1971), performs this process by the direct decreasing order of importance for each one of them, for which is usually assigned for the worst placed a value of 10 and, from this, values are listed increasingly, according to the degree of discrepancy in the existing behavior between them (Figueira, Greco, & Ehrgott, 2005; Galarza-Molina et al., 2015).

As for the technique named Swing Weighting, the logic of value assignment occurs in reverse, starting from the same ordering system, identifying which has greater relevance, adopting the value 100 and performing this same process to the remaining as far as reaching a value able to resume the less relevant item in relation to the hierarchical system level in question, as the difference found in each range is set again according to the characteristics of both (Gomes & Gomes, 2012),

where W is relative to each of the companies to be verified, ∀x∈{1,2,...,n}   and   y={i,u} related to a generic representation of the levels considered for the representation of weights of the elements in the hierarchical structure, which were determined by obtaining the values related to the calculation of multi-criteria Swing Weights methods (R_y1) and SMART (R_y2), based on the opinion of decision makers, according to Eqs. (6) and (7),

being F_y1 and F_y2 related to proportional scores obtained, b corresponding to the order of each factor given by the interviewed and FR_y2 directly related to the scale factor ∂, based on the determination of values for each of those according to a Likert scale ranging from a difference minimum of 1 to a maximum of 10.

And to become possible to perform comparisons of the results in all the methodological steps, it is necessary to set the goal for each of these according to the determination of a value that has as a behavior the increasing or the decreasing proportional variation, related to the expected level of rigor to measure that; in this case, it was suggested based on the agreement with the company managers as equal to 90%.

For the companies’ diagnostic step, a diagnosis instrument was applied through a structured closed interview consisting of 17 questions, each one related to the KPIs and held with the executive directors, also considered in this research as decision makers. Together, they attributed the necessary deliberations for carrying out the relativization of its values to the factors and criteria. The data obtained through the diagnosis were transferred to a database through Microsoft Office Excel^® software.

Results and discussions

In order to test the proposed methodology, the research was applied in mechanical engineering companies of Rio Grande do Sul, given that in this industry innovation and knowledge management have become essential for the competitiveness and also because of this sector's relevance, which is rather evident in the economy.

In Table 3, there is the position of managers in relation to the weights assigned to each of the factors (F_i) and criteria (C_u) raised, getting to the objective function. Therefore, the managers observed the proposed variables for the development of the research, diminishing the importance of each one for the whole in the measurement.

Table 3.

Weights assigned by managers.

Factors	Company A	Company B	Company C	Total
F1	50%	35%	45%	43%
F2	50%	65%	55%	57%

Criteria	Company A	Company B	Company C	Total
C1	10%	15%	25%	17%
C2	20%	5%	10%	12%
C3	5%	15%	10%	10%
C4	20%	35%	10%	22%
C5	30%	10%	15%	18%
C6	5%	10%	10%	8%
C7	10%	10%	20%	13%

Source: Elaborated by the authors.

A sort of balance was evident between the weights of the two factors, with a slight emphasis on the factor (F₂), Innovation Management, which can happen because of the greater dissemination of concepts related to innovation, since knowledge management is still not consolidated as a management focus of many organizations.

In the criteria relativization, we can highlight the (C₄) criterion, Organizational Alignment, which had the highest relative importance in the managers’ opinion, demonstrating a concern about the importance of the strategic issues involving innovation. The criteria (C₅), Organizational Support and Resources, and (C₁) also stood out positively from the rest, while criterion (C₆) had the lowest relevance rate for the performance of organizations surveyed.

From the data obtained through the diagnosis applied to the proposed model for the KPIs, it was possible to get the result for each factor. Initially, we observed the performance of each company by the evaluated criteria as it can be seen in Table 4, given the weights for each factor. The criterion (C₄), Organizational Alignment, was the one that performed better, pointing to a strategic mobilization of the surveyed companies regarding management of innovation.

Table 4.

Outcome of the criteria.

Criteria	Company A	Company B	Company C
C1	73.3	80.0	100.0
C2	86.7	93.3	86.7
C3	73.3	93.3	73.3
C4	100	100	90
C5	60	80	100
C6	90	90	90
C7	80	60	90

Source: Elaborated by the authors.

The worst performance was observed in criterion (C₇), related to Behavior and Organizational Model, pointing to a certain fragility of the surveyed companies in building corporate culture focused on innovation and the dissemination of organizational knowledge.

Finally, we came to the overall result regarding the performance for each of the companies surveyed, characterized as objective function, as shown in Table 5.

Table 5.

Results.

	Knowledge management	Innovation management	Objective function	Goal
Company A	82.55%	82.55%	82.55%	90%
Company B	87.61%	89.23%	88.66%	90%
Company C	91.11%	92.72%	92.00%	90%
Average	87.09%	88.17%

Source: Elaborated by the authors.

Observing the results obtained from the application of the proposed methodology, it can be observed that among the three organizations surveyed, only company C obtained performance above the stipulated goal of 90%. In both factors researched, company C managed to get positive results, showing effectiveness in knowledge management practices and commitment to innovation management, numbers that reflect its competitive position in the market.

Carvalho et al. (2015) observed various industry sectors in the light of the Innovation Radar and found results that placed the sectors of civil construction, agribusiness and mechanical metal as less innovative among those surveyed, which may explain the fact that only one of the companies surveyed has reached the set target.

Company B, despite not having reached the designated goal, shows itself quite close to it, with 88.66% of overall performance in the research questions. It is observed that in the innovation management factor, the company has a higher index than in the knowledge management one and almost reaches the goal, demonstrating that small adjustments in the company's management can put its performance above the goal.

On the other hand, company A had the lowest overall performance index, further from the goal, and hence less competitive than the other researched in the knowledge management and innovation management factors. The application of this methodology will be able to help company B to find the criteria and KPIs with lower rates in order to act on these and thus increase its performance and become more competitive.

By analyzing the average performance of each researched factor, it is understood that they are balanced but do not reach the goal, which can demonstrate that the researched sector still has to improve its practice and outcomes in the areas of knowledge management and innovation management, as it shows that company C is above average and ahead of the others.

Conclusion

Amid the new economic context that is characterized by the increasing competition in various sectors, the changes in customer attitude, society, competitors, employees and other stakeholders have contributed to increase competitiveness in organizations and innovation has become the key factor for companies’ survival. Similarly, the management of organizational knowledge has proved to be decisive for achieving objectives and competitive advantage, since the knowledge of organizations and their experiences have made the decision-making easier, precise and assertive.

In this context, this study showed that it is possible to measure aspects taken as intangible, such as innovation management and knowledge management, so that we can know more precisely on which competitive level the company is, through a specific methodology that takes into account the key indicators of measuring that performance.

The application of the proposed method was performed with 3 big companies from different regions of the state of Rio Grande do Sul, all belonging to the mechanical engineering sector. It showed the different levels of importance given by managers for each item of the methodology and also pointing out the most important index for the criterion (C₄), Organizational Alignment, whose function is to measure the level of alignment between the strategy adopted by the organization dedicated to innovation.

It was evidenced by the study that only company C reached the performance goal proposed by the methodology, placing itself in front of the others when it comes to innovation management and knowledge management aspects. The study also showed that the companies A and B are still on their way to achieve the proposed goal and may use the survey results to guide actions to improve their indicators.

The main limitation of this research is conditional approach to the management only in the internal and structural levels, not being contemplated, therefore, systemic variables. The absence of systemic aspects of modeling, such as taxes, laws, culture and social aspects is due to the fact that these conditions are very similar in all companies of the sector researched, with little or no distinction between them. In addition, the systemic aspects cannot be controlled by the companies, which prevents organizations to formulate strategies or earmark resources to increase innovation and knowledge management in these factors, leaving them only monitoring the external situation. Thus, the fact that the evaluation has taken place only in the business and structural levels alleviates the effects of possible economic downturns, political or social on the results presented, as these occur in the system level.

It is necessary to say that the performance measurement methodology proposed could be used in future studies and is intended to serve as a management tool for companies from the mechanical engineering sector as well other sectors in which innovation management and knowledge management are shown determinants.

Conflicts of interest

The authors declare no conflicts of interest.