Application of maturity assessment tools in the innovation process: converting system's emergent properties into technological knowledge

Abstract

This paper aims at establishing a theoretical construction between the concept of learning-by-using and the concepts of Technological Readiness Level (TRL) and System Readiness Level (SRL). The concept of learning-by-using reveals that the technical change that takes place in complex systems is given by the sum of small improvements in many different technological disciplines integrated in a specific configuration of this system. This kind of learning results from the iterative combination of scientific and technological knowledge, which is generated by the extensive use of products and their associated production processes. A stock of this combined knowledge might be required to cope with emergent properties of complex systems. The pattern of complex systems evolution involves the balance of technological and scientific frontiers as well as the fulfillment of customer expectation. Every innovation involves systemic uncertainty, which is positively correlated to the magnitude of the change introduced into the complex system. Maturity level of technological solutions allows organizations to assess pragmatically strategic risk exposure of implementing complex system innovation. The concept of SRL represents a proficuous tool to unveil emergent properties, which consider both the TRL of individual elements and how they are integrated into a complex system.

Introduction

Innovation is by nature a complex process, that is to say, a process that comprises a large number of variables of various different kinds. Variables involve not only the natural laws and measurable dimensions (Kline & Rosenberg, 1986), but also abstract or intangible variables such as: a low maturity of various technologies and their inter-relationships; managerial characteristics; and the relationships between the areas involved in the innovation process; and even areas inside and outside the organization not directly involved in the innovation process. Considering this innovation process as a complex system project (Hobday, 2000), it is necessary to consider those that will be affected by the project, and even the system's operational environment (Zandi, 2000). Another aspect that brings more complexity to these highly dynamic projects is the large number of elements involved in the innovation process, which constantly change their characteristics (Sterman, 1992). Thus, it is possible to note that the relationship between complexity and uncertainty of an innovative complex system project brings huge challenges to its decision-making process.

In the classical behavior theory, it is considered that the decision-making process are based entirely on rational principles which seek to optimize processes, i.e., utility maximization, but the fact that innovation is complex and uncertain means that it is not possible to achieve maximum return on each activity, so the adoption of the theory of bounded rationality in this case is positive: “However the strong positive case for the classical theory replacing by the model of bounded rationality begins to emerge when we examine their situations involving decision-making under uncertainty and imperfect competition” (Simon, 1978, pp. 349). It is important to consider that the capacity of mental models is limited since it is impossible to understand or analyze all the possibilities in a complex system, thus changing the focus of the decision-making process of utility maximization to the search for satisfactory results to achieve projects main goals, sacrificing or ignoring some aspects of the problem in this process (Simon, 1978).

Despite all the fantastic qualities of a mental model like the flexibility; ability to deal with information of different natures and constantly adapting, their weaknesses are also notable; it is limited. Mental models are not explicit; those cannot be examined or evaluated by others; it is difficult to see their premises; the same phenomenon interpretations may vary by observer; and also contradictions and ambiguities may remain unresolved in these models (Sterman, 1992).

The weaknesses of mental models become even more relevant when one is dealing with complex systems projects. The large number of information requires that of decisions different areas are taken by their respective experts. From this perspective, the need for tangible models that can be evaluated by the group involved in decision-making becomes clear. Therefore the models must overcome the limitations of mental models. Thus they must have the following characteristics: be explicit; its premises should be prone to those involved in the review and revision; and they allow the simultaneous connection between many different factors of the project (Sterman, 1992).

This paper proposes a theoretical construction to bridge the concept of technological maturity (Mankins, 2009) to the chain-linked innovation model (Kline & Rosenberg, 1986). This theoretical construction aims at increasing the ability of managers to understand the nuances and subtleness of the innovation process in order to provide decision-making yardsticks that cope with the uniqueness of complex systems projects.

The need for a better understanding of the innovation in complex systems projects is given by the fact that today the industry and academia expend a lot of resources developing technologies, but just a small fraction of these technologies reach the commercial success incorporated into products (Atkinson, 1999). A great deal of them remains in academia as a scientific demonstration or becoming a commercial failure after a costly process of technological development.

System readiness assessment

The design of an innovative system depends on the evolution of technical knowledge, “The development of new functionalities of a system typically depends on a previous successful advanced technology research and development efforts” (Mankins, 2009, p. 1216). Systemic and rigorous assessment of the understanding level or expertise of the organization in front to a new technology allows risks mitigation in a project, assisting the project manager in prioritizing resources for the development of critical technologies that prove to be immature at an early stage of the project.

If we adopt a low maturity technology that potentially may solve project's problem when it became fully developed, it represents a low cost at that point. But what should be considered is that it represents a high commitment of budget proportion in the later phases of the projects, as show in Figure 1 (Forsberg, Mozz, & Cotterman, 2005), in the early stages of the project, such as the system concept review, the project will have spent around 1% of the total budget, but will have committed approximately 70% of the total. By the time of preliminary design review, 85% of project funds will be committed, changes in the architecture in this stage have a deep impact in the project success given that there is no space in the budget for new developments. Thus, the expenditure profile proposed by Forsberg et al. (2005) show in Figure 1, exposes a major concern in evaluating the maturity of the technologies involved in the project architecture as early as possible to access the risks and, by consequence, the opportunities involved in the project.

Figure 1.

Typical profile of expenses in a project: committed versus spent.

Source: Forsberg et al. (2005).

To assess the maturity of a technology the TRL (Technology Readiness Level) methodology was developed in the seventies by NASA, which currently consists of a rating of nine levels shown in Table 1. The evaluation is done through a list of requirements that qualify technology to the next level; the level assigned to technology is the highest level that has the requirements met (Mankins, 2009). This methodology is widely accepted and applied, and spread to the most diverse branches of developed economies.

Table 1.

NASA technological readiness levels.

TRL	Definition
1	Basic principles observed and reported
2	Technology concept and/or application formulated
3	Analytical and experimental critical function and/or characteristic proof of concept
4	Component and/or breadboard validation in a laboratory environment
5	Component and/or breadboard validation in a relevant environment
6	System/subsystem model or prototype demonstration in a relevant environment
7	System prototype demonstration in an operational environment
8	Actual system completed and qualified through test and demonstration
9	Actual system proven through successful mission operations

Source: Mankins (2009)

Complex systems depend on the technological evolution in several and concomitant disciplines. These technologies will be integrated in a specific configuration so that these systems achieve its goals through the matching of the features derived from these technologies. However this integration of disciplines cannot generate accidental effects that affect the purpose of the system mission itself “Yet, the emergence of large complex systems created through the integration of diverse technologies has created the need for a more modern maturity metric” (Sauser, Gove, Forbes, & Ramirez-Marques, 2010). These are the emergent properties that comes from the interaction between system's elements in the operational environment (Hobday, 1998; Zandi, 2000). Systemic uncertainty (Rosenberg, 2006) stems from emergent properties of complex systems (Turner & Cochrane, 1993). Thus it is also necessary to improve the understanding of the interrelationships between these technologies and their implications for the system as a whole. Sauser, Forbes, Long, and Macgrory (2009) proposes the use of a scale, that is similar to the TRL, to evaluate in a scale of nine levels the maturity of integration as show in Table 2, the IRL (Integration Readiness Level), evaluating the integration of a technology with each other peer-to-peer.

Table 2.

Integration readiness levels proposed.

IRL	Definition
9	Integration is mission proven through successful mission operation
8	Actual integration is completed and mission qualified though test and demonstration, in the system environment
7	The integration of technologies has been verified and validated with sufficient detail to be actionable
6	The integrating technologies can accept translate, and structure information for its intended application
5	There is sufficient control between technologies necessary to establish, manage, and terminate the integration
4	There is sufficient detail in the quality and assurance of the integration between technologies
3	There is compatibility (i.e., common language) between technologies to orderly and efficient integrate and interact
2	There is some level of specificity to characterize the interaction (i.e., ability to influence) between technologies through their interface
1	An interface between technologies has been identified with sufficient detail to allow characterization of the relationship

Source: Sauser et al. (2009)

Having the TRL levels of the technologies used in a given system, and the IRL of these technologies with each other, Sauser, Ramirez-Marquez, Magnaye, and Tan (2008) propose the indicator of the maturity of the system architecture, SRL (System Readiness Level), the calculation consist in arranging the TRL values of technologies in a vector n × 1 (1), and IRL values in a n × n matrix (2).

The vector containing the SRL values is obtained by multiplying the vector TRL by the array IRL, then dividing each value by n, each SRL vector value represents the level of maturity of each technology in relation to the rest of the system, and then it is possible to calculate the total maturity level of the system architecture (3).

For a more effective comparison, Sauser et al. (2008) suggest the use of normalized values. Table 3 presents the proposed correlation between normalized SRL values and stages of life cycle of a project.

Table 3.

System maturity levels.

SRL	Acquisition phase	Definitions
0.90–1.00	Operations & Support	Execute a support program that meets operational support performance requirement and sustains the system in the most cost-effective manner over its total lifecycle
0.70–0.89	Production	Achieve operational capability that satisfies mission needs
0.60–0.79	System Development & Demonstration	Develop system capability or (increments thereof); reduce integration and manufacturing risk; ensure operational supportability; reduce logistics footprint; implement human systems integration; design for production; ensure affordability and protection of critical program information; and demonstrate system integration, interoperability, safety and utility
0.40–0.59	Technology Development	Reduce technology risks and determine appropriate set of technologies to integrate into a full system
0.10–0.39	Concept Refinement	Refine initial concept; develop system/technology strategy

The extensive use of the maturity assessment methods has proven their reliability as indicators of risk in a project. But likewise this use has exposed its weaknesses, such as: the lack of attention to feedback processes in development; the individual approach to technology; and the specificity of the contexts in TRL scales.

Learning-by-using

Innovation takes place through numerous small learning processes (Rosenberg, 2006). The model traditionally adopted to represent an innovation process, the linear model, considers the innovation process as a sequential chain of events as shown briefly in Figure 2. This model treats the flow of information as a one-way path, that is, scientific research feeds the development which in turn feeds the production, and never the other way. The linear model assumes that the innovation process occurs smoothly and continuously. This version of the linear model also believes that innovation is fostered only by scientific research. Kline and Rosenberg (1986) state that “In an ideal world of omniscient technical people, one would get the design of the innovation workable and optimized the first team. In the real world of inadequate information, high uncertainty, and fallible people, nothing like this happens” (Kline & Rosenberg, 1986, p. 286). In fact this statement is in line with the bounded rationality assumptions, as stated by Simon, in his classical Nobel Prize Lecture “And the failures of omniscience are largely failures of knowing all the alternatives, uncertainty about relevant exogenous events, and inability to calculate consequences. There was needed a more positive mechanism of choice under conditions of bounded rationality” (Simon, 1978, p. 356).

Figure 2.

Linear model.

In the real world, the innovation process bears little resemblance to the linear model, “Innovation is complex, uncertain and somewhat disorderly, and subject to changes of many sorts” (Kline & Rosenberg, 1986, p. 275). The chain-linked model presented in Figure 3, proposed originally by Kline (1985) was expanded and discussed by Kline and Rosenberg (1986), covers the main complexities involved in an innovation process. In this model, scientific knowledge is not the impeller of the innovation. On the contrary, there is the prospection of the needs of the market that boost scientific research. In this model, the feedback loops are taken into account; the search is no longer part of the main innovation chain and is present in all stages of the process. In this model, research acts when the stock knowledge does not meet the requirements of the innovation process.

Figure 3.

Chain linked model.

The chain-linked model stresses the idea that “the project needs to be conceptualized as a history-dependent and organizationally-embedded unity of analysis” (Engwall, 2003, p. 790). While a project is a temporary effort of an organization to achieve a goal specified (Lundin & Soderholm, 1995; Shenrar & Dvir, 2010), a project-based organization cannot be treated in the traditional project life cycle. The organizational and project learning that results from the innovation process needs to feed the organization's knowledge base (Cicmil, Williams, Thomas, & Hodgson, 2006), reducing the risks arising from system's complexity.

Emergent properties of complex systems

In any system, the whole is greater than the sum of its parts, that is to say, the aggregated explanation of the component parts of a system, does not explain all aspects of the resulting system's behavior. We consider emergent properties aspects of the interaction between different technologies, and between the system and its environment. In fact, the emergence does not mean that the collective behavior cannot be captured by the behavior of the part, it does mean that the collective behavior can be captured by the behavior of the part only if it is examined in the context, or in other words, with the component integrated and functioning in the full system in the operational environment (Bar-yam, 2003).

Emergency not always brings complexity to the system, in some cases, the whole is simpler than the parts. This phenomenon is known as emergent simplicity (Bar-yam, 2003). A useful example is the solar system, the planets orbiting the sun have a quite simple and predictable behavior, despite possible disturbs like comets or asteroids. However the Earth as a single system is extremely complex, impossible to predict precisely how many important phenomena will behave in the near future.

Unfortunately the emergent simplicity is quite rare when it comes to innovation regarding complex systems development. The commonplace in project management is the opposite situation: emergent complexity (Bar-yam, 2003).

Most of these aspects are unattainable with traditional analysis, and can compromise the effectiveness or even the operation of a system. In the case of highly complex systems composed of innovative technologies, the likelihood of emergent properties increases as we expand the spectrum of technologies and environmental variables (Zandi, 2000). Thus it is essential to have in hands a tool that allows a synthetic analysis of the risks associated with complex systems innovation process.

There is in the literature a variety of ways to define the complexity of a system. Attempts to define the complexity were already present in astronomy for millennia and more recently in biology, economics, psychology and various other branches of science. Simon (1996) proposes the study of complexity as a phenomenon in itself and not necessarily a characteristic of a particular complex system. However, the complexity appears to be a very general phenomenon and therefore do not have much content itself. Therefore, complex systems classes may be the focus of attention on the study of complexity. It is possible to list several critical factors which definees the character of an innovative product and its complexity, independent of the factors or the rates used, it is certain that the innovation will always bring high complexity to the subject (Hobday, 1998).

Considering that the emerging complexity of a system represents in some ways the inability to predict ex ante the behavior of a system in its operating environment (both the inner functioning of the system and the elements of the operating environment), we can say that complexity of a system is directly related to the integration maturity level of its technological elements and therefore the system architecture maturity (Sauser et al., 2008). Systems whose architecture integration features are highly known and controllable are also very predictable, so should be considered with lower systemic uncertainty, since cause-and-effect relationships are already mapped. On the contrary, a system whose characteristics are little known and therefore not fully controllable are difficult to predict and should be treated as highly uncertain from the systemic standpoint, since cause-and-effect relationships are not clear, they are subject to causal ambiguity (Chagas & Campanario, 2014).

IRL and SRL as systemic uncertainty indicators

The IRL/SRL methods cover the major aspects of complexity that brings uncertainty to the project. First, the size of the system heavily influences the indicator. However, as the maturity of the architecture increases, the systemic uncertainty is progressively unveiled. Second, emergent properties are evaluated in each technological interconnection inside the system; a diligent maturity assessment will predict ex ante a large portion of the unwanted emergent properties. Third, this method can evaluate interaction of any nature, so SRL is a common language to communicate amongst very distinct areas. At last but not at least, this method can evaluate the interaction between the system and the operational environment. The limit to determine all the emergent properties lays just upon the bounded rationality of the managers and experts.

In summary, maturity levels in the system do not represent directly the level of complexity of the system, yet they represent the uncertainty that emerges from complexity aspects embedded in the system, namely systemic uncertainty (Rosenberg, 2006). And through this method, these levels can be easily translated into manageable risks (Chagas & Campanario, 2014)

Discussion

The evaluation of TRL is an effective tool to assess risk derived from key immature technologies present in the scope of a project. But the TRL methodology consists in dividing the system into components and the evaluation of these individually, characteristic of an analytical process. However, when dealing with complex systems projects, the interaction between its technological elements may lead to unpredicted and unwanted emergent properties (Chagas & Campanário, 2014), which represent some degree of immaturity of the system architecture. This question added to the non-linearity feature of these systems poses challenges of major extent to managers and to the decision-making processes in these projects (Sterman, 1992).

The synthetic aspect of the IRL and SRL methodology allows visualizing the risk of unpredicted and unwanted emergent properties very consistently. But this methodology is still in line with the linear model, that is to say, it is expected that a particular investment of resources bring the technologies, their interrelationships and consequently the system architecture from a less mature level to a more mature level through a research process and development. However, when dealing with a real system, there are temporal environmental conditions which can cause a development to stagnate, regress or even being canceled, turning the project a failure. For example: technological dead-ends; loss of market interest; technological obsolescence. Thus, it is essential that the maturity assessment methodologies incorporate the knowledge gained in the manufacture and operation of both the project itself and previous projects. This proposal indicates that the system maturity assessment can be used as a tool with a reach far more effective than the universalistic approach of traditional management. It should consider organizations and projects as open systems, dependent on the organizational history (Engwall, 2003). The chain-linked model consistently describes these complexities of the innovation process.

Given these limitations, this paper proposes a theoretical construction that represents the union of the chain-linked model and maturity assessment tools shown in Figure 4. In this construction, the paths of the innovation process present in the chain-linked model are associated with the need to mature all the technologies involved and their integration into a specific configuration of a system architecture. The assessment of maturity must be presented throughout the process in order to indicate the need and the criticality of the research for the progressive maturity of system development.

Figure 4.

Chain-linked model including readiness level constraints.

The direct link between research and invention, represented as “D” in Figure 3, refers to immature technology identified as critical. At this point it is important to set priorities and deadlines for technological developments that may impact within the project deadlines and will be developed along the innovation process, here the contact between the project manager and the researchers is essential and should guide the initial development of system's architecture.

The process represented by the arrow “1” in Figure 5 emerges from the needs identified in the technological readiness assessment process. The link “K” represents the search for mature available solutions. At this point, contact with research institutes, academia, or other technology research organizations is essential.

Figure 5.

Relation between system readiness level and innovation process stages.

Exhausted the technological options available to solve the problem, is explicit the need for technological research, this process represented by the arrow “2” in Figure 5 goes through a decision-making process that must take into account the criticality of this technology, time for development, costs involved and the ability the development organization. At this point, it may be judged in a worst case scenario the unfeasibility of the project.

The link “3” in Figure 5 represents a technological maturation process in the research environment, that is to say, a process which can be verified by the increase in TRL, or in a broader view, the better understanding of the relationship between technologies (IRL).

Deliveries of the research process to the main chain of innovation, represented by the arrow “4” in Figure 5, do not necessarily occur simultaneously or completely, partial deliveries allow the project management updates in its architecture, review deadlines and priorities.

The feedback functions to the research environment with data from operational environment, represented in the model by links “I” (machines, tools and procedures) and “S” (operation) in Figure 3, incorporate into the model the concepts of “learning by doing” and “learning-by-using” respectively. This flow of information is vital for achieving the highest level of technological maturity.

Obviously at the time of an initial assessment of maturity, the functional architecture of the system may not contain all the necessary functions at the end of the innovation process, other functions can make necessary throughout the project, requiring a new solutions research cycle or development. However, it is expected that the repetition of this cycle becomes increasingly less frequent depending on the maturity of the system architecture. A large number of new technologies identified in the advanced stages of the innovation process reflect an inconsiderate definition of system architecture, increasing the risks to the end of the project.

Another opportunity that this theoretical construction brings to the innovation process is the ability to establish a reliable metric through the SRL indicating favorable times for the progress between the innovation stages. It cannot, for example, advance from the invention stage to the detailed design stage before the SRL has reached a level of maturity compatible to the challenges that will be imposed to the project in the detailed design and test stage.

The relationship between the stages of innovation process in the chain-linked model and maturity levels of systems proposed by Sauser et al. (2008) is shown in Figure 5.

It is possible to note that at the stage of invention, there will be intensive consultation to the stock of knowledge. This expected behavior signals the need for a joint effort between the project manager and the experts in the areas of interest at this stage, this cooperation should be strengthened and facilitated to reduce the need for research and development.

In the next stage, the detailed design, it is expected that the search for development of immature solutions be more solicited, since in this stage the search for mature solutions is exhausted and be limited to the need of new features identified as the system architecture maturation occurs.

The intensity of the information flow and learning can then be assessed in all phases of the innovation process and serve as a basis for guiding future projects for better structuring of the organization (Cicmil et al., 2006), this growing knowledge about the organization capabilities also lead to a better understanding uncertainty sources, improving the risk management process (Ward & Chapman, 2003). In complex system innovation, the organization are subjected to a dynamic environment and this understanding enable the flexibility required to cope with the environment behavior (Chagas & Cabral, 2010).

Case study description: VSB-30 sounding rocket

The VSB-30 sounding rocket was developed by Institute of Aeronautics and Space (IAE) associated with German Aerospace Center (DLR) as a substitute to the former Skylark 7 (Palmério, Roda, Turner, & Jung, 2005). VSB-30 is a two stage solid propulsion spin stabilized without active control capable of realizing microgravity environment experiments, this rocket was developed based in the previous VS-30 rocket consisting of the first stage of the well-established SONDA III rocket to deliver a small payload to the microgravity environment (Palmério, Silva, Turner, & Jung, 2003).

The VSB-30 comprises a S30 motor that was used in theVS-30 na SONDA III; a S31 motor as a booster, that is a shorter version of the S30 with a faster burning propellant (Carvalho, Damiani, Follador, & Guimarães, 2012); a fin set to stabilize the flight; a spin-up system that induces rotation after the lifting; and an event sequencer.

Case study methodology

This case study was realized in March 2015 and consisted of the literature review, including papers, restricted access reports and interviews with experts involved in the development of VSB-30. The aim of this research was to look for empirical evidence for supporting the theoretical construction herein proposed. The supporting information required to corroborate the maturity level as used by the U.S. Department of Defense are described in Table 4 (DoD, 2011).

Table 4.

TRL supporting information.

TRL	Supporting information
1	Published research that identifies the principles that underlie this technology. References to who, where, when
2	Publications or other references that outline the application being considered and that provide analysis to support the concept
3	Results of laboratory tests performed to measure parameters of interest and comparison to analytical predictions for the critical subsystems. Reference to who, where and when these tests and comparisons were performed
4	System concepts that have been considered and results from testing laboratory-scale breadboard(s). References to who did this work and when. Provide an estimate of how breadboard hardware and test results differ from the expected system goals
5	Results from testing laboratory breadboard system are integrated with other supporting elements in a simulated operational environment. How does the “relevant environment” differ from expected operational environment? How do the test result compare with expectations? What Problems if any were encountered? Was the breadboard system refined to more nearly match the expected system goals?
6	Results from laboratory testing of a prototype system that is near the desired configuration in terms of performance, weight, and volume. How did the test environment differ from the operational environment? Who performed the tests? How did the test compare with expectations? What problems, if any, were encountered? What are/were the plans, options, or actions to resolve problems before moving to the next level?
7	Results from testing a prototype system in an operational environment. Who performed the tests? How did the test compare with expectations? What problems if any, were encountered? What are/were the plans, options, or actions to resolve problems before moving to the next level?
8	Results of testing the system in its final configuration under the expected range of environmental conditions in which it will be expected to operate. Assessment of whether it will meet its operational requirements. What problems if any, were encountered? What are/were the plans, options, or actions to resolve problems before finalizing the design?
9	OT&E reports

Source: DoD (2011)

The IRL evaluation criteria used consists of seventh nine items proposed by Sauser et al. (2009) and should be proved through main integration and risk evaluation documents like interface control document (ICD), integration plan, concept of operations and integrations test reports.

Results

The technology maturity assessment of the VSB-30 technologies reveals that this rocket has highly mature technologies. The rocket completed fifteen successful missions already (IAE, 2015), mission reports data indicates that the architecture components worked individually as expected. Indeed using the TRL method in the subsystems, its found that the all subsystems completed the highest level of maturity (Table 5).

Table 5.

VSB-30 subsystems technological readiness levels.

Subsystems	TRL
Motor	9
Booster	9
SIR	9
Fins	9
Event sequencer	9

It is important to note that the S30 motor was already in a high level maturity once it has been used extensively by other rockets; the booster is a shorter derivation of the S30 with distinct propellant characteristics to provide a fast burn and hence greater impulse, the S31 went through the development cycle, being subjected to the necessary ground tests and finally proved in actual missions; the rotation induction system (SIR) was being developed to be used in another project, lacking flight test results by that time, the system performed his function in the VSB-30 as expected.

Similarly, applying the method proposed by Sauser to assess the IRL, the system shows a high integration maturity level, however, to achieve the highest rank in IRL metric, and there is a need for an elevated comprehension of the relationship between the components and the properties that emerges from these relations. Its noteworthy that the booster, the SIR and the fins form an assembly that have only mechanical interactions with the main motor, also the event sequencer is responsible by the main motor ignition and does not have direct interaction with the booster, fins or the SIR.

The IRL scale goes beyond the successful mission operation, and there is a need to the organization to master the failure rates, the relation between operational costs and client benefits, and other knowledge originated in the operational environment after a large number of missions.

The VSB-30 IRL values presented in Table 6 reveal the necessity to keep studying and increasing the system architecture maturity. This necessity emerges from the data that obtained in the operational environment that shows that the dispersion of payload impact point, even attending the initial requirements, could be improved, allowing safer operation in smaller isolation area, easier payload retrieval (Garcia et al., 2011). The dispersion of impact point is relevant in the operation in Esrange launch site in Sweden, where the payload impact occurs in ground and a large dispersion area represents safety risks, and in Alcantara launch site in Brazil, where the payload fall in the water, the larger dispersion area means a larger maritime isolation effort and a difficult retrieval mission.

Table 6.

VSB-30 IRL values.

IRL	Motor	Booster	SIR	Fins	Sequencer
Motor	9	8	8	8	8
Booster	8	9	8	9	9
SIR	8	8	9	8	9
Fins	8	9	8	9	9
Sequencer	8	9	9	9	9

Applying the calculation proposed by Sauser, the vector SRL (4) is found and the total SRL calculated is 0.94. These values reveal that the system as a whole is highly mature, however, there are some opportunities to improve even further the comprehension about the system function and the influence of its parts in the trajectory and thus the impact point dispersion.

These information that arise from operational environment, represents de “f”, in Figure 3, where the operational data drives improvements in the project itself, and also represents the “F” flow where the data feeds the organization base of knowledge to future developments toward client's needs and operational efficiency.

Conclusion

The theoretical construction proposed in this article allows one to visualize the maturity assessment methodologies as management tools are able to consolidate in the research environment the technological knowledge obtained in the operating environment. To do so, it is required that the innovation process adopts the evaluation of maturity as a guide to the decision-making. On the other hand, from the viewpoint of maturity assessment, the adoption of chain-linked innovation model makes it necessary to adapt this methodology adding requirements that arise from feedback processes for maturity assessment. We argue herein that maturity assessment should be increasingly considered as environmental context-dependent.

The learning-by-using approach adds to the innovation process the effects of feedback loop of operational environment data to research and development environments. This approach enriches technology assessment, adding flexibility to the process. While traditional TRL method is a one-way path, the chain-linked model allows the development to receive inputs from the operational environment and the market.

The adoption of this model allows the generation of valuable knowledge that stems from subtle complex systems dynamics that may not be realized straightforwardly. Not only the quality and the intensity of the flow of information within the organizational structure is assessed, but also allows the management team to be aware of the sources of these needs, the critical points and priorities of technological development. This theoretical construction allows the use of both models in the development of innovative complex systems projects bringing objectiveness as important decision-making yardsticks.

Another relevant contribution of the theoretical construction herein presented is to reveal the necessity of reporting operational data in the most accurate possible way, so the organization will be able to improve the system efficiency and direct the future projects toward an architecture to satisfy the client's needs.

Conflicts of interest

The authors declare no conflicts of interest.