Considering the absence of standards for culture collections and more specifically for biological resource centers in the world, in addition to the absence of certified biological material in Brazil, this study aimed to evaluate a Fungal Collection from Fiocruz, as a producer of certified reference material and as Biological Resource Center (BRC). For this evaluation, a checklist based on the requirements of ABNT ISO GUIA34:2012 correlated with the ABNT NBR ISO/IEC17025:2005, was designed and applied. Complementing the implementation of the checklist, an internal audit was performed. An evaluation of this Collection as a BRC was also conducted following the requirements of the NIT-DICLA-061, the Brazilian internal standard from Inmetro, based on ABNT NBR ISO/IEC 17025:2005, ABNT ISO GUIA 34:2012 and OECD Best Practice Guidelines for BRCs. This was the first time that the NIT DICLA-061 was applied in a culture collection during an internal audit. The assessments enabled the proposal for the adequacy of this Collection to assure the implementation of the management system for their future accreditation by Inmetro as a certified reference material producer as well as its future accreditation as a Biological Resource Center according to the NIT-DICLA-061.
According to the International Vocabulary of Metrology (VIM), reference material (RM) is the material that is sufficiently homogeneous and stable with reference to specified properties, which have been established to be fit for its intended use in measurement or in examination of nominal properties. The VIM defines certified reference material (CRM) as reference material accompanied by documentation issued by an authoritative body, which provides one or more specified property values with associated uncertainties and traceability, using valid procedures.1
For validation methods, interlaboratorial comparisons and quality control, reference materials can be used. In validation methods, the accuracy in parameter estimation must be performed with certified reference materials, when available. Certified reference materials should be used to provide values to other materials and for calibration in order to assure traceability. However, if no certified reference material is available, reference materials must be used for assigning values to other materials and for calibration.2
The use of reference material supports the laboratories to comply with the requirements of ISO 17025:2005 (General Requirements for the Competence of Testing and Calibration Laboratories), regarding validation methods and use of internal and external quality control.3 Ideally, the reference material must be produced in accordance with a management system that has been audited and acknowledged by a third party.2
It is essential that the reference material producers demonstrate their technical and scientific competence as a basic requirement to ensure the quality of the material produced. The competence of reference material producers will be demonstrated based on specific guides. The ISO 30 Series Guides gives directions on best practices in the production and characterization of reference materials. According to international guidelines, accreditation of reference material producers is based on the requirements of ABNT ISO Guide 34:2012 – General Requirements for the Competence of Reference Material Producers4 along with the requirements of ABNT NBR ISO/IEC 17025: 2005.5
Biological Resource Centers (BRCs) include microbiological collections from various fields, whose routine activities include the acquisition, characterization, authentication, preservation and distribution of biological material with the highest standard of quality. These activities generate information of scientific and technological interest on the deposited material in the collection, which are stored in a database. The definition of Biological Resource Centers (BRCs) as a key component in the scientific and technological infrastructure of life sciences and biotechnology was presented for the first time in 2001 by the Organization for Economic Cooperation and Development (OECD) in the publication Biological Resource Centers: Underpinning the Future of Life Sciences and Biotechnology.6 Later, OECD published in 2007 the Best Practice Guidelines for Biological Resource Centers.7 This document encourages collections to meet the high-quality operational standards required today, providing the basic rules and practical guidance for culture collections candidates to become biological resource centers.8
Therefore, collections that follow these Guidelines are able to achieve best practices in acquisition, maintenance and provision of biological materials to assure biological material with high standard of quality and authenticity, generating a reliable database in different laboratories and contributing to the protection of the health of staff, the public and the environment.7
As a response to the OECD initiative on BRC, in 2008 the demonstration project for the Global Biological Resource Centers Network (GBRCN) was initiated. The project recommended that all stakeholders should implement policies to ensure that the biological material supplied worldwide is of high quality. The design of GBRCN was established to improve access to high-quality biological resources, associated information and services.9
Brazil, which has been implementing the Brazilian BRC Network (Br-BRCN), has been following these Guidelines, as well as many other countries that are also part of GBRCN. Since there is no international standard for culture collections neither for BRCs, some countries have established internal rules mainly based on ISO 9001 that specifies requirements for a quality management system, such as France.8,10
Considering this scenario and due to the concern of the Brazilian Government regarding quality assurance within culture collections, the General Coordination for Accreditation (Cgcre) of the National Institute of Metrology, Quality and Technology (Inmetro), which is the official accreditation body in Brazil, created the Technical Committee for the Study of an Accreditation Model for the Activities of Biological Resource Centers (TC-BRC). These Technical Committee, constituted by experts from various culture collections and from other relevant areas from Brazilian Institutions that compose the Br-BRCN, was responsible for discussing and proposing a model of accreditation for BRCs.11
Thus, the TC-BRC elaborated and approved the first edition of the document on the accreditation program requirements for BRCs, the NIT-DICLA-61 (Requirements for the accreditation of testing laboratories and reference material producers from BRC), which is a document of the Division of Accreditation Laboratory-DICLA/Inmetro and that was published in the end of 2012.12
The NIT-DICLA-061 is based on the accreditation by Cgcre of BRC testing activities, using the ABNT NBR ISO/IEC 17025 (General Requirements for the Competence of Testing and Calibration Laboratories) as criteria or of BRC as a reference material producer using the ABNT ISO Guide 34:2012 (General Requirements for the Competence of Reference Material Producers) in combination with ABNT NBR ISO/IEC 17025 as criteria, and both possibilities complemented by the applicable requirements of the OECD Best Practice Guidelines for BRCs.10,11
Oswaldo Cruz Foundation (Fiocruz), which was one of the institutions whose representatives participated in the TC-BRC, is a Brazilian federal organization from the Ministry of Health. Fiocruz coordinates a well-structured network of epidemiological control and public health, hosts different culture collections holding strains of archaea, bacteria, fungi and protozoa, representing part of the Brazilian microbial diversity and expressing a variety of infectious agents related to different tropical diseases.13
In 2007 Fiocruz started a project, funded by the Ministry of Science, Technology and Innovation (MCTI), that had the objective of preparing its various institutional microbial collections to be part of the future Fiocruz Health-BRC,13 including the Fungal Collection that constitutes the Collection of Reference Microorganisms on Health Surveillance (Fiocruz-CMRVS), registered on the World Federation for Culture Collections (WFCC) under the World Data Center for Microorganisms (WDCM) number WDCM 575.
Within this context, in 2010 a study comprising the evaluation of the Fungal Collection based on the standard ABNT NBR ISO/IEC 17025:2005 and the OECD Best Practice Guidelines for BRCs was conducted to check the conformity to these documents in order to guarantee the implementation of the requirements, aiming at reliability, traceability and quality of microbial strains provided and the adequacy of the Collection to the Fiocruz Health BRC Project.14
The result of this study was the accreditation of viability, purity and authentication tests performed by the Fungal Collection, in 2012, by Cgcre in accordance with the requirements of ABNT NBR ISO/IEC 17025:2005, making it the first Brazilian Collection to have accredited tests. This step was essential to the conduction of the present study.
Therefore, the objective of this work was to assess the Fungal Collection according to the criteria of ABNT ISO GUIA 34:2012 correlated with the ABNT NBR ISO/IEC 17025:2005 as well as to evaluate it as a Biological Resource Center (BRC) according to the requirements of the Brazilian norm NIT-DICLA-061.
Materials and methodsThe evaluation of the Fungal Collection was performed by applying a checklist designed to facilitate the implementation of ABNT ISO GUIA 34:2012 (General requirements for the competence of reference material producers). The decision to use this checklist was to avoid that any requirement is overlooked at the time of evaluation, considering that this norm is not part of the scope of the management system of the National Institute for Quality Control in Health (INCQS/Fiocruz), one of the Fiocruz Institutes and where the Collection is located, neither is it part of the INCQS routine audits. An example of the structure of the checklist is presented (Table 1).
Example of the structure of the checklist applied in the Fungal Collection (Fiocruz-CMRVS).
| Item ABNT ISO GUIDE 34 | Correspondent item ABNT ISO/IEC 17025:2005 | Definition | Requirements | Meet, does not meet, partly meet the requirements and does not apply Describe the evidences of complience |
|---|---|---|---|---|
| 4 | 4 | Organization and management requirements | – | – |
| 4.1 | 4.2 | Management system requirements | – | – |
| 4.1.1 | 4.2.1 | General | The reference material producer shall establish, implement and maintain a documented management system appropriate to the scope of its activities, including the type, range and volume of the reference material production it undertakes. |
In addition to the checklist, an internal audit was conducted in order to evaluate the necessary documents and to follow the production of reference materials. All data and evidences collected during the internal audit were recorded in the Annexes of SOP No. 65.1120.043 (Internal Audit). The internal audit was an essential complement to the checklist in order to formalize the process within the INCQS Management System.
Whereas the checklist application and the internal audit were conducted at different times, and since both procedures were part of a sampling process, the results might be different for the same requirement.
For each result, the checklist presented the following options to answer: meet the requirements, does not meet the requirements, partly meet the requirements and does not apply. For each requirement complied, the evidences found during the application of the checklist whenever applicable were listed. In some cases besides the evidences, additional information was inserted in the last column of the checklist.
The checklist was applied during interviews with the Collection staff conducted by auditors. Besides the checklist interviews, the critical analysis of clauses 4 – Requirements of the Management System and 5 – Production and Technical Requirements of ABNT ISO GUIA 34:2012 was also conducted. The evidences were recorded on the checklist itself. The internal audit was based on the ABNT NBR ISO 19011:2012 (Guidelines for auditing quality and/or environmental management system). The audit used the criteria from the ABNT ISO GUIA 34:2012 and ABNT NBR ISO/IEC 17025:2005.
The audit of the Collection started in the last week of January 2013 and was finalized at the beginning of February 2013. During the audit the production of Cryptococcus gattii INCQS 40324 (CFP 78MC3) as a pilot batch was followed. The production of C. neoformans INCQS 40323 (CFP 78MC2D), also as pilot batch, was followed by the analysis of records related to this production.
The NIT-DICLA-061 was applied in the Fungal Collection, after the internal audit, by assessing all the requirements that were answered by indicating meet the requirements, does not meet the requirements, partly meet the requirements and does not apply in a specific report. This report was divided into the NIT-DICLA-061 items:
- •
10 – Application of ABNT NBR ISO/IEC 17025 standard for the accreditation of testing laboratories from BRC;
- •
11 – Additional requirements to ABNT NBR ISO/IEC 17025 standard for the accreditation of testing laboratories from BRC;
- •
12 – Application of ABNT ISO GUIA 34 in combination with ABNT NBR ISO/IEC 17025 standard for the accreditation of reference material producers from BRC;
- •
13 – Additional requirements to ABNT ISO GUIA 34 in combination with ABNT NBR ISO/IEC 17025 standard for the accreditation of reference material producers from BRC.
The assessment was based on the analysis of non-conformities identified during the internal audit and based on the results of the checklist application. The result of the NIT-DICLA-061 application was also examined.
The proposal for the adjustments of the Fungal Collection was designed considering the results of these assessments.
ResultsResults from the internal audit on ABNT ISO GUIA 34:2012The audit identified nineteen non-conformities regarding non-compliance with the requirements of ABNT ISO GUIA 34:2012.
Fig. 1 shows the 19 non-conformities identified during the audit by type of requirements of ABNT ISO GUIA 34:2012.
Results regarding the application of the checklist of ABNT ISO GUIA 34:2012 requirements
The application of the checklist allowed the identification of 45 (22%) items that did not meet the requirements from a total of 209 items. On the other hand, 95 (45%) items met the requirements, 44 (21%) items partly met the requirements and 25 (12%) items did not apply to the Fungal Collection (Fig. 2).
Application of NIT-DICLA-061
The result from the NIT-DICLA-061 application, regarding item 10 – Application of ABNT NBRISO/IEC 17025 standard for accreditation of testing laboratories from BRC was 15 (48%) items that met the requirements from a total of 31itemsrequirements, in addition to 7 (23%) that partly met the requirements, 8 (26%) items that did not meet the requirements and just 1 (3%) that did not apply to the Fungal Collection (Fig. 3).
For item 11 – Additional requirements to ABNT NBR ISO/IEC 17025 standard for the accreditation of testing laboratories from BRC there were a total of 10 requirements, from which 4 (40%) items met the requirements, 2 (20%) partly met, 3 (30%) did not meet the requirements and 1 (10%) did not apply to the Fungal Collection (Fig. 3).
For item 12 – Application of ABNT ISO GUIA 34 in combination with ABNT NBR ISO/IEC 17025 standard for the accreditation of reference material producers from BRC there were a total of 44 requirements, from which 30 (68%) items met the requirements, 6 (14%) items partly met the requirements and 8 (18%) items did not meet. There was no requirement that did not apply to the Fungal Collection (Fig. 3).
For item 13 – Additional requirements to ABNT ISO GUIA 34 in combination with ABNT NBR ISO/IEC 17025 standard for the accreditation of reference materials producers from BRC there were a total of 17 requirements. From these, 6 (35%) items met the requirements, 5 (30%) items partly met the requirements and 6 (35%) items did not meet the requirements. Again in this case there was no requirement that did not apply to the Fungal Collection (Fig. 3).
DiscussionConcerning the internal audit on ABNT ISO GUIA 34:2012, the number of non-conformities related to Production and Technical Requirements (clause 5) was higher than the number related to Requirements of the Management System (clause 4). Probably this result is due to the management system implemented in the Fungal Collection, as well as the fact that the tests for purity, viability and authentication performed by the Collection are accredited according to the requirements of ABNT NBR ISO/IEC 17025.
During the process of producing CRM (clause 5) it was observed that most of the non-conformities were related to the lack of procedures concerning CRM production. In relation to clause 4, most of the non-compliances were related to adequacy/development of procedures/records as well as aspects referring to organization and definition of quality policy.
Regarding the non-conformities by type of requirements of ABNT ISO GUIA 34:2012, the largest number of non-compliances were related to Production Planning (item 5.4), 5 (27%) out of 19 non-conformities (Fig. 1). Despite the fact that the production planning was available, it was not present in the Management System and did not meet certain requirements, therefore it needed amendments. The requirement regarding Management System (item 4.1.3) had three non-conformities and that was due to the absence of some standard operating procedures (SOPs).
The items that did not meet the requirements reflects the need for the adequacy of the INCQS Management System to the organization and direction requirements of the ABNT ISO GUIA 34:2012, as also observed during the execution of the internal audit. Items that were not applicable were related to some requirements on metrological traceability, validation method developed by the laboratory and a specific requirement, with various sub-items, on non-certified reference material.
By analyzing Fig. 2 it is clear that the Fungal Collection met or partly met most of the requirements of ABNT ISO GUIA 34:2012. From the items that did not meet the requirements, most of them referred to the adequacy/preparation of documents related to the production, besides a subcontracting issue. Within the INCQS Management System the subcontracting is not described, however for the CRM production the characterization and transport will have to be subcontracted, therefore requiring proper documentation.
In relation to the application of the NIT-DICLA 061, the non-compliance concerning item 10 (Application of ABNT NBR ISO/IEC17025 standard for accreditation of testing laboratories from BRC) (Fig. 3) referred mostly to issues related to internal audit and to critical analysis, as well as to biosecurity, incident response plan and definition of level for visitor access in accordance with the Best Practice Guidelines on Biosecurity for BRCs from the OECD Best Practice Guidelines for BRCs.
The requirements partly met by the Collection referred to adjustments needed in the Management System mainly related to air monitoring, quarantine and information on the validity of the culture media, as well as other aspects concerning control system for visitors, information on access level, definition of a biosecurity agent, and the need of a strict control of the chain of custody, which are also in accordance with the Best Practice Guidelines on Biosecurity for BRCs from the OECD Best Practice Guidelines for BRCs.
Biosecurity is a new topic in Brazil, which has been discussed by the Federal Government and by research institutes, such as Oswaldo Cruz Foundation, where pathogens of high risk are the object of research. However there is neither specific legislation nor regulations for this issue in the country yet. Currently, the only formal document that deals with this in Brazil is the NIT-DICLA-061, to the best of our knowledge. The lack of official legislation and regulations on biosecurity within national level justifies in some extent the non-compliance by the Collection regarding this subject.
For item 11 (additional requirements to ABNT NBR ISO/IEC 17025 standard for the accreditation testing laboratories from BRC) most of the requirements were met or partly met by the Fungal Collection (Fig. 3). The requirements that were not met referred to the issues related, in general, to the incident response plan and security screening of the staff members.
Requirements that the Collection partly met were related to adjustments needed in the Management System of the Collection, such as preparation of a health and safety plan as well as the control of information of the biological material, both aspects are related to biosecurity.
Most of the requirements related to item 12 (Application of ABNT ISO GUIA 34 in combination with ABNT NBR ISO/IEC 17025 standard for the accreditation of reference material producers from BRC) were met (Fig. 3). The requirements that were not met referred to aspects related, in general, to training on biosecurity, regular risk assessment, biological material delivery, internal audit and critical analysis. Whereas the requirements that were partly met by the Collection concerned quarantine, suitability of the environment, visitor control, documenting inventory control, adequacy of the documentation provided to ABNT ISO GUIA 31 (Reference materials- Contents of certificates and labels) and control of biological material supply. There were no requirements that did not apply.
For item 13 (Additional requirements to ABNT ISO GUIA 34 in combination with ABNT NBR ISO/IEC 17025 standard for the accreditation of reference materials producers from BRC) the Collection met or partly met most of the requirements (Fig. 3). The requirements that were not met were related generally to visitor control and incident response plan, which were already identified previously, and data exchange. While the requirements that the Collection partly met referred to information on access level, definition of a biosecurity agent, strict control of the chain of custody, which are aspects already identified regarding item 11, besides risk assessment of the information and database.
Based on this assessment conducted in the Fungal Collection by internal audit and checklist application a proposal for its adequacy to ABNT ISO GUIA 34:2012 was prepared focusing mainly on management of the production process.
The organizational structure of INCQS will have to change in order to insert the reference material producer and to define responsibilities for the production of reference materials. Additionally, the quality policy of INCQS will have to be revised to cover other issues related to ABNT ISO GUIA 34:2012.
Following these modifications, all documents of INCQS Management System will have to be modified concerning its scope in order to include the requirements of ABNT ISO GUIA 34:2012. Some of the technical procedures will have to be reviewed and others will have to be established in order to comply with ABNT ISO GUIA 34:2012.
Analyzes of root causes will have to be performed and the corrective actions relating to non-conformities of the internal audit report 002/2013 will have to be determined. Consequently, the proposed corrective actions will have to be implemented.
Regarding the checklist application, the items that did not meet and partly met the requirements will have to be evaluated and corrective actions also will have to be determined.
After the implementation of all these proposed corrective actions, the effectiveness of these actions will have to be verified by a new internal audit in the Fungal Collection having as criteria the ABNT NBR ISO/IEC 17025:2005 and ABNT ISO GUIA 34:2012, including all technical requirements. For a better result, it is suggested the inclusion of a statistical expert and a mycologist in the audit team.
Only after evaluating the results of this new audit will be possible to forward the necessary documentation to Cgcre/Inmetro for the request of the Fungal Collection accreditation as certified reference material producer in the biological area. And if this Collection is accredited it will be a pioneer in Brazil in this modality.
Based on the analyzes of the results obtained by the application of NIT-DICLA-061 a proposal of corrective actions was also set in order to prepare the Collection for the accreditation by Cgcre/Inmetro of the testing and reference material production activities performed by the BRC and, consequently, within the scope of the Fiocruz Health BRC project. Therefore, the steps defined above were also followed for the Fungal Collection adequacy regarding NIT-DICLA-061.
The compliance of most of the requirements, such as the implementation of policies, procedures and forms for records, depends on the action of the Fungal Collection itself, many of these adjustments are already in course. However, issues of infrastructure and personnel availability for the production of reference materials, both aspects considered weaknesses, deserve special attention by the Direction of INCQS.
Regarding NIT-DICLA-061 application, there were some difficulties in understanding some of the items. Since this was the first time that the NIT-DICLA-061 was applied in a microbial collection, the critical analysis of all its requirements allowed the verification of difficulties of understanding and of implementation. Therefore, this study will help TC-BRC to improve this new norm and to work in its new version.
From this assessment it is also concluded that the issues related to Biosecurity are considered critical aspects and deserve special attention from all parties involved, including the Direction of INCQS, the Fiocruz Administration, TC-BRC and the Brazilian Federal Government.
The implementation of these documents in the Fungal Collection and its accreditation as a reference material producer in the biological area, will bring confidence to the results issued by institutions that use the certified reference materials produced by this Culture collection as well as the other services provided as BRC, contributing significantly to the Brazilian scientific research, epidemiological surveillance and biotechnological development, besides reducing dependency on imported certified reference materials. The importance of this autonomy must be considered regarding the high costs of biological reference materials importation and the technical and sanitary barriers.
Additionally, this work can be a pilot for the production of other certified reference materials in the biological area in the country, since there is no certificate reference material producer accredited in Brazil. It also could be used by other collections and laboratories from Fiocruz and by other public and private institutions of Brazil that have interest in producing reference materials in the biological area.
Due to the confidential nature of the audit, the corrective actions designed for the nonconformities detected during this study were not described in this paper. However, the putative solutions for the non-compliances are provided in supplementary material that include template of the documents used in this work, which can be requested by email to the address vdquali@incqs.fiocruz.br.
Conflicts of interestThe authors declare no conflicts of interest.
The authors thank Renata Martins Horta Borges for the fundamental insights in moments of doubt.