Quality of systematic reviews of COVID-19 in people with diabetes: A systematic review

Abstract

Objective

To investigate the methodological quality of published systematic reviews of factors associated with COVID-19 in people with diabetes.

Method

Systematic review with registration protocol in PROSPERO, under the number CRD42020222418. Searches were carried out from October to November 2020 in the databases of the MedLine, Web of Science, Scopus, Lilacs, Embase and Cochrane libraries, in addition to searching the reference list of the selected studies. Systematic review studies with or without meta-analysis and without date and language restrictions were included. Data were extracted in a standardized way and the quality of the studies was assessed using the Assessment of Multiple Systematic Reviews scale.

Results

Twelve reviews, published between 2020 and 2021, with a predominance of the English language, systematic reviews of observational studies with meta-analysis with a sample ranging from six to 83 studies, were included. Regarding financing, most of the study reported did not receive this type of support. Regarding to the assessment of the methodological quality of the studies, three were of moderate quality, five were classified as low quality and three with critically low quality.

Conclusions

The analyzed articles presented a bias in the preparation of reports on their studies, suggesting the need to use mechanisms to improve adherence to the established reporting guidelines and methodological evaluation tools.

Resumen

Objetivo

Investigar la calidad metodológica de las revisiones sistemáticas publicadas de factores asociados con COVID-19 en personas con diabetes.

Método

Revisión sistemática con protocolo de registro en PROSPERO, bajo el número CRD42020222418. Las búsquedas se realizaron de octubre a noviembre de 2020 en las bases de datos de las bibliotecas MedLine, Web of Science, Scopus, Lilacs, Embase y Cochrane, además de la búsqueda en la lista de referencias de los estudios seleccionados. Se incluyeron estudios de revisión sistemática con o sin metanálisis y sin restricciones de fecha e idioma. Los datos se extrajeron de forma estandarizada y la calidad de los estudios se evaluó mediante la escala Assessment of Multiple Systematic Reviews.

Resultados

Se incluyeron doce revisiones, publicadas entre 2020 y 2021, con predominio del idioma inglés, revisiones sistemáticas de estudios observacionales con metaanálisis con una muestra que va desde seis a 83 estudios. En cuanto a la financiación, la mayoría de los estudios informaron que no recibieron este tipo de ayuda. En cuanto a la evaluación de la calidad metodológica, tres fueron de calidad moderada, cinco fueron clasificados como de baja calidad y tres RS fueron evaluados como críticamente bajos.

Conclusiones

Los artículos analizados presentaron un sesgo en la elaboración de informes sobre sus estudios, sugiriendo la necesidad de utilizar mecanismos para mejorar la adherencia a los lineamientos de reporte establecidos y herramientas de evaluación metodológica.

What is known?

Covid-19 affects individuals in different ways, and the majority of the population who are infected has mild to moderate symptoms. Nevertheless, those with diabetes and/or other chronic diseases are more likely to develop more severe symptoms, and this requires correct clinical decisions.

What is added?

Reviews are recommended to follow the guidelines available in the AMSTAR 2 tool together with the PRISMA protocol, to guarantee higher quality scientific evidence.

Introduction

In December 2019 SARS-CoV-2 was identified as the cause of a high number of cases of acute respiratory syndrome in Wuhan, China. SARS-CoV-2, which caused Covid-19 disease, quickly spread around the world and became a worldwide medical emergency and pandemic.¹

Although the information about COVID-19 is evolving and not all of its clinical aspects have been completely elucidated, it causes a high rate of mortality in individuals with chronic underlying diseases. Within the current context, systemic arterial hypertension, cardiovascular diseases, diabetes mellitus (DM), renal diseases and chronic obstructive pulmonary diseases are among those which are the most prevalent in patients hospitalized due to COVID-19. In these individuals infections are able to progress swiftly, so that well-founded clinical decisions are required.^2,3

Given the high prevalence of DM and the high level of COVID-19 infections, research has been produced and published around the world with the aim of elucidating the gaps in knowledge about the association of these two pathologies and the relationship between risks and prognoses.⁴ It should be pointed out that the lack of large sample groups and high quality reports are the chief obstacles in this scientific research, particularly at the start of disease outbreaks, when evidence is mainly obtained from observational studies that are presumably influenced by numerous confusion factors and liable to possible methodological defects.⁵

Systematic reviews (SR) are evidence of high quality, and they consist of secondary studies which have the aim of bringing together all of the empirical evidence which fulfils predefined selection criteria in response to a specific scientific research question. This uses clear and systematic methods that are selected to minimize distortion factors, to offer more reliable results that can be used to draw conclusions and make clinical decisions.⁶

The aim of this study is to study the methodological quality of published SR on factors associated with COVID-19 in individuals with diabetes.

Methods

This is an umbrella study of SR. This review was undertaken using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) tool, and its protocol was registered and approved in PROSPERO (the International Prospective Register of Systematic Reviews) with registration number CRD42020222418.

PICO was used to prepare the question used in this study: “which factors are associated with COVID-19 infection in individuals with diabetes?” This acronym stands for Population/Intervention-Exposure/Comparison/Result).

Selection criteria

SR were included that contain research meta-analysis with human beings, together with original studies in the selected categories that used quantitative methods to study the association between diabetes and COVID-19. Editorials and expert opinions were excluded, as were case reports, narrative reviews, SR protocols, reviews that were not peer-reviewed, summaries or those with unavailable data and animal studies.

Data sources

The systematic search for studies started on 13 October and finished on 5 November 2020. However, a systematic update of the review was required on 28 February 2022 by two reviewers (JSM and MOM) simultaneously and independently, in the following electronic databases: Medical Literature Analysis and Retrieval System online (MEDLINE — PubMed), Web of Science, SCOPUS, Literatura de Ciencias de la Salud de América Latina y el Caribe (LILACS), Embase and the Cochrane Library. A search was also made in the references to papers that fulfilled the selection criteria in the search of grey literature. There were no restriction in the date or language of study publication.

Search strategy

A combined search strategy was implemented, using MeSH (Medical Subject Headings), DeCS (Health Science Descriptors) Emtree terms and free terms. The latter are terms that are not found in DeCS and MeSH, but which are relevant for the search, together with their synonyms (key words). The Boolean operators “AND” and “OR” were used, and the “reviews” filter was applied to each database, as is shown in Table 1.

Table 1.

Database search strategies.

Database	Search strategy
MEDLINE (via pubmed)	(((“Diabetes Mellitus”[Mesh]) OR (diabetes)) AND ((((((((((((“COVID-19” [Supplementary Concept]) OR (“2019 novel coronavirus disease”)) OR (“COVID19”)) OR (“COVID-19 pandemic”)) OR (“SARS-CoV-2 infection”)) OR (“COVID-19 virus disease”)) OR (“2019 novel coronavirus infection”)) OR (“2019-nCoV infection”)) OR (“coronavirus disease 2019”)) OR (“coronavirus disease-19”)) OR (“2019-nCoV disease”)) OR (“COVID-19 virus infection”))) AND (((((((((“Risk Factors”[Mesh]) OR (“Factor, Risk”)) OR (“Factors, Risk”)) OR (“Risk Factor”)) OR (“Population at Risk”)) OR (“Risk, Population at”)) OR (“Populations at Risk”)) OR (“Risk, Populations at”)) OR (“Associated factors”))
Web of Science	(TS=(((“Diabetes Mellitus”[Mesh]) OR (“Diabetes”)) AND ((((((((((((“COVID-19” [Supplementary Concept]) OR (“2019 novel coronavirus disease”)) OR (“COVID19”)) OR (“COVID-19 pandemic”)) OR (“SARS-CoV-2 infection”)) OR (“COVID-19 virus disease”)) OR (“2019 novel coronavirus infection”)) OR (“2019-nCoV infection”)) OR (“coronavirus disease 2019”)) OR (“coronavirus disease-19”)) OR (“2019-nCoV disease”)) OR (“COVID-19 virus infection”)) AND (((((((((“Risk Factors”[Mesh]) OR (“Factor, Risk”)) OR (“Factors, Risk”)) OR (“Risk Factor”)) OR (“Population at Risk”)) OR (“Risk, Population at”)) OR (“Populations at Risk”)) OR (“Risk, Populations at”)) OR (“Associated factors”)))
Scopus	TITLE-ABS-KEY (((“Diabetes Mellitus” [mesh]) OR (diabetes)) AND ((((((((((((“COVID-19” [supplementary AND concept]) OR (“2019 novel coronavirus disease”)) OR (“COVID19”)) OR (“COVID-19 pandemic”)) OR (“SARS-CoV-2 infection”)) OR (“COVID-19 virus disease”)) OR (“2019 novel coronavirus infection”)) OR (“2019-nCoV infection”)) OR (“coronavirus disease 2019”)) OR (“coronavirus disease-19”)) OR (“2019-nCoV disease”)) OR (“COVID-19 virus infection”))) AND (((((((((“Risk Factors” [mesh]) OR (“Factor, Risk”)) OR (“Factors, Risk”)) OR (“Risk Factor”)) OR (“Population at Risk”)) OR (“Risk, Population at”)) OR (“Populations at Risk”)) OR (“Risk, Populations at”)) OR (“Associated factors”))
Lilacs	((“Diabetes mellitus”) OR (Diabete) OR (“Diabete Melito”) OR (Diabetes) OR (“Diabetes Melito”)) AND ((“Infecções por Coronavirus”) OR (“Coronavirus Infections”) OR (“Infecciones por Coronavirus”) OR (“COVID-19”) OR (“Doença pelo Novo Coronavírus (2019-nCoV)”) OR (“New Coronavirus disease (2019-nCoV)”) OR (“Nueva enfermedad por coronavirus (2019-nCoV)”) OR (“Doença por Coronavírus 2019-nCoV”) OR (“Coronavirus disease 2019-nCoV”) OR (“Enfermedad del coronavirus 2019-nCoV”) OR (“Doença por Novo Coronavírus (2019-nCoV)”) OR (“New Coronavirus Disease(2019-nCoV)”) OR (“Nueva enfermedad por coronavirus(2019-nCoV)”) OR (“Epidemia pelo Coronavírus de Wuhan”) OR (“Epidemia de coronavirus de Wuhan”) OR (“Wuhan Coronavirus epidemic”) OR (“Epidemia pelo Novo Coronavírus (2019-nCoV)”) OR (“New Coronavirus epidemic (2019-nCoV)”) OR (“Nueva epidemia de coronavirus(2019-nCoV)”) OR (“Epidemia por Coronavírus de Wuhan”) OR (“Wuhan Coronavirus Epidemic”) OR (“Infecção pelo Coronavírus 2019-nCoV”) OR (“Coronavirus infection2019-nCoV”) OR (“Infección de coronavirus2019-nCoV”) OR (“Surto de Coronavírus de Wuhan”) OR (“Brote de coronavirus de Wuhan”) OR (“Wuhan Coronavirus Outbreak”)) AND ((“Fatores de Risco”) OR (“Risk Factors”) OR (“Factores de Riesgo”) OR (“Fatores associados”) OR (“Associated factors”) OR (“Factores asociados”) OR (“Fator de Risco”) OR (“Risk fator”) OR (“Factor de riesgo”) OR (“Fatores de Risco Biológicos”) OR (“Biological Risk Factors”) OR (“Factores de riesgo biológico”) OR (“Fatores de Risco Não Biológicos”) OR (“Non-Biological Risk Factors”) OR (“Factores de riesgo no biológicos”) OR (“Fatores de Riscos Biológicos”) OR (“Biological Risk Factors”) OR (“Factores de riesgo biológico”) OR (“Fatores de Riscos Não Biológicos”) OR (“Non-Biological Risk Factors”) OR (“Factores de riesgo no biológicos”) OR (“População em Risco”) OR (“At-Risk Population”) OR (“Población en riesgo”) OR (“Populações em Risco”) OR (“At-Risk Populations”) OR (“Poblaciones en riesgo”))
Embase	((‘diabetes mellitus’ OR ‘diabetes’) AND ‘coronavirus disease 2019’ OR ‘covid-19’ OR ‘sars-cov-2 infection’) AND ‘risk factor’ OR ‘associated risk’))
Cochrane library	(“Diabetes Mellitus” OR Diabetes) AND (“2019 novel coronavirus disease” OR “COVID19” OR “COVID-19 pandemic” OR “SARS-CoV-2 infection” OR “COVID-19 virus disease” OR “2019 novel coronavirus infection” OR “2019-nCoV infection” OR “coronavirus disease 2019” OR “coronavirus disease-19” OR “2019-nCoV disease” OR “COVID-19 virus infection”) AND (“Factor, Risk” OR “Factors, Risk” OR “Risk Factor” OR “Population at Risk” OR “Risk, Population at” OR “Populations at Risk” OR “Risk, Populations at” OR “Associated factors”) in Title Abstract Keyword

Study selection

The selection of studies involved two stages: the first stage consisted of initial screening based on reading titles and abstracts, to preselect the studies for consideration in this SR. The reviewers evaluated studies using the following criteria: the presence of the term “systematic review” and/or “meta-analysis” in the title or abstract, the use of scientific databases in searches and systematic data gathering techniques. If a study fulfilled at least one of the above criteria, or if one of the reviewers doubted whether it should be selected, the study was selected for the next stage of the selection process: complete reading.

In the second stage the reviewers obtained the preselected studies in electronic format to verify the selection criteria in the complete published version. The whole selection process was carried out independently by two researchers (JSM and MOM), while a third evaluator (FLSJ) resolved any disagreements. Rayyan⁷ software was used to manage references and eliminate duplicates.

Data extraction

The reviewers (JSM and MOM) extracted data independently, using a standardized formula which contains information on study identification (first author, year and journal where it was published), type of review, language, number of studies included and financial support.

Evaluation of the risk of bias

Three independent qualified reviewers (JSM, ATG and MOM) carried out the methodological evaluation of the SR, and differences between them were resolved by consensus meetings. The Assessment of Multiple Systematic Reviews (AMSTAR 2)⁸ instrument was used in this evaluation as a measurement tool to assess the SR. It is composed of 16 questions, with the following response options: “yes, no”; “yes, partial yes, no” or “yes, no, meta-analysis not performed”. Seven of these 16 questions were considered to be critical domains in which, based on critical and non-critical points, AMSTAR 2 generates a four-level classification of a SR: high, moderate, low and critically low. Each SR was qualified in the official AMSTAR 2 website (https://amstar.ca/Amstar_Checklist.php) using an online calculator.

AMSTAR 2 was developed to critically evaluate SR of intervention studies,⁸ and given the selected studies in this SR, five items of this instrument were adjusted⁹ to analyse SR which were did cover this type of design, as is shown in Table 2.

Table 2.

Adaptations of AMSTAR 2 items to enable analysis of systematic reviews which do not include studies with intervention.

Description of the original elements	Description of the adapted elements
Item 3: this requires reviewers to justify the inclusion of only random trials with controls (RSC) or the inclusion of only non-randomized intervention studies (NRSI), or both.	Item 3: the SR that do not include studies with intervention cannot have included RSC or NRSI because these designs involve intervention. Given this, the SR that justify the inclusion of studies based on specific designs or which justify the inclusion of studies with any design is considered positive. The SR that have neither of these justifications were considered to be negative.
Item 9: this is connected with the use of techniques to evaluate the risk of bias in the studies included in a SR. More specifically, this item makes it necessary to evaluate certain elements in the design of ECR and NRSI.	Item 9: the SR that do not include studies of intervention cannot have included RSC or NRSI. Nevertheless, these SR should have used a satisfactory technique to evaluate the risk of bias in the types of studies which they included. Thus the SR which described and justified the risk of bias in the studies included were considered to be positive, and the SR that did not do this were considered negative.
Item 11: this states that a meta-analysis was performed and that the reviewers used appropriate statistics to combine the results. This item therefore requires the evaluation of certain design elements in the ECR and NRSI.	Item 11: the SR that did not include studies of intervention cannot have included RSC or NRSI. Nevertheless, these SR should have used appropriate statistical techniques to construct their results. Therefore, the SR that described the appropriate methods for the statistics in the studies they included were considered to be positive, and the SR that did not do this were considered negative.
Item 12: this cover whether a meta-analysis had been performed, if the reviewers evaluated the potential risk of bias in the individual studies of the results of the meta-analysis. The instrument recommends a positive conclusion for: (1) SR that include only RSC with a low risk of bias, or (2) if the combined estimation is based on RSC and/or NRSI in the risk of bias variable, and the authors undertook and analysis to study the possible impact of the risk of bias on the estimations of the effect.	Item 12: the SR that did not include studies of intervention cannot have included RSC or NRSI. Nevertheless, these SR should have evaluated the potential impact of the risk on the results of the meta-analysis studies. Thus the SR that evaluated the impact of the risk of bias on the results of the studies included were considered positive, and the SR that did not do this were considered negative.
Item 13: this is about the inclusion of risk of bias analysis in the discussion of results. The instrument recommends that the following should be considered positive: (1) SR that include only RSC with a low risk of bias, or (2) SR that include RSC with a moderate or high risk, or NRSI with a discussion about the probable impact of the risk of bias on the studies included in the results.	Item 13: SR which do not include studies of intervention may not have included RSC or NRSI, although these reviews may discuss the probable impact of the risk of bias on the studies included in their results. The SR that used a satisfactory strategy to evaluate the risk of bias in the studies they included were therefore considered to be positive when they discussed the probable impact of this risk in their results, and the SR that did not do this were considered negative.

Results

Study selection

5652 references were found in the first search of the databases. 661 studies were excluded because they were duplicates. 95 manuscripts were preselected in the first analysis of titles and abstracts. After reading these documents in full, 83 studies were excluded due to the following reasons: not being a SR, studies that did not cover the subject in question and because they were incomplete reviews. Twelve studies were therefore included for the final analysis, as is shown in Fig. 1.

Figure 1.

Study selection process flow diagram.

Study characteristics

Table 3 shows the references published from 2020^10–15 to 2021,^16–21 with a sample that runs from six to 83 studies. English was the publication language, and all of the selected studies were characterized as observational study SR published in the following journals: Diabetes Research and Clinical Practice^10–19; Nutrition, Metabolism & Cardiovascular Diseases¹¹; Hormones¹²; MEDICINE¹³; Archives of medical research¹⁴; Diabetes & metabolic syndrome¹⁵; Acta Diabetologica^16,21; Diabetologia¹⁷; Journal of Diabetes and Metabolic Disorders¹⁸ and Frontiers in Endocrinology.²⁰ The majority of the studies reported that they had not received any financing.

Table 3.

Characteristics of the studies included in the systematic review.

First author	Journal	Year	Type of review	Language	Sample	Financing
Abdi et al.^10	Diabetes Research and Clinical Practice	2020	Observational studies	English	27	No
Mantovani et al.^11	Nutrition, Metabolism & Cardiovascular Diseases	2020	Observational studies	English	83	Yes
Palaiodimos et al.^12	Hormones	2020	Observational studies	English	14	No
Miller et al.^13	MEDICINE	2020	Observational studies	English	16	No
Shang et al.^14	Archives of Medical Research	2020	Observational studies	English	76	Yes
Hussain et al.^15	Diabetes & Metabolic Syndrome	2020	Observational studies	English	43	No
Wu et al.^16	Acta Diabetologica	2021	Observational studies	English	6	No
Schlesinger et al.^17	Diabetologia	2021	Observational studies	English	22	Yes
Saha et al.^18	Journal of Diabetes and Metabolic Disorders	2021	Observational studies	English	22	No
Yang et al.^19	Diabetes Research and Clinical Practice	2021	Observational studies	English	17	No
Kan et al.^20	Frontiers in Endocrinology	2021	Observational studies	English	18	Yes
Kaminska et al.^21	Acta Diabetologica	2021	Observational studies	English	19	No

When AMSTAR 2 evaluated the quality of the SR, three SR were moderate quality,^11,12,18 five SR were classified as low quality^{14,15,17,20,21} and three SR were classified as critically low quality,^10,13,19 as is shown in Table 4.

Table 4.

Evaluation of studies by the AMSTAR 2 instrument.

Code	Questions																Quality of evidence
	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16
A1^10	S	S	S	PY	S	S	PY	S	N	N	S	N	N	S	S	S	Critically low quality
A2^11	S	PY	N	S	S	N	PY	S	S	N	S	S	S	S	S	S	Moderate
A3^12	S	PY	N	S	S	S	PY	S	S	N	S	S	S	S	S	S	Moderate
A4^13	S	S	N	S	S	S	PY	S	N	N	S	N	N	S	S	S	Critically low quality
A5^14	N	N	N	PY	S	S	N	S	S	N	S	S	N	S	S	S	Low quality
A6^15	N	N	N	N	S	S	N	PY	S	N	S	N	N	S	S	S	Low quality
A7^16	S	N	N	PY	N	N	PY	S	S	N	S	S	S	S	S	S	Low quality
A8^17	N	S	S	N	S	S	S	PY	S	N	S	S	S	S	S	S	Low quality
A9^18	S	S	S	PY	S	N	S	PY	S	N	S	N	S	S	S	S	Moderate
A10^19	S	N	N	PY	N	S	N	PY	S	N	S	N	S	S	S	S	Critically low quality
A11^20	S	S	S	PY	S	S	S	S	S	N	S	N	N	N	N	S	Low quality
A12^21	S	N	S	PY	N	S	N	PY	S	N	S	S	N	N	N	S	Low quality

Legend: S (yes); N (no); PY (partial yes); includes only NRSI; includes only RSC; no meta-analysis; no meta-analysis was performed.

According to the evaluation by the instrument, nine^{10–13,16,18–21} studies used PICO components to prepare the research questions (Question 1), five^{10,12,17,18,20} studies reported that the protocol was registered (Question 2), and five ^{10,17,18,20,21} SR justified the selection of the designs they included (Question 3). Regarding the exhaustive bibliographical search (Question 4), only two^15,17 studies did not correspond to the aspects evaluated. Three^16,19,21 reviews did not report on the role of at least two reviewers in the selection of studies (Question 5), and for the extraction of information, three ^11,16,18 studies did not independently report on the extraction (Question 6). Although five^10–13,16 manuscripts supplied a list of excluded papers, they did not individually justify the exclusion of each potential study (Question 7). Nevertheless, of the included studies, the majority^{10–14,16,20} of the SR described the studies satisfactorily (Question 8).

Regarding the evaluation of the risk of bias in the included studies (Question 9), only two studies^10,13 failed to perform this evaluation. Regarding the source of financing of the studies included in the reviews (Question 10), none of the papers^10–21 reported this information. All twelve^10–21 SR used satisfactory statistical methods to analyse their results (Question 11). Six studies^{11,12,14,16,17,21} evaluated the potential impact of the risk of bias in the results of the meta-analysis studies (Question 12) and six studies^{11,12,16–19} discussed the probable impact of this bias on their results (Question 13). Two^20,21 manuscripts did not offer a satisfactory explanation of the heterogeneity found in the studies (Question 14), as well as an appropriate study of publication bias (Question 15). Finally, all of the studies^10–21 reported conflicts of interest (Question 16).

Discussion

SR is a type of research that follows specific protocols, seeking to understand and make sense of a documental “corpus”, especially analysing what functions and what does not in a certain context. SR there have a certain level of evidence, and this is classified as the gold standard for analysing the evidence for medical interventions. It is indispensable in professional decision-making and as a guide for clinical practice and health policies.^22,23

Due to the Sars-CoV-2 virus (COVID-19) pandemic, a notorious quantitative SR with or without meta-analysis was quickly published. This had the aim of showing the swift response of researchers and scientists to an emerging disease pandemic.⁵ However, complete SR require a lot of time, and they often take up to three years. Moreover, like any other study design they are liable to certain forms of bias which researchers should plan for and/or take into account with exactitude.²⁴

Given this context, the methodological quality of SR may have insufficiencies due to faults in their design, reporting and methodologies. Poor coordinated SR may lead to incorrect data on the efficacy of an intervention, or unrealistic conclusions and information about clinical approaches, leading to a waste of resources and a lack of applicability.^6,25

After the application of the AMSTAR 2 instrument in this study and detailed evaluation of the published reviews, the quality of the SR reports was found to be from moderate to critically low. Similarly, a study with the aim of offering suggestions about aspects of SR methodological quality during the COVID-19 pandemic found that the level of confidence was not satisfactory, and that in the majority of studies evidence quality was critically low.⁵

Regarding protocol development and registration, only three SR^17,18,20 give the protocol or information that was registered. Furthermore, in Overviews of Cochrane Systematic Reviews (OoR),²⁶ no protocols for two OoR were found in the Cochrane Library or in Archie, and this is one of the factors which limits a higher score for the studies that were analysed.

Including a protocol prior to starting a SR may help to optimize the use of finite resources by preventing unnecessary duplications and also permitting considerations on methodological quality and balanced study preparation, to reduce publication bias or the selective reporting of results. We therefore recommend that this protocol should be registered in a database which is open to other researchers and health managers, thereby demonstrating transparency in the SR execution process.^27,28

Respecting the explanation for the selections of study designs included in the review, five^{10,17,18,20,21} SR justified design selection. According to this study, the methodological evaluation undertaken with ten²⁷ SR found that only a small proportion (20%) of them explained the inclusion of design selection in the review study, so that this is an item which deserves appropriate attention by authors.

In the list of excluded studies and the justification of these exclusions, five^10–13,16 manuscripts supplied a list of excluded papers, but they did not justify the exclusion of each potential study. As a result of this a general review of the potential effect of dyslipidaemia on the severity of COVID-19 displayed critical failings, including not supplying a list of excluded studies and justifying these exclusions.²⁹ Luo et al.²⁷ consider that unjustified exclusion may distort the findings of a review, so that they suggest that authors should supply a list.

Respecting the inclusion of the risk of bias analysis in the primary studies included, six^{11,12,16–19} of the reviews studied offered relevant data and discussed their probable impact. They also described the use of statistical tests to evaluate bias: funnel graphs and Egger’s formal test.

Bias is defined as any distortion during the research process, and it may arise in any type of design. When the analysis of the risk of bias in the included studies is considered, the trustworthiness of the results of an SR increases.^24,28 The impact of potential bias should therefore be considered during the preparation of the results and conclusion of a SR, and this should be clearly reported as SR associated with bias may be understood to be poorly prepared, which would restrict their usefulness and application in clinical practice.²²

None of the papers offered information on the source of financing.^10–21 Proportionally, two other SR^27,30 on subjects other than this one also offered no information about the sources of financing of the studies included in their research. To guarantee the transparency and quality of the data that is collected, authors should report the sources of financing of the primary studies which are included, to reduce bias and ensure that readers are able to evaluate whether there is any assessment bias, based on the financing sources reported in each study within their review.

In the analysis of these SR it was found that the quality and reporting of the reviews published around the world vary widely, so that this may cause confusion in academics and researchers who are starting out in their scientific work. Tools have therefore been developed to help to verify the minimum quality criteria for reviews, and use of these tools should be encouraged while planning, executing and publishing research.³⁰

The limitations of this study include the lack of a specific instrument for the critical examination of SR of non-experimental studies, as this made it necessary to use adaptations of the AMSTAR 2 instrument to analyse the SR that did not include studies with intervention.

A result of this study is that it would be recommendable for future reviews to unrestrictedly follow the guidelines available in the AMSTAR 2 tool together with the PRISMA protocol, to ensure higher quality scientific evidence. Moreover, research should be undertaken to prepare for the creation of a specific instrument for the critical evaluation of SR of studies without interventions.

As final considerations, this study had the aim of analysing the methodological quality of published SR on factors associated with COVID-19 in individuals with diabetes. It found that the studies analysed were moderate to critically low in quality, suggesting the need for mechanisms to be used to improve adherence to the guidelines set for the presentation of reports and methodological evaluation tools.

The importance of producing and publishing new studies in this field should be underlined. Nevertheless, it is important to ensure that future studies are well-planned and executed, given the relevance of the production of scientific knowledge and the need to obtain trustworthy results for direct clinical practices and/or health policies which are sensitive to the needs groups which are vulnerable to severe events associated with COVID-19.

Financing

This work was supported by the “Coordinación de Perfeccionamiento del Personal de Educación Superior – Brasil (CAPES)– Código de Financiamiento 001”.

Conflict of interests

The authors have no conflict of interests to declare.