Mobile application development to improve the active search for functional impairment and persistent symptoms in individuals post-COVID-19

Abstract

Objectives

This study aimed to develop a mobile application (App) to be used by primary care teams in the active search for functional impairment, long-term symptoms, and disabilities in individuals who have recovered from COVID-19, contributing to early treatment and referrals for multidisciplinary care and rehabilitation. This experimental study used the minimum viable product (MVP) methodology to develop an App named ReabilitaCOVID.

Methods

This methodology involves ideation, content creation, prototype creation, usability tests, and adjustments based on feedback. The study was conducted in Brazil, and the population of the study included community health workers, who were the App's target users, and individuals from the community who had previously had COVID-19 and were at risk of developing PCS.

Results

The App included a sociodemographic questionnaire, a clinical questionnaire, the post-COVID-19 Functional Status Scale (PCFS), the Modified Medical Research Council (MRC) Dyspnea Scale, and a flowchart. Usability tests were conducted, with feedback collected and adjustments made for improvements. Barriers to community health workers’ use of the application were identified.

Conclusion

A tailored app was developed for primary care teams to use in the active search for PCS. Functional and usability tests were performed in simulated and real environments. The App has the potential to facilitate referrals for multidisciplinary care and rehabilitation efficiently, and it will be available freely to public health care services. MVP is a suitable approach for developing a tailored App for healthcare teams.

Resumen

Objetivos

Este estudio tuvo como objetivo desarrollar una aplicación móvil (App) para ser utilizada por los equipos de atención primaria en la búsqueda activa de deterioro funcional, síntomas a largo plazo y discapacidades en personas que se han recuperado de la COVID-19, contribuyendo al tratamiento temprano y a las derivaciones para atención multidisciplinaria y rehabilitación. Este estudio experimental utilizó la metodología de producto mínimo viable (MVP) para desarrollar una aplicación llamada ReabilitaCOVID.

Métodos

Esta metodología implica ideación, creación de contenido, creación de prototipos, pruebas de usabilidad y ajustes basados en la retroalimentación. El estudio se realizó en Brasil y la población del estudio incluyó a trabajadores de salud comunitarios, que fueron los usuarios objetivo de la aplicación, y personas de la comunidad que habían tenido COVID-19 previamente y estaban en riesgo de desarrollar PCS.

Resultados

La App incluyó un cuestionario sociodemográfico, un cuestionario clínico, la Escala de estado funcional post-COVID-19, el modificado del Medical Research Council, la Escala de disnea y un diagrama de flujo. Se realizaron pruebas de usabilidad, se recogieron comentarios y se realizaron ajustes para mejorar. Se identificaron barreras para el uso de la aplicación por parte de los trabajadores de salud comunitarios.

Conclusión

Se desarrolló una aplicación personalizada para que los equipos de atención primaria la utilicen en la búsqueda activa de PCS. Se realizaron pruebas funcionales y de usabilidad en entornos simulados y reales. La aplicación tiene el potencial de facilitar las derivaciones para atención multidisciplinaria y rehabilitación de manera eficiente, y estará disponible de forma gratuita para los servicios de atención médica pública. La metodología MVP es un enfoque adecuado para desarrollar una aplicación personalizada para los equipos de atención médica.

Introduction

The management of post-COVID-19 syndrome (PCS) has been a considerable challenge faced by health systems worldwide since the pandemic was overcome.^1,2 Global prevalence of PCS for 120 days after COVID-19 infection was estimated to be 49% (95% CI: 40–59).³ In a recent systematic review including eighteen papers that reported one-year follow-up data from 8591 COVID-19 survivors, the most prevalent long-term symptoms were fatigue/weakness (28%, 95% CI: 18–39), dyspnea (18%, 95% CI: 13–24), arthromyalgia (26%, 95% CI: 8–44), depression (23%, 95% CI: 12–34), anxiety (22%, 95% CI: 15–29), memory loss (19%, 95% CI: 7–31), concentration difficulties (18%, 95% CI: 2–35), and insomnia (12%, 95% CI: 7–17) after twelve months.⁴ Indeed, the volume of current and future patients with PCS is so high that primary care may not be able to cope with their care, and strategies are needed to improve their diagnostic and healthcare attention.⁵

The pandemic highlighted the role of primary care in handling health emergencies and delivering quality care. In many low- and middle-income countries, community health workers (CHWs) were the first people to contact the health system and played an essential role in the pandemic.^5–7 With respect to PCS, primary care is crucial for actively searching, diagnosing, managing, and referring patients to multidisciplinary care and rehabilitation.⁵

In Brazil, primary healthcare, mainly through CHWs, plays a significant role in actively searching for cases within each region.⁸ CHWs are health professionals who act as crucial links between the community and healthcare services and carry out health promotion activities, disease prevention, health surveillance through home visits, and educational actions, including local epidemiological investigations.^9,10 In this way, CHWs possess knowledge of the local territory, facilitate access, and establish bonds between patients and healthcare teams, thus contributing to more efficient health strategies and interventions, especially in more vulnerable communities.¹¹ Therefore, it is important to offer tools that can improve their ability to screen PCS cases and refer them to diagnostic and treatment methods.

The COVID-19 pandemic accelerated advances in technological resources for healthcare. Mobile applications (App) have emerged for diagnosis, telemedicine, and home-based exercise programs.^12–17 However, we did not find any App to help primary care in the active search for patients with PCS.

Therefore, this study aimed to develop an App to be used by CHWs in the active search for functional impairment, long-term symptoms, and disabilities in individuals who had COVID-19, contributing to the management and referrals for multidisciplinary care and rehabilitation.

Methods

This experimental study was performed from January 2021 to June 2023 in Espírito Santo State, Brazil.

The study used the minimum viable product (MVP) methodology to develop an App and evaluate its utilization by the final user (CHW).

The MVP consists of building and delivering an early prototype with the minimum functionalities to be tested by the target users. It is an iterative process that encompasses idea generation, prototyping, presentation, data collection, analysis, and learning and can be synthesized in 3 cyclic phases: build-measure-learn.^18,19 Once the first version of the product with basic functionalities is made available, usability tests are performed by users to detect failures, identify whether the product solves a real problem, and ensure that the final product meets the users’ needs and expectations by making improvements and adding new functionalities to the next versions until the final version is reached.^18,19Fig. 1 shows the cyclic MVP methodology used.¹⁸

Figure 1.

The cyclic MVP methodology.

Table 1 presents all the methodological steps for App development, and the details of each step are described below.

Table 1.

Steps of the methodological App development process.

	Phases	Participants	Description
Ideation	Project elaboration and action planning	Interdisciplinary healthcare teamInformation technology team	Literature review. Situational analysis. Meetings to identify the community needs, and discuss the clinical relevance of literature findings, and their applicability in telehealth.Project writing, submission to Municipal Health Department, approval, and meetings with them.Submission to Ethical Committee and approval. Submission to a research funding agency and approval.
Content creation	Literature review	Interdisciplinary healthcare team	Searches in databases (PubMed, PEDro, BVS and SciELO) to identify persistent symptoms in people who had recovered from COVID-19 and validated instruments for functional impairment assessment.
	Content definition	Interdisciplinary healthcare teamInformation technology team	Meetings to discuss, improve and approve questionnaires, and selected assessment tools.Definition of the name, logo, and graphic esthetics of the App.
Prototype creation	Functionality definition	Interdisciplinary healthcare teamInformation technology team	Meetings to define the functional requirements and the flowchart systematization.
	Prototype creation	Information technology team	Creation of a preliminary App and delivery to the first usability test.Tutorial development for App use training.
Assessment and incremental deliverables using MVP	1st Functionality and usability test	Interdisciplinary healthcare teamInformation technology team	Training of health researchers with tutorials for App use, guided by the information technology team, simulation of App uses, detection of improvement needs, and usability assessment.
	Adjustments	Information technology team	Incremental improvements and deliveries. Improvements in the tutorial.
	2nd Functionality and usability test	Community health workersInterdisciplinary healthcare team	Training with tutorial guided by researchers from the health care team, simulation of application use. Functionality and usability assessment.
	Adjustments	Information technology team	Incremental improvements and deliveries.
	3rd Functionality and usability test-real environment	Community health workersParticipants from communityOne healthcare team researcher	The community health workers used the App with the individuals from the community who have recovered from COVID-19 for functionality and usability tests, under the supervision of a researcher from the healthcare team.
	Adjustments	Information technology team	Incremental improvements and deliveries
	4th Functionality and usability test- real environment	Community health workersParticipants from communityOne healthcare team researcher	The community health workers will use the App with the individuals from the community who have recovered from COVID-19 for functionality and usability tests, under the supervision of a researcher from the healthcare team.
	Adjustments	Information technology team	Incremental improvements and deliveries.
	App launch	Information technology team	Delivery of the final version of the App

Legend: App, mobile application.

Ideation

This pre-development step encompassed situational analysis, project elaboration, concept generation, and action planning (Table 1).

Content creation

This step encompasses a literature review and content definition (Table 1). The literature review included the languages English, Portuguese, and Spanish. The App was developed in Brazilian Portuguese and was named ReabilitaCOVID.

The variables to be collected were as follows: (1) sociodemographic data, including name, age, sex, gender, date of birth, marital status, race/skin color, address, number of people residing in the same household, and number of people living at home who were diagnosed; (2) clinical data, including information regarding health conditions and clinical characteristics before and after COVID-19, such as date of diagnosis, information of hospitalization (if hospitalized), and the symptoms presented by patients before and after COVID-19 were resolved, such as shortness of breath, fatigue, dry cough, cough with secretion, pain in any part of the body, changes in smell, differences in taste, difficulty with speech, difficulty in swallowing, changes in sensitivity, difficulty moving, balance difficulty, memory loss, delirium, and cognitive deficits; (3) functional status according to the Post-COVID-19 functional status scale²⁰; and the subjective degree of dyspnea according to the Modified Medical Research Council Dyspnea Scale²¹ (Table 2).

Table 2.

Content included in the App.

Content	Description
Sociodemographic questionnaire	Sociodemographic data: name, age, sex, gender, date of birth, marital status, race/color, address, number of people residing in the same household, and number of people living at home who had the diagnosis.
Clinical questionnaire	Information regarding health conditions and clinical characteristics before and after COVID-19: date of diagnosis, information of hospitalization – if hospitalization. Before and after the disease was cured: symptoms (shortness of breath, fatigue, dry cough, cough with secretion, pain in any part of the body, change in smell, the difference in taste, difficulty with speech, difficulty in swallowing, change in sensitivity, difficulty moving, balance difficulty, memory loss, delirium, and cognitive deficits).
Post-covid-19 functional status scale^20	Tool to assess the course of the disease and its impact on functional status, among adult subjects with confirmed and presumed COVID-19.
Modified Medical Research Council Dyspnea Scale^21	It consists of five statements about perceived breathlessness. The patient reports his subjective degree of dyspnea.
Flowchart	Based on the results of questionnaires and scales, the App suggests to the community health agent whether the individual should be referred to the interdisciplinary team.
Self-care guidelines	Two booklets were prepared, one with general guidelines and the other with exercises, based on scientific evidence, for being delivered to post-COVID patients who do not need to be referred to BHU.

Legend: App, mobile application; BHU, Basic Health Unit.

Prototype creation

The first phase of prototype creation was Functionality Definition, which included the definitions of the requirements, features, and functionalities that the App should contain to meet the user's needs and cover unexpected situations, providing the desired functionality (Table 1).

In the second phase of this step, the App prototype was developed on the Android platform, which was chosen because, in Brazil, the Android operating system represented approximately 86% of the market in January 2021.²² Another advantage is that Android did not have any cost during App development and maintenance, which is relevant since the App will be freely offered for public services after its launch.

The App presents a flowchart after the evaluation, suggesting, if necessary, patient referral to the basic health unit (BHU). In addition, some basic self-care guidance can be provided by CHWs through digital or printed booklets, which do not contain patient data.

After questionnaire completion, patient data are sent to the application server, which concentrates all the data in a single central database. To ensure data confidentiality, after these questionnaires are sent to the server, the CHWs can no longer access them for consultation or even editing, and no patient's data can be accessed on the app, only a referral number.

To access the database of all questionnaires, a web application was developed. To guarantee data confidentiality, only one user in a management position at each BHU has access to the questionnaires submitted by the CHWs of each BHU. Access is accomplished by generating a file in CSV format, which is a text file that stores data in a structured table format and can be opened in spreadsheets (Excel, LibreOffice Calc, Google Sheets, etc.) or statistical software (such as SPSS, among others). There is a button in this application that generates this file and saves it on the user's computer, in this case, the UBS manager, and it is protected by password. This file contains each field of the questionnaire proposed in the mobile application with the respective answer given by the patient consulted. The necessary data can be included in the electronic medical charts for the use of health professionals, and they must be used with the same confidentiality as the other information in the medical charts. The access for the BHU manager is granted by the web system administrator, who has majority access, and this access credential is unique for security reasons.

The first App prototype was delivered to be tested and adjusted according to detected needs via an iterative-incremental approach, which means that each new set of provided functionalities could be integrated with previous versions.

Usability tests

The 1st usability test was performed by the researchers of the healthcare team, who learned to use the App via the tutorial and received instructions from the information technology team. They simulated patients’ evaluations and gave feedback to the technology team, which adjusted the App and delivered the new version for the next test.

The main final users of the App will be the CHW, so they constitute the target population recruited for the following tests.

For the 2nd and 3rd usability tests, the CHWs were selected from the BHUs into the Health Territory of the Federal University of Espírito Santo (UFES), which encompasses 8 BHUs. However, one BHU was excluded because it reported a lack of regular home visits, so seven BHUs were included, totaling 104 CHWs.

The inclusion criteria for CHWs were as follows: working as CHWs at an included BHU, over 18 years old, owning a personal cell phone, already using it daily in their work activities, agreeing to participate and signing the informed consent form. The exclusion criteria were not performing fieldwork activities and being absent from work during the data collection period.

In the 2nd usability test, the CHWs were trained at their BHU. They performed simulated tests of App usage guided by healthcare researchers. The feedback was used for App improvements before the new version was delivered.

The 3rd usability test was performed in a real environment. The CHWs performed the patient assessment via the App during their routine home visit. One researcher accompanied the CHWs to observe them working through the test scenarios, explained the research to the patients, them to participate, and obtained informed consent. Feedback was collected from CHW for App adjustments.

The inclusion criteria for community-dwelling individuals were as follows: who resided in the community served, were over 18 years old, had previous infection by COVID-19, agreed to participate, and signed informed consent forms. The exclusion criteria were cognitive impairments and unavailability at the evaluation time.

Ethical approval

The Human Research Ethics Committee of the UFES approved the study (CAAE number: 46292521.4.0000.5060).

Results

Content creation

Table 2 presents the content included in the APP, which was divided into six sections: a sociodemographic questionnaire, a clinical questionnaire, the post-COVID-19 Functional Status Scale (PCFS),²⁰ the Modified Medical Research Council (mMRC) Dyspnea Scale,²¹ and a flowchart. Self-care guidelines were created to be delivered to patients according to flowcharts.

1st usability test

Four physiotherapists, one nurse with experience coordinating CHW teams, and three physiotherapy undergraduate students participated. Adequate insertion of the tools in the App was verified, ensuring alignment with the source materials and identification of any issues. Table 3 describes all the needs for corrections detected in this test.

Table 3.

Usability tests feedback about corrections or improvements needed in the App.

1st functionality and usability test	2nd functionality and usability test	3rd functionality and usability test
Adjustments in the obligatoriness or not of questions.	Deliver the printed booklet only.	Adequacy of terms for commonly spoken language.
According to the answer YES/NO, open the correct correspondent questions.	Adjust the form into the format to enter dates.	User login errors.
Text corrections according to the original materials and validated tests/questionnaires.	No question about the number of times you had Covid-19.	Application low speed to pass the interfaces (one page to another).
Add automatically calculated quantitative information (e.g., days hospitalized through the day of hospitalization and discharge).	Exchange of terms and adjustments in writing for easier understanding.	The interviews were not being saved.
Fails and low speed when passing the screens.	Login errors.	It was suggested to reorganize the mandatory and non-mandatory answers to questions.
Error on flowchart when indicating the grade of the scales and the recommendation after the assessment.	Adjustments in the order and visual design of some questions presentation.	It was requested to replace descriptive answers with option lists.
	Implement a system to lock or open questions that relate according to the answer.	In the flowchart, besides the need for referencing, show the description of the functionality degree found instead of grade numbering.
	In scales and flowchart, show the written result instead of the score numbering.

Legend: App, mobile application.

2nd usability test

According to the project, the CHW sample should be selected through a random methodology within each unit. However, owing to a lack of technology skills for CHWs (Table 4), it was necessary to select the CHWs who were most technologically proficient to participate in the study.

Table 4.

Identified barriers to App use during the tests.

Barriers	Description
Mobile device issues	Lack of storage, internet, and suitable device models for app installation and functionality
Lack of technology skills	Agents may resist or struggle to learn how to use current cell phone features.
Security	As a result of the pandemic, an official order was issued prohibiting CHWs from entering homes, and interviews were conducted on sidewalks instead. So, the CHWs reported feeling insecure when carrying phones in public spaces, particularly in streets and neighborhoods with rates of theft and violence.
Work organization	Agents’ visits were disrupted by the need to attend courses, participate in other actions, or perform additional functions.
Visiting hours	The CHWs conduct home visits during the daytime, typically within business hours, when most people who work during the day are away from home. This limited availability reduces the reach of the visits, as they primarily target individuals who are at home during those hours.
Diagnosis by exams	Starting in 2021, it was observed that community members stopped getting tested for Covid-19, even when they exhibited clinical symptoms. This posed a challenge in identifying individuals who had confirmed cases of Covid-19.
Taboo/disbelief of diagnosis	Instances where individuals show disbelief or fear of acknowledging a positive COVID-19 test result.
Behavioral barriers	Skepticism regarding the App's implementation, its applicability, and the patient referral according to demand, lack of understanding regarding the research's importance.

Legend: App, mobile application.

The sample of this test was compounded by 18 CHWs from the included BHUs. They were trained to use the App and simulated the tests in BHU. The feedback is presented in Table 3.

3rd usability test

Although 18 CHWs had participated in training in the 2nd test, two trained CHWs could not participate in this test because they were designated for other labor activities. Therefore, 16 CHWs assessed 52 individuals from the community in the 3rd usability test. Each CHW evaluated a minimum of three individuals. Table 3 shows the feedback about improvement needs from this usability test.

Barriers

Table 4 presents details about the barriers identified during the tests. The main barriers to the use of the App by the CHWs during the tests were mobile device issues, lack of technology skills, security, work organization, visiting hours, diagnosis by exams, taboo/disbelief of diagnosis, and behavioral barriers.

Next steps

This study describes the App development process and the preliminary results. The next step will be the 4th usability test, the App launch that will be freely available for public health services, and the validation tests. The results of these steps will be presented in future publications.

Discussion

This study developed a mobile App to record information regarding functional impairment, long-term symptoms, and disabilities reported by individuals who have recovered from COVID-19 to be used by CHWs during their field visits.

Primary care clinical guidelines for PCS emphasize the need for a broader approach, providing an active search for early diagnosis, multidisciplinary referral and follow-up.⁶ Therefore, when developing the App's content, the clinical questionnaire included the most prevalent symptoms identified in the literature^{23,24,3,25–27} to perform triage, compare pre- and post-disease symptoms, and deliver more targeted guidance.

Careful consideration was given to the assessment tools to be included in the App, selecting the most suitable tools for this population and the App objectives. The mMRC Dyspnea Scale was chosen because it is a concise, easily applicable, and understandable scale that provides a subjective overview of the patient's perception of dyspnea and has been used in studies on the PCS.^28–30 The PCFS scale was particularly relevant for this group.²⁴ This approach has implications for post-COVID-19 functional assessment, where clear and objective questions can report limitations and restrictions. A cross-cultural adaptation of the Brazilian Portuguese version was used.²⁹

A Brazilian study about barriers to the adoption of a mobile health information system by CHWs identified difficulties related to text input errors.³⁰ On the basis of these findings, we included mainly objective questions in the questionnaire. Even so, in the usability tests, the CHWs requested replacing some of the few descriptive answers with option list answers, which were adjusted.

In usability tests, barriers related to professionals’ daily routine, such as excessive tasks, leading to an overload of responsibilities and a lack of time, were identified. In other countries, increased workloads were also reported by healthcare aides as a barrier to adopting new technological resources.³¹ However, after the training and implantation phases, the app is expected to reduce work overload.

In our study, the lack of familiarity with technology affected the methodological planning for selecting CHWs and the usage of the app. Resistance and slower app usage were observed among the CHWs who had more difficulty using mobile phones, which disrupted their workflow. Another study highlighted challenges associated with the use of technologies by healthcare aides.³¹ Continued digital education is necessary to face these challenges because digital skills have become increasingly essential for health workers.

Behavioral aspects were identified, including skepticism regarding the App's implementation, applicability, and patient referral according to demand. Part of this skepticism is probably related to the scarcity of rehabilitation services in the referral system. A recent study reported that post-discharge follow-up was considered insufficient/nonexistent by patients after COVID-19 for new problems, rehabilitation needs, or monitoring.³² This type of behavioral barrier has also been observed in other countries.³¹

In Brazil, CHWs enter people's homes during field visits. However, during the pandemic, the frequency of home visits was reduced, and sometimes, visits were conducted at the doorsteps of houses. Many CHWs reported insecurity when carrying smartphones in public spaces or when they were unable to enter residences and needed to conduct interviews on sidewalks, especially in neighborhoods where theft and violence are common. A study including CHWs from another Brazilian state also mentioned the insecurity professionals feel when carrying expensive smartphones.³³ A recent systematic review on mobile health adoption by health care professionals reported challenges in developing countries, such as concerns about professional safety, not experienced by professionals from developed countries.³⁴

The COVID-19 pandemic highlighted the urgent need to address inequities and reorient health systems around the principles of primary care.³⁵ However, a recent scoping review revealed that the implementation of primary health care remains challenging worldwide and should be tailored to local contexts, reinforcing the importance of sharing lessons learned since many barriers are common across countries and regions.³⁵ Therefore, a strength of this study was that MVP is a methodology that can be applied in regions with few resources to build apps tailored for primary care teams to address the challenges of PCS management. In MVP, the development process is lean since the tests validate the functionalities that meet the end-users’ needs, building a shippable product in an incremental way without wasting.^18,19The main limitation of this study is that the CHW sample was not randomized, which may compromise the generalizability of the results and may imply additional difficulties in the App implementation phase, demanding more training. However, it should be noted that five users are considered optimal for the usability test in the MVP methodology,³⁶ so the larger sample included in the present sample and the different scenarios of the tests allowed analysis of the App's functionality under adverse conditions in different environments.

In conclusion, a tailored App was developed to be used by primary care teams to actively identify functional impairments, long-term symptoms, and disabilities post-COVID-19. This App has the potential to help the management of screening, registration, systematic monitoring, and referral for multidisciplinary care and rehabilitation. The App may also contribute to epidemiological and clinical studies. MVP is a suitable approach for developing a tailored App for healthcare teams, so this methodology may be applied to build Apps tailored to address the challenges of PCS, including in low-income and developing countries.

Contributions

These authors contributed equally to this work and share first/last authorship.

Authors’ contributions

Conception or design of the work: Laís Mello Serafim; Raiany Franca Guimarães, Lisandra Vanessa Martins, Grace Kelly Filgueiras Freitas, Fernanda Mayrink Gonçalves Liberato, Giuliano Prado de Morais Giglio, Paulo Henrique Bressanini Rizzi, Hiago Carlos Moreira, Leonardo Alporges Martins, Sheila Cristina de Souza Cruz, Valeria Alves da Silva, Flávia Marini Paro.

Data collection: Laís Mello Serafim; Raiany Franca Guimarães, Lisandra Vanessa Martins, Grace Kelly Filgueiras Freitas, Fernanda Mayrink Gonçalves Liberato, Giuliano Prado de Morais Giglio, Paulo Henrique Bressanini Rizzi, Hiago Carlos Moreira, Leonardo Alporges Martins, Sheila Cristina de Souza Cruz.

Development of the application: Giuliano Prado de Morais Giglio, Paulo Henrique Bressanini Rizzi, Hiago Carlos Moreira, Leonardo Alporges Martins.

Data analysis and interpretation: Laís Mello Serafim; Lisandra Vanessa Martins, Grace Kelly Filgueiras Freitas, Fernanda Mayrink Gonçalves Liberato, Giuliano Prado de Morais Giglio, Flávia Marini Paro.

Drafting the article: Laís Mello Serafim; Raiany Franca Guimarães; Flávia Marini Paro.

Critical revision of the article: Laís Mello Serafim; Raiany Franca Guimarães, Lisandra Vanessa Martins, Grace Kelly Filgueiras Freitas, Fernanda Mayrink Gonçalves Liberato, Giuliano Prado de Morais Giglio, Paulo Henrique Bressanini Rizzi, Hiago Carlos Moreira, Leonardo Alporges Martins, Sheila Cristina de Souza Cruz, Valeria Alves da Silva, Flávia Marini Paro.

Final approval of the version to be submitted: Laís Mello Serafim; Raiany Franca Guimarães, Lisandra Vanessa Martins, Grace Kelly Filgueiras Freitas, Fernanda Mayrink Gonçalves Liberato, Giuliano Prado de Morais Giglio, Paulo Henrique Bressanini Rizzi, Hiago Carlos Moreira, Leonardo Alporges Martins, Sheila Cristina de Souza Cruz, Valeria Alves da Silva, Flávia Marini Paro.

Ethical statement

The Human Research Ethics Committee from the Health Sciences Canter of the Federal University of Espirito Santo (UFES) approved this study (CAAE number: 46292521.4.0000.5060).

Funding statement

Fundação de Amparo à Pesquisa do Espírito Santo (FAPES), Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), Ministério da Saúde/Departamento de Ciência e Tecnologia (MS/DECIT).

Conflict of interest

The authors declare that they do not have conflicts of interest.

Acknowledgements

FAPES, CNPq, MS/DECIT, Secretaria Municipal de Saúde de Vitória, Secretaria Estadual de Saúde do Espírito Santo.