Implementing a Redcap-based research data collection system for mental health

Abstract

Background

The implementation of new technologies in medical research, such as novel big storage systems, has recently gained importance. Electronic data capture is a perfect example as it powerfully facilitates medical research. However, its implementation in resource-limited settings, where basic clinical resources, internet access, and human resources may be reduced might be a problem.

Methods

In this paper we described our approach for building a network architecture for data collection to achieve our objectives using a REDCap® tool in Colombia and provide guidance for data collection in similar settings.

Conclusions

REDCap is a feasible and efficient electronic data capture software to use in similar contexts to Colombia. The software facilitated the whole data management process and is a way to build research capacities in resourced-limited settings.

Resumen

Contexto

La implementación de nuevas tecnologías en la investigación médica, como los nuevos sistemas de gran almacenamiento de datos, recientemente han ganado importancia. El almacenamiento electrónico de datos es un ejemplo perfecto ya que facilita poderosamente la investigación médica. Sin embargo, su implementación en ambientes con recursos limitados, donde los recursos básicos clínicos, el acceso a internet y el recurso humano podrían ser reducidos, suponen un problema.

Métodos

En este artículo describimos nuestro acercamiento para construir una red arquitectónica para la recolección de datos, en aras de alcanzar nuestros objetivos mediante la utilización de la herramienta REDCap® en Colombia y proveer una guía para la recolección de datos en condiciones similares.

Conclusiones

REDCap es un software de almacenamiento electrónico de datos eficiente y encontramos que resulta posible su utilización en contextos similares a Colombia. Este software facilitó el proceso del manejo de los datos y es una manera de construir capacidades investigativas en contextos donde los recursos son limitados.

Introduction

The increasing need for rigor and reproducibility in biomedical research demands exhaustive data management to ensure the trustworthiness and accuracy of the results. Low and middle income countries (LMICs) might need innovative approaches to improve data management process because the level of funding, data management capacity and technological and basic infrastructure may be scarce.¹ Some efforts to strengthen the data management process include improving staff training, and server, network and/or software development. Also, new technologies and big data storage systems improve the overall efficiency of the research process, including data management.^1,2 Scaling-up research capacities in LMICs is important due to the growing interest in global health and international collaborative research projects, and they require simultaneous access to data in different locations and reliable systems of data collection and storage.^2–4

In LMICs, the standard method for data collection is often a paper-based method, which often results in dubious data with numerous errors.^4,5 Electronic data capture (EDC) systems have evolved to replace traditional paper-based methods, accelerate data collection and provide expedite access to data to researchers.^6,7 Currently, there are many electronic data collection (EDC) tools that may improve the data management process in LMICs and fulfill the requirements for international collaborative projects (Microsoft access, Epinfo, Microsoft excel, the APCDR electronic questionnaire) but, to our knowledge, they have not been compared yet and complicate the selection of the appropriate software. One tool, highly used in biomedical research, is the Research electronic data capture (REDCap®) system.^8,9 This software was designed by an informatics team at Vanderbilt university to support clinical and translational research. It has many useful features that other EDC software often lack, such as solid security settings, intuitive software management, easy development of data collection forms, internet support, and the fact that it does not require a high speed internet connection to run adequately.^2,10 Therefore, it may be a good choice when implementing data collection systems, particularly in limited-resource settings.¹¹

IBM Clinical Development, Videoc, SurveyMonkey and Qualtrics are solid commercial alternatives to REDCap and have a similar performance. However, unlike REDCap, these tools are costly or they were not built specifically to support clinical researcher.¹² REDCap is currently used by researchers from 128 countries in all continents and from different branches of medicine and science. Even if this software was initially used by researchers from high income countries such as Canada, USA, Singapore or Japan, the number of users from LMICs have been increasing steadily since 2013. Now, more than 20% of REDCap partners belong to LMICs¹³ and may be related to the increasing production of academic outputs in many LMICs, including Colombia.¹⁴ In this Latin American country, the number of academic research and international partnerships has recently increased,¹⁵ demanding commitment to rigorous ethical and scientific standards.

Many factors, facilitators and barriers from the context where research takes place can strongly influence it.^16–18 Potential research sites in Colombia vary considerably. Some have sufficient resources for high quality biomedical research and health care services, whereas others face several difficulties that impede the development of academic projects and collaborations. Those include scarce resources and infrastructure, poor funding for some regions, little or non-support from the government or academic institutions, geographic challenges, and very low rates of education and literacy.^1,19,20 Moreover, even if there is government support in these regions, the scientific bureaucracy and heavy clinical workload often discourages researchers to go further and hinders research activities.^17,21

The DIADA project

The DIADA project is a large international collaboration currently underway in Colombia. This multicenter project is run by researchers from the Pontificia Universidad Javeriana (PUJ), in Colombia, and Dartmouth College (DC), in the United States. The study seeks to evaluate the implementation of a technology-based model of care for depression and alcohol consumption disorders in different settings in Colombia (6 sites, including both rural and urban areas). The model includes the use of a mental health therapeutics app (Laddr from Square2) that provides enhanced service delivery model for depression and alcohol use disorders in primary care and has been proven to enhance mental health outcomes.^22,23 An overview of DIADA project has been described elsewhere,²⁴ though it should be noted that we collected baseline data and follow-ups every three-months over the course of a 12-month participation in the study.

The aim of this case report is to describe the implementation of an integrated data management system for the DIADA project using the REDCap® tool.

Methods

We implemented our research data collection using REDCap to provide security, privacy and confidentiality to all participants. REDCap had tools that helped the research team overcome challenges in data collection from our six study sites throughout Colombia: offline functionality, mobile app support, and the e-consent framework. We set up REDCap® for the DIADA project at the PUJ servers in Bogotá, Colombia, thus, all data was received in a central location. The servers provided an encrypted Internet connection via 128-bit Secure Sockets Layer (SLL), the standard technology for securing eCommerce and eBanking transactions on the internet.

We established a working group to design the database and the case report forms (CRFs). The database working group evaluated overall database architecture by site, user requirements, follow-up intervals and events. Database development tasks were distributed in such a way that Dartmouth researchers developed the CRFs, and Bogotá researchers created the database and its events and arms. Database internal logic was developed together. Then, we tested the overall database functionality with dummy data.

We developed a Data Management Plan where members from both institutions could access the data collected at any time. Two team members shared the Database Manager (DM) role and responsibilities (one from PUJ, and one from DC). This role consisted of supervising the entire data management process and included multiple tasks, such as, controlling the database access and user rights for team members, data quality monitoring and backup, data export, database updates and overall database administration. The DM from Bogotá was primarily in charge of controlling the database access, database quality, updates and data exports, whereas the DM from Dartmouth performed regular data quality checks, discrepancy management, database updates and data exports.

We conducted a formative assessment at each site to learn about their capacity, computer literacy, and internet access prior to the implementation of the project. We provided each site with desktop computers and tablets with REDCap mobile app installed. All research staff REDCap use training sessions with dummy data.

Data collection

Data collection was performed in both the web version of REDCap and the mobile app installed in the tablet. REDCap mobile app allowed offline data collection, overcoming the challenges of poor internet connection, and eased the electronic informed consent process. Research staff completed the assessments on paper and transcribed them in REDCap at the end of the day in those situations when data collection was impossible, such as no internet connection or electricity problems.

We used REDCap e-consent framework for the informed consent process. Patients could fill their information and sign the consent in the tablet, after that we handed each individual a physical copy of the consent form that included the investigators contact information. The electronic informed consent increased the acceptability of the process and served as a useful tool for the internal and external audit process. In some study sites potential participants have poor computer and digital literacy, thus, we trained the research staff to support potential participants and participants completing the consent form.

Finally, we followed internal data monitoring using a collaborative approach. The DM at Dartmouth performed regular quality checks (type of variables, length of the information, range of values, coherence between information) to avoid data errors and protocol deviations. When an issue was identified, the DM raised a query, assigned it to a local team member, and notified the site coordinator. The Site Coordinator arranged a meeting with the corresponding team member to solve the query or provide a relevant response that was later reviewed and closed by the DM.

Results

At the time of writing this paper, our use of REDCap enabled and eased the gathering of longitudinal data of 1254 individuals diagnosed with major depression, alcohol use disorders, or both. One of the most interesting REDCap components is the longitudinal mode. The longitudinal mode allowed us to capture data longitudinally and repeating CRFs with a minimal burden to the research staff and DMs. Also, the automatic creation of a unique identifier for each participant eased further longitudinal analysis.²⁵ With the implementation of the longitudinal mode our research staff easily performed data collection both during recruitment and follow-ups. Most of the issues and challenges were easily addressed through staff training, retraining and site engagement. We found similar challenges in the implementation across settings, however some were related to the peculiarities of each study site.

Urban sites

We found that internet connection was unstable, even in urban sites; surprisingly, the quality of internet connection changed within the same building. To face this challenge, our research staff identified the areas with strong connectivity and collected the data using the online REDCap features. In those areas with unstable connection, they used the REDCap mobile app that allowed offline data collection, including the e-consent signatures, and uploaded the data at the end of the day. In the extreme cases of power outages or high internet instability, they used paper versions of CRFs and afterwards entry the data into REDCap at the end of the day. This paper versions were securely stored or scanned to preserve a hard copy in case of data errors identified during our internal data monitoring process.

Another challenge was that research assistants had no previous experience with REDCap, nor had a digital or computational background. All researcher staff received training on REDCap main functions and performed practical exercises with dummy data to prevent this from causing errors. In addition, two researchers with expertise on the topic, were available to solve any doubts or problems about REDCap use or overall data management and collection.

One last challenge that we faced was insecurity. In our context it is not rare for theft to happen in medical settings, especially targeted to technological devices. Therefore, in each setting we destined one room to keep equipment safe and one person was assigned to oversee the lock down of this room on daily basis. The tablets were embedded in kiosks with screws, which make them difficult to remove.

Rural sites

Availability of resources was a main challenge in rural and semi-rural settings. Due to the lack of internet connection and computers or other technological devices, we provided the sites with a desktop computer, tablets for screening and data collection. Rural sites had limited internet access and most of them did not have wireless internet connection. Therefore, we supported the sites in setting up Wi-Fi networks (the highest quality available) for research and internal use.

At the time of the project implementation most of clinical and administrative data was collected using traditional paper-based methods. We decided to include members of the community to build research capacity at very site regardless of its location. Thus, we faced similar challenges regarding computer and digital literacy as those we faced in urban sites, with research staff and participants having limited computer literacy. There were difficulties with the general use of computers, smartphones, e-mails, excel sheets and other software. We offered additional training, created templates to ease software usage, and a concise set of notes that research assistants could check for quick reference if needed.

We also provided smartphones with unlimited data and voice calls that local research staff used to schedule and complete participant follow-ups, and to contact researchers at PUJ at any time. However, since the coverage was not ideal, the resolution of doubts was sometimes delayed. In those cases, local researchers relied mainly on offline features of REDCap, and to a lesser extent on paper-based methods.

Regarding REDCap, members from the data management team (both from Javeriana and Dartmouth) were always available to answer doubts or solve emerging issues. Finally, one PUJ researcher, with background in Biomedical Informatics visited the sites approximately every month. This was an opportunity to retrain the staff, check the devices and resolve any issues.

Discussion

As a LMIC, we realize that acquiring a proprietary EDC software is not always feasible. Therefore, REDCap availability at no charge for institutional partners is one of its greatest features. Even if an institution requires a license, joining the REDCap network and obtaining one is not difficult. Furthermore, we found that the use of this software in our institutions prompted the collaboration between departments and divisions (medicine, language, engineering, technology, etc.).^12,25

In general, there were no unsolved queries or doubts about the software. After the training meetings, research assistants managed to collect and entry (if necessary) all data into REDCap and overcome all challenges. One useful strategy to increase data quality for our team consisted of pairing research assistants to review each other’s data and decrease the error rate.

Although other researchers describe REDCap as a very intuitive software,⁸ we noticed that in our context this may be true for those individuals with some computer literacy or background in informatics or computers. Even when the individuals had prior knowledge about computers, our research staff reported that advanced features were not as easy to use as have been reported in other studies.²⁵ One additional issue that has been mentioned in other studies is that individuals often delete records by mistake.¹² We did not find this event, perhaps due to the efforts in training our staff, and specifically, to the practical exercises (simulation) in which, they could experience real problems and try solving them.

One feature that our team found very useful was the Online Designer. It enabled us to design our CRFs according to the specific needs of Project DIADA and facilitated all ambulatory assessments. It has been reported that it can even allow participants to fill the questionnaires, upload images and other type of files according to the design stablished by the researchers,²¹ however we had no need to use this feature.

On the other hand, as a collaborative team, the internal data collection monitoring benefited of the tool ‘Open query’ to ensure data was correct, consistent, and applicable. REDCap version 10.3.1 provides a solution with the tool “REDCap messenger”, where the team collecting the data and the reviewers have a direct communication to solve data issues and inconsistencies, reducing the time to solve them.

To conclude, REDcap’s collaborative work features²⁶ were very important for Project DIADA considering that our research team was an international collaboration. These features enabled researchers’ simultaneous access to data and allowed researchers from Dartmouth assess data quality without stopping data collection from local researchers.

Conclusions

From our experiences, REDCap is a feasible and efficient electronic data capture software to use in further research in low-resource settings, especially in similar contexts to Colombia. The software allowed us to face (1) logistical problems, such as simultaneous data collection in different places, (2) privacy and data protection, and (3) remote data quality assurance. REDCap facilitated the whole data management process, which included collection, storing, and monitoring the data. Even if some REDCap features were not as intuitive for staff and participants with little computer literacy, in our experience, a short training was enough to overcome this issue. The only requirement to fully implement data capture with REDCap is a reliable Internet connection, however, when the network is not stable, researchers can collect the data without it and pull the information into the central repository afterwards.

Ethical statement

This research project was approved by the Institutional Review Board at Dartmouth College in the United States of America and Pontificia Universidad Javeriana in Colombia.

Funding

Research reported in this publication was funded by the National Institute of Mental Health (NIMH) of the National Institutes of Health (NIH) under Award Number 1U19MH109988 (Multiple Principal Investigators: Lisa A. Marsch, Ph.D. and Carlos Gómez-Restrepo, MD). The contents are solely the opinion of the authors and do not necessarily represent the views of the NIH or the United States Government.

Conflicts of interest

Authors report not having any conflicts of interest. Dr. Lisa A. Marsch, one of the principal investigators on this project, is affiliated with the business that developed the mobile intervention platform that is being used in this research. This relationship is extensively managed by Dr. Marsch and her academic institution.