Phenomenological toolkit of the metaverse for medical informatics’ adaptive learning

Abstract

Introduction

The concept of "adaptive learning systems" assumes development and implementation of personalized learning platforms that adapt to students’ learning strategies, the sequence and difficulty of the task abilities, the time of feedback and students’ preferences. Metaverse technologies can implement adaptive personalized learning to a greater extent due to the possibilities of wider application of student-oriented technologies and provision of rational support for personal educational progress, taking into account individual characteristics.

Material and methods

The article describes the phenomenological toolkit of the metaverse based on the Microsoft solution to ensure adaptive learning of medical informatics, taking into account modern technologies.

Results

Two components of the implementation of adaptive learning with Medical Informatics are suggested: (1) creating responsive, professionally relevant learning content that spans simulated accurate digital models of the real world and augmented reality, (2) creation of a metaverse virtual environment for the organization of immersive adaptive learning based on a combination of personally oriented design and productive interaction.

Our choice of the Microsoft solution assumes the possibility of implementing in the educational environment of the metaverse such educational tools as voice virtual assistants with artificial intelligence support (Microsoft Bot solution), Internet of things for education and generating educational data (Azure IoT, Power Platform), mirror worlds of educational destination (Microsoft Twins), augmented, virtual, and mixed reality (Microsoft Mesh, Microsoft HoloLens, Dynamics 365 Guides).

Conclusion

The proposed solutions of Microsoft make it possible to ensure high functionality in a number of issues when building a student-oriented virtual environment of the metaverse for adaptive and authentic learning.

Resumen

Introducción

El concepto de "sistemas de aprendizaje adaptativo" supone el desarrollo y la implementación de plataformas de aprendizaje personalizadas que se adapten a las estrategias de aprendizaje de los estudiantes, la secuencia y dificultad de las habilidades de las tareas, el tiempo de retroalimentación y las preferencias de los estudiantes. Las tecnologías de metaverso pueden implementar un aprendizaje personalizado adaptativo en mayor medida debido a las posibilidades de una aplicación más amplia de tecnologías orientadas al estudiante y la provisión de apoyo racional para el progreso educativo personal, teniendo en cuenta las características individuales.

Material y métodos

El artículo describe el toolkit fenomenológico del metaverso basado en la solución de Microsoft para garantizar el aprendizaje adaptativo de la informática médica, teniendo en cuenta las tecnologías modernas.

Resultados

Se sugieren dos componentes de la implementación del aprendizaje adaptativo con Informática Médica: (1) crear contenido de aprendizaje receptivo y profesionalmente relevante que abarque modelos digitales precisos simulados del mundo real y la realidad aumentada, (2) creación de un entorno virtual de metaverso para la organización de aprendizaje adaptativo inmersivo basado en una combinación de diseño orientado personalmente e interacción productiva.

Nuestra elección de la solución de Microsoft supone la posibilidad de implementar en el entorno educativo del metaverso herramientas educativas tales como asistentes virtuales de voz con soporte de inteligencia artificial (solución Microsoft Bot), Internet de las cosas para la educación y generación de datos educativos (Azure IoT, Power Platform), mundos espejo de destino educativo (Microsoft Twins), realidad aumentada, virtual y mixta (Microsoft Mesh, Microsoft HoloLens, Dynamics 365 Guides).

Conclusión

Las soluciones propuestas por Microsoft permiten garantizar una alta funcionalidad en una serie de problemas al construir un entorno virtual del metaverso orientado al estudiante para un aprendizaje adaptativo y auténtico.

Introduction

Promising and innovative opportunities for the transfer of medical knowledge are currently opening up for medical education thanks to the technologies of the metaverse. The concept of metaverse means a combination of the virtual and real world, which is formed as a result of the integration of many innovative technologies. In the metaverse, users can work with virtual reality objects and interact with each other through avatars using virtual, augmented, or mixed reality technologies (Virtual, Augmented, Mixed, and Extended reality – VR, AR, MR, XR).

At the same time, users being in their real-world environment could experience possibility of virtual teleportation and achieve the effect of joining with the simulated virtual world. It is providing the advantages for distance learning.¹

Metaverse is a multidisciplinary technology that is formed by integrating developments from different fields. The advances of such technologies as VR, AR, MR, XR, digital twins, Internet of things (IoT), artificial intelligence (AI), blockchain strengthens the role, scaling and functionality of the metaverse in the field of health care. VR, AR, and MR technologies provide an interactive component of the meta-universe for an immersive experience. The term and concept of VR refers to a digital world created by technical means that is a fully virtual online 3D semi-reality that simulates complex components of the real world (e.g. virtual hospitals, virtual therapy rooms, virtual medical simulations, etc.).²

AR augments the real world with virtual content by superimposing digital images on physical objects (e.g., 3D medical imaging).³ In MR, virtual and physical objects are combined and can interact in real time (e.g., the use of HoloLens to display holographic instructions and visualizations in order to accurately perform medical procedures). XR technology is designed to reproduce in the virtual world various tactile sensations with the help of sensors and actuators that ensure a seamless interaction between the real and virtual worlds.³ Such technologies are already used in telesurgery and telemedicine.

Digital Twins can be used to mirror and explore the real world in the metaverse. They are a virtual model of a physical entity with dynamic bidirectional IoT communications between the physical entity and its corresponding counterpart in the digital environment.⁴ Examples of Digital Twins in medicine can be the patient's entire body or a separate system, a diseased organ, disease, syndrome, cells of a certain type, molecular processes, and there can also be corresponding digital twins of organizations. Digital Twins play a key role in the development of personalized approaches in medicine and education. They allow you to study and discover new knowledge, generate effective hypotheses, and test them.

IoT is an ecosystem of interconnected devices and sensors that collect data for further analysis.⁵ In the field of health care, the important tasks of IoT sensors are to regulate, monitor, control, and track the patient's health indicators.

Important for the metaverse are AI technologies, which include Machine and Deep Learning, Computer Vision, and Natural language processing. They allow automated processing of various data, interpretation of voice instructions, analysis of processes, and generation of effective solutions. Blockchain technologies based on smart contracts allow building a reliable metaverse system for medical education and health care with a high degree of data protection.

Current research results, highlighted in systematic reviews and meta-analyses, testify to the effectiveness of the use of immersive technologies in medical education.⁶ The use of these technologies creates conditions for the implementation of authentic training in order to form a valuable professional experience that is close to the real thing when working with AR, VR, 3D authentic simulations, 360-degree images, and video reviews. The concept of authentic learning is understood as an educational approach that uses high-fidelity virtual simulation to conduct research and gain knowledge in contexts involving real problems and projects.⁷

Adaptive learning has significant educational potential for improving learning outcomes in medical education.⁸ The concept of "adaptive learning systems" assumes development and implementation of personalized learning platforms that adapt to students’ learning strategies, the sequence and difficulty of the task abilities, the time of feedback and students’ preferences. Metaverse technologies can implement adaptive personalized learning to a greater extent due to the possibilities of wider application of student-oriented technologies and provision of rational support for personal educational progress, taking into account individual characteristics.

These questions require complex researches related to consideration of metaverse technologies and adaptive learning taking into account the phenomenological approach. This will make it possible to define key technological innovations for their potential mutual enhancement and implement the construction of a functional metaverse environment. In this context, educational decisions regarding the educational applications of metaverse technology need justification. Metaverse technologies are especially promising in medical education, in particular, when future doctors study the "Medical Informatics" (MI) course, which is a fundamental basis for further mastering professional digital competencies.

Analysis of recent research and publications

Various issues related to particular metaverse technologies and their applications in education have been addressed in many studies. We conducted an analysis of scientific publications within period 2017–2023 in Elsevier, Springer, PubMed, IEEE, Willey, and Google Scholar. In the recent papers during 2023, scientists define the essence of the metaverse concept, its taxonomy, substantiate the structure and features of the metaverse, explore the potential applications of this technology for education, and propose topics for future perspective research.^9–14

The use of metaverse technologies in medical education and the advantages of this technology in emergency medicine are thoroughly covered.¹ The types of metaverse are defined and the potential and limitations of metaverse educational applications are explained.¹⁵ A detailed analysis of the application of IoT in education, as one of the technologies of the metaverse, was carried out in the study.^16,17

Modern approaches to the formation of a doctor's digital competences are substantiated in a number of studies which are devoted to the technology of individual training in the distance education, the cloud-oriented learning environment, synergistic principles of education’s modernization in the field of medical and biological physics and informatics, immersive technologies in distance learning systems, the implementation of cyber and robotic technologies in MI teaching.^18–22

Despite the presence of a significant number of studies related to the use of metaverse, virtual, and augmented reality technologies in education, there are a number of issue a still incomplete, promising, and needs to be resolved: (1) phenomenological substantiation of metaverse technologies toolkits and their educational potential, (2) creation of a student-oriented virtual metaverse environment to ensure adaptive learning of MI.

The purpose of the research: to define and characterize the phenomenological toolkit of the metaverse based on the Microsoft solution to ensure adaptive learning of MI, taking into account modern technologies, to analyze the Microsoft solution in the creation of a student-oriented virtual environment of the metaverse to improve the methods of teaching and transferring knowledge in MI.

Research materials and methods

To substantiate the phenomenological tools of the metaverse, a review of literary sources was conducted using bibliographic databases (ScienceDirect, PubMed, etc.) for the period 2017–2023. In Fig. 1, by means of the VOS viewer service, a visualization of the integration potential of metaverse technologies in health care and medical education is presented using the bibliographic data indexed in PubMed for the period from 2021 to 2023.

Fig. 1.

Visualization of the integration potential of metaverse technologies in health care and medical education based on bibliographic data indexed in PubMed for the period from 2021 to 2023 (additional interactive visualization settings is accessible by the link https://tinyurl.com/2dux38xk).

The literature review made it possible to determine the main existing directions of scientific research in the specified area, and accordingly to propose a phenomenological toolkit of the metaverse in integration with Microsoft solutions, which provides improved opportunities for studying simulated phenomena of biomedical systems in virtual reality.

The proposed research examines the phenomenological toolkit of metaverse technologies, which provides an opportunity to gain experience in MI adaptive learning in the metaverse environment. For this purpose, we defined a framework and conducted a parametric evaluation of metaverse technical solutions and phenomenological toolkit to ensure high functionality for the integration of technologies that make up the metaverse and allow organizing MI adaptive and authentic training with the possibility of including high-precision virtual modeling, conducting research work in virtual laboratories of the metaverse and building a student-oriented virtual environment of the metaverse.

The next step of the research was the analysis of technical solutions that comprehensively cover all the necessary components of the metaverse and ensure the organization of adaptive and authentic learning. As a result of the study, the implementation of the metaverse phenomenological toolkit in the development of MI projects such as "Virtual patient as a metaverse" and "Virtual hospital in metaverse" is proposed. We attribute the considered solutions to the phenomenological toolkit for medical informatics’ adaptive learning, understanding by this concept the toolkit intended for the study and observation of phenomena in the environment of the metaverse.

In the medical universities, significant part of informatics training of students is carried out during studying the Medical (or Health, or Clinical) Informatics course and involves competences formation which are mainly associated with the use of broad range of digital technologies (DT) in the field of health care.^18,23–25 The study of MI creates the necessary prerequisites for the professional development of future doctors capable of working in the conditions of high-tech medicine.

The content of the professional DT competence of a doctor supposed to include various directions of DT application in biomedical research and the health care, starting with the study of molecular and cellular processes and continuing with patient care.¹⁸ Among the broad range of main expected DT competencies of a doctor, we can distinguish the following most common: (1) DT in the study of molecular and cellular processes, genomic and pharmacokinetic modeling, study of the action and synthesis of drugs, (2) DT in medical visualization, processing of medical images and biosignals using DICOM (Digital Imaging and Communications in Medicine), and PACS (Picture Archiving and Communication System), (3) DT in the clinical and scientific practice of a doctor, which includes the use of computerized systems for diagnosis, monitoring, and forecasting of diseases, medical information systems, online services for patients and doctors, telemedicine systems, interaction with eHealth and mHealth applications, VR, AR, AI, IoT, robotic technologies, and (4) DT in public health.

As a result of MI studying, medical students develop a system of scientific, innovative, and research knowledge and skills about basic information principles and approaches to the study of processes in living nature, the use of mathematical and statistical methods in biomedical research, modern digital innovations, which are an integral component of professional doctor’s competence.

Our research was carried out at the Bogomolets National Medical University (Kyiv, Ukraine) at the Department of Medical and Biological Physics and Informatics (MBPI), where the authors of the article work. The MBPI department provides teaching of STEM-oriented disciplines for master's degree students in the specialties "Medicine", "Pediatrics", "Therapy and physical rehabilitation", "Medical psychology", "Dentistry", "Pharmacy", "Public health" of the first- and second year of study. For the medical students the basic STEM disciplines such as "Medical informatics" and "Medical and biological physics" are defined as mandatory courses in the university curriculum. In addition, variable components of curriculum are provided with a number of comparably short courses which are constantly updated and supplemented. For example, students can choose to study the following disciplines: "Medical information systems", "Modern information and computer technologies", "Telemedicine and electronic health", "Information technologies in scientific research", "Electronic health care system", "Artificial intelligence in medical diagnosis and treatment", etc.

Every year, the MBPI department conducts training for approximately 3000 first- and second-year master's degree students. In the basic MI course, due attention is paid to the study of digital medical imaging. Students get acquainted with various types of medical images, study 3D anatomical models in the VR and AR environment, learn modern digital technologies in the processing of medical images. In the constantly conducted course evaluation surveys, the students testify to a high level of interest in modern digital technologies and especially metaverse technologies. This encourages the use in the educational process of modern virtual medical laboratories (e.g., the Acadicus platform), interdisciplinary STEM projects for health care (e.g., the Labster platform, Microsoft Healthcare tools), metaverse projects (e.g., 3D VR animated tours with full immersion into a virtual human body, educational medical programs with game design in a virtual environment, projects "The Internet of medical things in the patient-oriented digital clinic ecosystem", "Virtual patient as a metaverse", and "Virtual hospital in metaverse", etc.).

Considered here phenomenological toolkit of the metaverse allows further implementation of the planned practical phase of the phenomenological study in order to evaluate the experience and perception of the metaverse projects for adaptive MI learning from the point of view of users (students, teachers, and technical development specialists). For the participants of this planned stage of the phenomenological study, familiarization with the available components of the metaverse and the organization of learning will be provided. After the metaverse experience, each research participant will be interviewed. In addition, participants will be asked to fill out a questionnaire about their experience working in the metaverse environment and usability evaluation. The results of the evaluation of the practical stage of the phenomenological research will be considered in a separate article.

A key feature of the metaverse is the convergence of the virtual and real worlds. The metaverse is considered the next generation of the Internet, which radically changes the way users interact, eliminating the separation of the virtual and real world. It is a multifunctional virtual universe for multi-sensory user interaction in a virtual simulated environment combined with the physical real world and integrated with various innovative technologies.

In the overview of the metaverse roadmap, 2 key continuums are chosen to analyze the deployment of this technology.²⁶ Those continuums forming separate axes:

(1) Identity/World – illustrates the spectrum of metaverse technologies, which varies from orientation on participator’s identity, the manifestation of an individual, his/her activities in the metaverse through the use of an avatar to orientation on the world around the user and it interaction;

(2) Real Worlds/Constructed Worlds – illustrates a range of technologies ranging from an orientation on the real world by supplementing it with new metaverse capabilities to an orientation on simulated virtual worlds, i.e., created parallel realities, offering a completely new simulated environment in the context of the metaverse.

The combination of these key continuums gives 4 structure components of the metaverse^15,26: (1) Augmented reality (AR), (2) Lifelogging, (3) Mirror worlds, (4) Virtual worlds (VW) (Fig. 2).

Fig. 2.

Metaverse model roadmap with key continuums and structure components.

Such a model, which was used in our study, allowed us to display the existing educational and technical solutions for the application of metaverse technologies in the teaching.

As a result of our analysis of educational and technical solutions for creating a student-oriented virtual environment of the metaverse and ensuring adaptive learning, Microsoft's solution turned out to be the most comprehensive. This solution covers modern technologies which are potentially deeply integrated to the metaverse (e.g., AI, VR, AR, MR, IoT, wearable devices, digital twins, big data, cloud computing, blockchain, etc.), and also which sufficiently correspond to the functional implementation of all structure components of the metaverse model roadmap. Other educational and technical solutions allow as well ensuring the application of metaverse particular components, while illustrating the effectiveness and educational advantages compared to traditional teaching methods.

Concept development

In the general structure of the metaverse, such main components as hardware, software, and content are distinguished. The structure of the metaverse for educational purposes, compared to traditional or distance learning, has a more complex architecture and involves the combination of powerful technologies such as AI, VR, AR, MR, XR, IoT, etc. The advantage of choosing comprehensive end-to-end solutions from Microsoft is their effective integration in the Metaverse.

The Microsoft Mesh platform works on the basis of Microsoft Azure Cloud technology and implements 3D-holographic virtual collaboration using mixed reality applications. The metaverse at Microsoft comes to Teams via Mesh. One can join a virtual meeting using a HoloLens mixed reality display device, VR headsets, mobile phones, tablets, or PCs through Mesh-enabled applications. Recommended programs are Mesh for Teams, Mesh App for HoloLens, and AltspaceVR. Microsoft's metaverse is not limited to VR, but covers all areas of the metaverse, including AR. The advanced HoloLens mixed reality system allows to superimpose a metaverse on the surrounding world. Mesh broadcasts a full 3D environment and displays dynamic avatars of users synchronized with their emotions and movements in real time. Significantly, Microsoft provides a wide range of cloud services for use in the metaverse.

The creation of digital twins using the Azure Digital Twins solution is particularly promising for immersive adaptive training in MI. This solution allows modeling of any subdivision, system, or entire environment, keeping digital twins up-to-date with Azure IoT in real time. Digital copies of existing doctors' workplaces and complex digital models of university clinics and healthcare facilities can be synchronized with the physical world using IoT and IoMT connections. Gaining unique efficiency in the process of integrating real and synthetic healthcare data, analyzing and modeling processes, tracing the history of digital twins and predicting future states is possible obliged to the combination of such Microsoft technologies as Power Platform, Azure AI, Azure Synapse Analytics, etc.

A wide range of didactic opportunities can be implemented in a student-oriented virtual metaverse environment using Microsoft Dynamics 365 Guides. This solution provides the reproduction of step-by-step holographic instructions to guide students to work better and be more accurate in completing tasks.

One of the means of implementing adaptive personalized learning can be the use of IoT technology. The Microsoft Azure IoT solution allows you to connect physical devices, programs, and tools for comprehensive processing and analytics of student educational data, creating longitudinal personalized records of behavior and educational activities. Such IoT devices can include the following: (1) accessories for effective time management, learning coaching, tracking educational progress and productivity, personal success and achievements (e.g., smartphone, smart watches, biosensors for collecting biometric data and tracking health indicators, neurosensors for determining the cognitive activity of the brain and psycho-emotional indicators, sensors for measuring the level of fatigue, satisfaction, motivational indicators, devices for monitoring sleep and daily routine, fitness bracelets for optimal support of physical fitness, smart vests, e-textiles, smart pens, smart notebooks, etc.); (2) sensors for tracking environmental parameters and monitoring the learning process.

The use of IoT for educational purposes enhances adaptive personalized learning in the virtual environment of the metaverse. Comprehensive processed data from personal devices and the results of the user's educational success in the learning management system are the basis for personal reflection and further determination of ways of rationalization educational processes and achieving personal educational success.

At the same time, the Microsoft Power Platform solution offers a wide range of technologies that will be useful for the implementation of personally oriented training design. Among them, the defining technologies are Power Virtual Agents and Power BI. Microsoft services for chatbots allow you to create powerful personalized learning coaches. The strength of the Microsoft Bot solution is the support of a voice interface with AI, which makes virtual assistants as realistic as possible.

The Power BI solution, which is part of the Microsoft Power Platform, is important for construction of a student-oriented virtual environment of the metaverse. This technology can provide comprehensive processing and analytics of the student's educational data using information panels, analyze longitudinal personal data and reports on academic performance in real time. Such an implementation is significant in ensuring adaptive learning and planning educational activities based on personalized approaches.

The advantage of using Microsoft solutions for the metaverse is a high level of security, identification, and privacy of user data. These critical issues can be adequately addressed with Azure Blockchain Solution, Multi-Factor Authentication, and Microsoft Defender, etc. In general, Microsoft provides the creation, control, and management of high-performance metaverse infrastructure from the edge to the cloud level.

Results

In Table 1, we offer a selection of metaverse phenomenological tools for educational purposes based on the Microsoft solution, taking into account the metaverse roadmap.

Table 1.

Selection of metaverse phenomenological tools for educational purposes based on Microsoft solutions, taking into account the metaverse roadmap.

Metaverse structure components	Metaverse-associated Microsoft solutions for educational purposes	Educational purpose
Mirror worlds	• Azure Digital Twins to create digital copies of existing workplaces and complex digital models of physical worlds that are synchronized through IoT connections;• Power Platform, Azure AI, Azure Synapse Analytics for monitoring the work and data processing of digital twins;• Azure Maps to locate things.	Obtaining simulated learning content, accurate digital representation models of the real physical world, provision of authentic training.
Augmented reality	• HoloLens mixed reality system for overlaying the metaverse on the surrounding world;• Microsoft Dynamics 365 Guides for reproducing step-by-step holographic instructions.	Creating supplemented educational content for additional information.
Lifelogging	• Azure IoT to ensure the connection of physical devices, programs, assets for control, management and automation of data processing processes from the edge to the cloud level;• Microsoft Power Platform for rationalization of tasks using interconnected technologies Power BI, Power Apps, Power Automate, Robotic Process Automation, and Power Virtual Agents;• Microsoft services for creating chatbots;• Azure AI for access to high-quality AI models, execution of common AI scenarios in the learning process.	Implementation of person-oriented educational design, personalized training, use of technologies aimed at education seekers, development of their talents and creativity.
Virtual worlds	• Mesh for Teams, Mesh App for HoloLens, AltspaceVR for virtual collaboration in the metaverse.	Ensuring interaction and cooperation with others, synergy in the joint solution of issues and projects.

The proposed selection of phenomenological tools allows us to determine the educational goals that can be achieved with the help of the proposed Microsoft solution for the metaverse. In this implementation, the Mirror Worlds and AR components illustrate the possibility of obtaining simulated educational content, accurate digital models from the real physical world and combining it with AR, which provides authentic learning. The Lifelogging and Virtual Worlds components reflect the implementation of person-oriented educational design, personalized learning, the use of technologies aimed at education seekers, the development of their talents and creativity, which is combined with productive cooperation with others, and the possibility of achieving synergy in the joint solution of issues and projects.

Thus, we propose 2 components of the implementation of adaptive learning with MI: (1) creating responsive, professionally relevant learning content that spans simulated accurate digital models of the real world and augmented reality, (2) creation of a metaverse virtual environment for the organization of immersive adaptive learning based on a combination of personally oriented design of the beginning for the acquisition of significant individual educational achievements of students and productive interaction, cooperation with others to achieve synergy in joint solving of issues, and implementation of projects.

The advantage of using virtual teaching assistants in the metaverse is their ability to constantly support and monitor the student's educational activities. They provide a guided learning path, support motivation, help with assessment, measuring learning progress, and providing feedback. Virtual assistants can shape and deliver learning content for learning based on the student's individual learning progress.

An effective application of voice virtual agents in education is the possibility of introducing such educational practices with proven effectiveness as interval training and microlearning. In accordance with this educational practice, the chatbot can schedule a specified interval of repetition of the educational material for better retention in long-term memory. All these contribute to the maximization of the student's personal educational success, the development of scientific potential, creativity for readiness for productive interaction, cooperation with others in the virtual environment of the metaverse.

The result of our study was the rationale for the implementation of the "Virtual Patient as metaverse" (VPaM) model in the process of teaching MI. To build VPaM, we chose the Microsoft solution integrated with Microsoft Cloud for Healthcare to create a student-oriented virtual MI learning environment in a metaverse with support for modern AR, VR, AI, and IoT technologies. The purpose of creating the model VPaM is developing of virtual learning environment elements and appropriate phenomenological tools for the formation of students' system of scientific knowledge and practical skills. With the help of VPaM, students will have an opportunity to apply metaverse cognitive technologies in the process of modeling patients’ digital twins for further mastering the diagnosis, treatment, prevention, and care of the patient's health.

Within MI subject study and implementation of immersive adaptive learning in the process of working on the VPaM project, it is acceptable to combine Mesh for Teams and HoloLens with educational solutions for high-precision medical simulators (e.g., CAE Healthcare), anatomical laboratories (e.g., HoloAnatomy), 3-dimensional holographic human atlases (e.g., HoloPatient), etc. Such implementations allow students to assess, diagnose, and treat a variety of standardized health conditions in virtual patients using holographic simulations.

Specialized industry solution Microsoft Cloud for Healthcare containing a set of modules Virtual Health, Virtual Visits, Virtual Clinic, Home Health, Care Management, Patient Portal, Personalized care, Health Analytics, Azure Health Data Services, Azure Health Bot, etc., can also be one of the components for implementing the VPaM model in MI education. Its application demonstrates the integrative components of the modern digital healthcare ecosystem and the adoption of various eHealth innovations.

The Microsoft Cloud for Healthcare solution provides: (1) automation of various work processes of a doctor and a healthcare facility, (2) development of individual patient healthcare programs, (3) organization of remote patient monitoring, home telemedicine, (4) creation longitudinal patient records based on HL7 FHIR (Fast Healthcare Interoperability Resources), (5) performing effective healthcare data analysis, (6) performing predictive and clinical analytics to identify clinical trends, (7) using voice-based virtual healthcare assistants AI-enabled, (8) application of clinical decision-support systems and medical intelligence services based on evidence-based approaches for diagnosis, treatment, and prevention, etc. In addition, the valuable feature of that solution could be its ability to convert DICOM digital medical images (e.g., CT, MRI, US, PET, etc.) into mixed reality 3D holograms for better viewing of individual patient anatomy and pathologies (Holo Dicom solution for Microsoft Hololens).

Microsoft Health Bot Service is a cloud-based platform that healthcare organizations can use to build and deploy AI-powered virtual medical assistants and bots. Microsoft Cloud for Healthcare focuses on the ability to create and deploy powerful AI-enabled virtual voice assistants. Built-in medical intelligence to support clinical decisions is key feature for creating functional voice care assistants. It is possible to include your own scripts and integrate with other IT systems and data sources.

Importantly, Health Bot uses several trusted providers of medical content to create and test clinical content. For example, Microsoft may offer an effective integration of Infermedica expert system in the development of virtual healthcare providers. Infermedica offers an improved mechanism for analyzing the patient’s symptoms by ML means with a high level of clinical accuracy. Infermedica decision-making system is based on clinically validated probabilistic models and evidence-based approaches. The Capita Healthcare Decisions Triage resource also has strong capabilities to implement the VPaM model and support the adoption of evidence-based clinical guidelines.

The Microsoft Genomics solution offers modern genomic technologies and ML models to enable the analysis of genomic data on Azure, which can be implemented in the VPaM model as a tool for precision medicine research. Genomics Notebooks provides the power of Jupyter Notebooks on Azure to analyze genomic data using the GATK, Picard, Bioconductor, and Python libraries. It provides a wealth of R-based bioinformatics tools for analyzing and understanding high-throughput genomic data, leveraging the available Genomics data repository on the Azure Open Dataset platform.

In order to implement the VPaM model in MI education, we suggest the enhancement of the phenomenological toolkit for adaptive learning with: (1) Microsoft Cloud for Healthcare industry solution for mastering modern digital innovations and e-health services for patient health care, (2) Microsoft HoloLens integrated with CAE Healthcare, HoloAnatomy, HoloPatient, HoloDicom, Infermedica expert system, and The Capita Healthcare Decisions Triage resource for Microsoft Health Bot for acquiring medical visualization skills in the metaverse, using computer systems to diagnose, monitor, and predict diseases based on EMB (Evidence-Based Medicine) approaches, (3) Microsoft Genomics for genomic data analysis and precision medicine research. In Fig. 3, we suggest choosing the appropriate metaverse phenomenological toolkit to ensure adaptive training in MI and the implementation of the VPaM project based on the Microsoft solution.

Fig. 3.

Microsoft metaverse solutions for medical informatics’ adaptive learning.

As a result of our research, it was established that the phenomenological toolkit of the metaverse chosen by us based on Microsoft's decision improves the educational activities of students and maximizes the educational advantages of the metaverse by:

• (1)

  Provision of personalized, adaptive training taking into account the individual characteristics of education seekers, construction of a controlled learning path, selection of personalized educational content, and continuous individual feedback based on AI data analysis.

• (2)

  Implementation of ubiquitous learning, microlearning with a high level of mobility and involvement.

• (3)

  Provision of authentic and experimental training, formation of professionally significant competences.

• (4)

  Improvement of knowledge acquisition processes due to minimization of cognitive load, organization of interval training for interactive repetitions, activation of knowledge, and ensuring its long-term maintenance.

• (5)

  Introduction of student-oriented educational design, organization of educational coaching, and support of motivation with the help of powerful virtual assistants with AI.

• (6)

  Creation of a learning environment based on cooperation, maximum involvement of students in learning.

• (7)

  Use of wearable gadgets to record productivity and form positive motivation.

• (8)

  Transition from linear learning materials to learning based on individual requests through the network design of learning content.

• (9)

  Implementation of diagnostic and formative assessment, feedback with AI support.

• (10)

  Unique opportunities for remote collaboration and online communication through avatars, haptic sensations which facilitates batter skills of working with virtual objects.

Discussion

In our research, we analyzed good practices of metaverse using and the implementation of metaverse projects in health care. PubMed bibliographic database results include 2 systematic reviews (as of January 2023) investigating the potential of the metaverse in education (in particular, the metaverse in distance learning,²⁷ the metaverse in English language learning).²⁸ The results of these studies prove that metaverse platforms are increasingly used in education and arouse significant interest among students.^27,28

The most common solutions for organizing training in the metaverse is the Second Life²⁷ platform. It allows you to create exciting learning experiences for students in various STEM fields, includes virtual laboratories, realistic simulations, has opportunities for conducting experiments, offers virtual tours related to science. Along with this, other platforms of the metaverse are used in the educational process, in particular Engage (offers virtual classes, collaborative learning, tools for creating interactive content), Minecraft (allows you to gain practical experience in a creative mode), Breakroom (combines VP and AI, contains educational simulations and interactive content), AltspaceVR (provides the opportunity to hold various events, master classes, presentations, discussions), etc. The advantage of these solutions is availability, relative ease of use, no need for complex technical deployment and system administration. However, the functionality of these environments is still limited. The issue of security and data privacy due to significant vulnerability to hacker attacks also raises concerns.

For the organization of MI training in institutions of higher medical education, the processing of sample digital medical images, electronic medical records, the use of electronic health services, work in medical virtual laboratories for the implementation of research projects, for which an important requirement is high reliability and data protection, is provided. This encourages the development of highly functional, reliable metaverse solutions and digital innovations that can be harmoniously integrated into a single digital ecosystem of a university clinic compatible with an eHealth system. At the same time, it is important to choose complex digital solutions for a medical university, which is by its instance an innovative center for creating new knowledge, conducting scientific medical research, supporting powerful experimental work, and introducing innovative electronic health services for patients and doctors. These issues require a rather complex technical implementation and the due work of the university’s technical division to support such an emerging digital ecosystem.

Analysis of research results shows that modern clinics and medical universities are trying to develop their digital ecosystem by introducing Smart Health in their institutions. For the implementation of metaverse technologies, considerable attention is paid to the use and development of specialized VR platforms with appropriate functionality for health care. The most common platforms for training surgeons (Precision OS, Osso VR, OpenSurgery, SurgicalAR, etc.), precise planning of surgical intervention, creation of individual 3D models of patient anatomy, training of complex procedures before surgery (Surgical Theater, etc.), research of models of human heart (Stanford Virtual Heart, etc.), visualization of patient data, planning of surgical intervention, and remote collaboration between medical professionals (Microsoft HoloLens and Magic Leap), research of 3D models of the human body in teaching anatomy and visualization (3D Organon, VR Anatomy, etc.).

Among the described applications and implemented projects of the metaverse, the following can be distinguished: metaverse in emergency medicine,¹ cardiovascular medicine,² neurorehabilitation,³ gastroenterology,²⁹ cancer care,³⁰ surgery,³¹ ophthalmology,³² and others. There is a high interest in health care for Microsoft solutions. The analysis of research results confirms the wide application of the Microsoft HoloLens solution in health care and metaverse projects. Seven meta-analyses and systematic reviews examining the potential of Microsoft HoloLens in the contexts of medicine, clinical care, and surgery were reviewed and analyzed. Convincing advantages of using the Microsoft HoloLens solution in medicine have been proven.³³

The Gig Immersive Learning platform offers a library of various holographic programs with Microsoft HoloLens support for studying 3D models of the human body. For example, the virtual expedition "Insight Heart" is dedicated to the study of the human heart. The virtual assistant in this simulation demonstrates various heart conditions, including normal heart rate, myocardial infarction, hypertension, atrial fibrillation, coronary artery disease, and atrial fibrillation. The program offers students the opportunity to simulate one of the many forms of the disease in order to visualize the effect of this pathological condition on the organ and other parts of the body. In many cases, this makes it possible to study examples of very rare or critical conditions, which are practically impossible to study in the conditions of a real clinic.

It is worth noting that the Microsoft corporation actively participates in projects and initiatives in the field of medicine and medical education. The corporation implemented a number of useful solutions for health care. These include: Microsoft Healthcare Bot (an AI-based chatbot platform for healthcare organizations that enables such agents to interact with patients, automate routine tasks and provide personalized health advice), Project InnerEye (uses AI for medical image analysis, radiation therapy planning), Microsoft Azure for healthcare (offers a range of services specifically designed for healthcare, providing secure data storage and powerful analytics capabilities to support healthcare organizations), Microsoft Genomics (supports analysis of genomic data), HoloLens, etc.

Microsoft played a key role in the development and improvement of ChatGPT, which is a product of OpenAI. OpenAI, together with Microsoft, used knowledge and technological expertise to create a powerful text generation model based on the GPT (Generative Pre-trained Transformer) architecture. ChatGPT can help answer common questions about medical terms, symptoms, treatments, and procedures, as well as provide quick access to current medical knowledge and updated information. ChatGPT can also be used as an interactive learning tool for medical students and other medical professionals. It could help with the study of anatomy, physiology, pathology, and other medical disciplines, answer questions and provide explanations.

These projects demonstrate Microsoft's commitment to innovation, using AI, cloud computing, and mixed reality to advance medical education, health care. and improve patient outcomes.

Microsoft's broad set of solutions gave us the opportunity to substantiate the phenomenological toolkit of the metaverse for AI adaptive learning. The proposed toolkit covers all types of metaverse according to the roadmap. This creates an opportunity to hold virtual meetings in the metaverse during distance learning, while integrating Microsoft Office 365 and Teams tools. This integration allows users to collaborate, interact, communicate, and access their workflows through the metaverse environment. The metaverse experience is greatly enhanced by Microsoft's AI capabilities and virtual voice assistants with natural language processing. For research projects and the interaction of the physical and virtual worlds, Microsoft supports the deployment of Digital Twins. This component currently has less use in the metaverse compared to other toolkits. But it is promising in the further development and digital transformation of health care.

The proposed phenomenological toolkit allowed us create background and initiate 2 projects for the further implementation: "Virtual patient as a metaverse" (VPaM) and "Virtual hospital in metaverse" (VHM). With the help of VPaM, students will have an opportunity to apply metaverse cognitive technologies in the process of modeling patients' digital twins for further mastering the diagnosis, treatment, and prevention. Application in education of such an important innovation as VHM can allow medical students to work with a digital version of a university clinic. The integration of such modern technologies as AR, VR, AI, IoT in the VHM allows students to be introduced to a wide range of digital medical services, to form work experience in simulated clinical workplaces of doctors, virtual hospital wards, to look after the health indicators of digital twins, analyze the holographic representation of patients' vital statistics. Such implementation can be a component of the digital ecosystem of a university clinic, on the basis of which additional e-health services can be developed (e.g., metaverse treatment rooms to support the mental and cognitive health of patients and students, physical activity, motivational health coaching, awareness raising about health, and the formation of behavioral strategies for combating diseases in patients, etc.). This creates an environment for medical students to innovate and implement their own projects.

Using the concept of a digital twin, in which a virtual simulation of a patient is developed based on genetic data, medical history, health indicators generated from IoT wearable medical devices, the creation of an exact digital copy of the patient's anatomy and physiology can occur. This approach makes it possible to model a variety of interventions to improve health, analyze their effects, predict treatment outcomes, and identify disease risks. Accurate digital versions of patients allow us to understand the current state of health and identify trends of future changes, opening up new opportunities for research and in-depth study of various aspects of health.

Previously, we have developed the concept "The Internet of medical things in the patient-oriented digital clinic ecosystem"²⁰ based on the Microsoft Healthcare solution and substantiated the technical implementation of this innovation. We see a significant potential of Microsoft solutions for the organization of MI training, since the educational goals of this discipline are compatible with the innovations offered by Microsoft for health care and can be integrated into the educational process. In addition, the introduction of complex Microsoft solutions at the medical university will contribute to the development of research centers and the wider application of modern digital innovations in scientific work.

The similarity of our research compared to the existing works consists in considering the prospects of using the standard building blocks of the metaverse according to the road map, taking into account the potential of modern technologies.^{1–3,29–32} Instead, the difference and feature of our research is a clear focus on the MI content and the challenging task of integrating medical knowledge into the digital environment of the metaverse in order to improve the quality of education. Also, the proposed approach is part of the consistent development of the components of the complex digital ecosystem of the medical university.

The selected metaverse phenomenological toolkit for the organization of adaptive MI training has a number of technical and educational advantages. The technical advantages of the selected metaverse phenomenological toolkit include: powerful infrastructure; the possibility of cloud computing, which provides the necessary scalability and reliability of the platform; integration of various Microsoft tools, in particular integration of the Microsoft Healthcare solution; cross-platform compatibility, access from many devices and platforms, including PC, mobile devices, and VR headsets; reliable security measures, data protection, privacy control, compliance with user safety regulations, compliance with regulatory acts in the field of health care, such as GDPR (General Data Protection Regulation); providing the necessary computing power for healthcare applications; availability of personalized healthcare resources; Microsoft's support of medical research to improve healthcare delivery and drive healthcare innovation.

Further, we will consider the key educational applications of Microsoft's solution for the metaverse and their implications for learning.

Microsoft HoloLens solutions, Dynamics 365 Guides for Augmented Reality, and Azure Digital Twins, Power Platform, Azure AI, Azure Synapse Analytics solutions for Mirror Worlds.

Implementation of educational tasks:

• •

  unique possibilities of visualizing educational content and visualizing difficult-to-understand concepts;

• •

  interactive support in the formation of procedural skills with the help of immersive instructions;

• •

  organization of experimental and authentic training;

• •

  creation and use of a digitized textbook with AR support;

• •

  the use of a digital twins as a means of studying virtual representations of physical educational objects;

• •

  reproduction of individual educational trajectories of students with the help of digital twins to predict probable learning outcomes and support maximum educational efficiency;

• •

  optimization of educational processes;

• •

  development of scientific creativity as the basis of education 4.0.

Key advantages compared to traditional learning methods:

• •

  improvement of knowledge acquisition processes;

• •

  provision of authentic education;

• •

  creation of a learning environment based on cooperation and competences;

• •

  improvement of educational and scientific research and the possibility of determining effective strategies for improving training or experimental measures as a result of the analysis of the generated sensor data of a real object.

  Microsoft Azure IoT solutions, Power Platform, Microsoft services for creating chatbots, Azure AI for Lifelogging and Mesh for Teams solutions, Mesh App for HoloLens, AltspaceVR for Virtual Worlds.

Implementation of educational tasks:

• •

  support of the student's personal success and productivity by analyzing collected daily personal data and activity records;

• •

  the student's reflection on his own educational achievements;

• •

  creation of conditions for self-expression and development of creative abilities, communication, leadership, soft-skills;

• •

  time management for organizing interval training for interactive repetitions for long-term and productive retention of knowledge;

• •

  conducting interactive classes with full immersion and maximum involvement in VR, gaining significant authentic experience, professional competencies;

• •

  organization of multi-sensory experimental training and development of high-level thinking skills;

• •

  the possibility of acquiring the appropriate level of procedural skills through repeated training and repetition of educational scenarios in a safe, reproducible, exciting, and interactive environment;

• •

  organization of interactive cooperation in VR, development of creative potential.

Key advantages compared to traditional learning methods:

• •

  provision of personalized, adaptive training taking into account the individual characteristics of the students of education;

• •

  effective time management and organization of academic affairs;

• •

  ensuring maximum involvement in the educational process;

• •

  unique opportunities for remote collaboration, acquiring skills for working with virtual objects in VR, development of scientific creative potential.

Along with the advantages of the presented metaverse phenomenological toolkit for adaptive MI learning, there are certain limitations. In some countries, there may be restrictions on access to specialized hardware IoT devices for health care, telemedicine equipment due to their policies and strict requirements for the use of digital devices. In addition, specialized healthcare tools such as the Infermedica decision-making system and The Capita Healthcare Decisions Triage are mainly targeted for use in EU and US countries. This article substantiates the phenomenological toolkit, which allows us to justify the advantages and plan the further use of the metaverse technology in MI education. Detailed methodical aspects of the step-by-step application of this resource in teaching MI are not covered by this study. In the future, it is planned to conduct a phenomenological study to evaluate the experience and perception of users of the proposed metaverse toolkit for MI training.

It is important to be aware that when working in the metaverse environment and using virtual laboratories with animated effects, users may experience certain problems, in particular, symptoms of cybersickness (dizziness, deterioration of well-being, headaches). Various effects of metaverse environments can cause cognitive overload in users. A harmonious combination of traditional education and work with virtual laboratories of metaverse environments is appropriate to avoid these situations.

It is also worth noting the serious technical complexity of deploying and integrating the entire toolkit of the metaverse into the digital multifunctional university ecosystem. Applying the entire set of metaverse phenomenological tools will require teachers to have the necessary level of VR digital technology skills. Appropriate training should be planned in advance. Metaverse technologies are still in their infancy, but corporations are gradually developing and improving their metaverse solutions and making them more accessible for use in the educational process.

Conclusion

The proposed solutions of Microsoft make it possible to ensure high functionality in a number of issues when building a student-oriented virtual environment of the metaverse for adaptive learning:

• •

  organization of virtual meetings, creation of productive virtual educational environments, and implementation of immersive adaptive learning;

• •

  support for adaptive, personally oriented design of studies, motivational coaching, maximizing the student's personal success, individual development of scientific potential and creativity;

• •

  comprehensive processing and analysis of student educational data, creation of longitudinal personal records and reports on educational success, digital automation of educational processes;

• •

  creation, control, and management of high-performance metaverse infrastructure from the Edge to the Cloud level;

• •

  security, identification, and privacy of user data.

Tools for the implementation of person-oriented learning design are changing the educational paradigm. Due to these technologies, education becomes a trendy life concept and a lifestyle that encompasses all activities and requires continuous involvement and improvement. At the same time, under such circumstances, the role of technology, the role of teachers and students are based on cooperation and personalization of learning, maintaining the study in partnership, and seeking mutual professional development. Thus, there are broad opportunities to maximize the creative potential of each participant in the educational process.

Financial Disclosure

None reported

Human subjects or patients

None reported

Due to the characteristics of the study, the the ethics committee declarations are not necessary

Author contribution

Introduction, Conceptualization, Results, Discussion and Conclusion – Inna Kryvenko and Kyrylo Chalyy

Source of funding

There was no funding obtained for this project.

Funding information

There was no funding obtained for this project.