Comparison of the teaching of digital competences between health science faculties in Andalusia and Catalonia

Abstract

Introduction

Covid-19 brought great restrictions to face-to-face classes throughout the university system in Spain. In the field of health sciences, face-to-face teaching has been transformed into a virtual format. In most cases, teaching activities could not be prepared or planned, leading to unprecedented challenges for students and teachers, making it clear that some teachers have not developed an optimal level of digital skills.

Materials and methods

This descriptive-comparative study aims to analyse the level of digital teaching competence among the university health sciences teaching staff in two Spanish autonomous communities: Andalusia and Catalonia. 561 teachers participated in the study and answered the DigCompEdu check-In questionnaire.

Results

The results show how, in general, the level of their professed digital skills is basic, without significant differences between the participating communities. Further, technological mastery appears to be greater than pedagogical mastery with respect to one of the classroom technologies. The lack of technological training is highlighted and there is a discussion of the structuring of personalized learning itineraries by competences.

Conclusions

With regard to possible limitations of the research, we could reflect on the different areas for improvement and how comparative studies between future works with similar characteristics should be approached. One of the main weaknesses lies in the types of samples used, since the intentional sampling method indicates that it is not random, therefore, the results found should be considered non-extrapolatable to other areas of knowledge. Incorporating probabilistic samples for each area to be studied in future studies will allow the collection of more representative data that can be extrapolated globally.

Resumen

Introduction

Covid-19 trajo importantes restricciones a las clases presenciales en todo el sistema universitario español. En ciencias de la salud, la enseñanza presencial se transformó en un formato virtual. En la mayoría de los casos, no ha sido posible planificar las actividades docentes, lo que ha supuesto un reto para alumnado y profesorado, en el que algunos docentes no han desarrollado un nivel óptimo de competencias digitales.

Materials and Methods

Este estudio descriptivo-comparativo analiza el nivel de competencia digital docente de los profesores universitarios de ciencias de la salud de dos comunidades autónomas españolas: Andalucía y Cataluña. Participaron 561 profesores que respondieron al cuestionario DigCompEdu check-In.

Resultados

Los resultados muestran que, en general, el nivel de competencias digitales impartidas es básico, sin diferencias significativas entre las comunidades participantes. El dominio tecnológico parece ser mayor que el pedagógico con respecto a una de las tecnologías del aula. Se debate sobre la estructuración de los itinerarios de aprendizaje personalizados por competencias.

Conclusiones

En cuanto a las posibles limitaciones de la investigación, podríamos reflexionar sobre las diferentes áreas de mejora y cómo se deben abordar los estudios comparativos entre futuros trabajos de similares características. Una de las principales debilidades radica en los tipos de muestras utilizadas, ya que el método de muestreo intencional indica que no es aleatorio, por lo tanto, los resultados encontrados deben considerarse no extrapolables a otras áreas del conocimiento. La incorporación de muestras probabilísticas para cada área a estudiar en futuros estudios permitirá recopilar datos más representativos y extrapolables a nivel mundial.

Introduction

The teaching of digital competences is a matter of discussion in scientific literature. Digital competence in teaching is the framework under which teachers have to actively exercise their profession by using digital tools. However, digital competence does not only enable them to teach better, it also empowers them in their personal lives as digital citizens. However, the acquisition of digital competence in teaching does not happen all at once, it is a process that requires continuous training. The evolution of technology itself and the emergence of new pedagogical methodologies implies that there is an evolution in the acquisition of digital competences.

In 2013 the European Commission published the Digital Competence Framework for citizens, based on five areas and 21 competences in which reference is made to digital literacy.¹ This framework describes the different competences needed for a fully digital society. In 2017, the joint research centre (JCR) of the European Commission published the European Digital Competence Framework for Teachers "DigCompEdu".² The objective of DigCompEdu was to define specific digital competences for teachers at all educational levels, based on 6 areas and 22 competences. The 6 areas are:

• 1.

  Professional commitment: the ability to use digital technologies to improve professional interaction, including communication with the organization, students, and families, and network collaboration with other teachers.

• 2.

  Digital resources: the ability to use digital resources for teaching, as well as to create and co-create, modify, organize, protect and share these resources.

• 3.

  Digital pedagogy: the ability to integrate digital technologies throughout the different stages of a student-centred teaching-learning process.

• 4.

  Evaluation and feedback, focused on the use of digital tools and strategies to improve different evaluation approaches

• 5.

  Empower students, related to accessibility, inclusion and personalization, in addition to the active participation of students.

• 6.

  Facilitate the digital competence of students to be digital citizens.

Several authors discuss the different existing international frameworks widely cited in the scientific literature.^3,4 The UNESCO ICT Competency Framework,⁵ the mentioned DigCompEdu or the Common Framework of Digital Competence for Teachers in Spain⁶ are some of these frameworks.

All of these frameworks, and others, evaluate an instructor’s familiarity with the digital landscape across different areas such as technology confidence, experience, information literacy, or security awareness. These frameworks also indicate that teachers and other professionals as well as the wider population should acquire digital literacy. During the COVID-19 lockdown period, it was observed that many of the teaching staff of higher education institutions in several different fields lacked adequate training and digital competences to switch to distance learning. This lack of training and competences affected the design of classes, repeating the same face-to-face model and the non-generation of digital materials suitable for distance learning. Students were also unsatisfied with these teaching models.^7,8

However, the scientific literature tends to focus on digital competences among students who will become teachers or current teachers in the field of education, such as future teachers,⁹ primary school teachers¹⁰ or secondary school teachers.¹¹ However, it also concerns other teachers who will work as specialists at higher education institutions. These teachers are not in a faculty of education, so digital competences should have a broader framework across all fields of knowledge, such as foreign language teachers¹² or health sciences teachers in Andalusian universities.¹³ Nevertheless, these studies are still scarce, especially those focused on students,¹⁴ and there is a need for studies in fields other than Education.

This article aims to determine whether there are differences in the level of digital competences shown by professors of health sciences at Andalusian universities and Catalan universities. Our research questions are:

RQ1 What are the digital teaching competences of Andalusian and Catalan university instructors in health sciences?

RQ2 What are the differences between Andalusian and Catalan health science teachers?

The article is composed of the following sections: Section 2 presents a literature review based on digital teaching competences in higher education, specifically in health sciences. Section 3 then describes the methods and materials used in this article. Section 4 shows the results. Finally, section 5 shows the discussion and conclusions and recommendations for future research on the development of digital competence in teaching in health sciences.

Digital competences in professors

Professors in all fields need to acquire digital competences and transmit them to their students. There are some studies in health sciences as a discipline or specifically in nursing.

In relation to health sciences,¹⁵ performed a study of Andalusian universities (Spain) with health sciences professors (N = 300) using DigCompEdu Check-in translated into Spanish.¹⁶ The goal of their study was to ascertain the differences concerning gender, teaching experience, and the domain of information and communication technologies. They found that younger professors had stronger digital competences than older professors, and females tend to teach virtually with a higher level of digital teacher competence (DTC). Those with between 4 and 14 years of teaching experience had a higher level of DTC. In Perú,¹⁷ performed a study of health sciences teachers at a private university. They assessed the digital teacher competence (DTC) with a sample of 34 professors using the DTC version from.¹⁸ They found no differences related to the gender of professors, and the lowest assessed area was digital security. They assessed the areas of information literacy, communication in virtual environments, digital security practices, and problem solving.

Materials and methods

Objectives

This research aims to:

• 1.

  Validate the DigCompEdu model in the context of health sciences at universities (O1).

• 2.

  To understand the level of digital competence of professors of health sciences at two Spanish public universities (O2).

• 3.

  Identify whether there are significant differences between the levels of the Teaching of Digital Competence (TDC) at two Spanish public universities (O3).

Sample

561 professors from public universities in the autonomous communities of Andalusia and Catalonia participated in the research, including 148 women and 113 men from Catalan universities were included in the research, along with 158 women and 142 men from Andalusian universities, all of whom are university professors in the field of health sciences. Their age and years of experience are presented in Table 1.

Table 1.

Percentages of age and teaching experience of the participants.

Dimension	Descriptor	Andalusia	Catalonia
Age	under 25 years old	0.7%	0.0%
	25-29 years old	5.3%	0.8%
	30-39 years old	16.0%	12.3%
	40-49 years old	22.7%	33.0%
	50-59 years old	35.3%	38.3%
	More than 60 years old	20.0%	15.7%
Experience	1-3 years	13.3%	7.7%
	4-5 years	10.0%	15.3%
	6-9 years	8.0%	17.2%
	10-14 years	15.3%	19.2%
	15-19 years	10.7%	12.3%
	More than 20 years	42.7%	28.4%

In terms of the years of use of technologies among the professors from Andalusia, the majority of users have 10-14 years (22%) of experience while in Catalonia the largest group has between 1-3 years (23.8%) of experience (Table 2). The results seem to indicate that professors of Health Science in Catalonia have fewer years of experience with the use of technologies.

Table 2.

Years of use of technologies.

Dimension	Descriptor	Andalusia	Catalonia
ICT time of use	I don't use technology as an educational tool	1.3%	16.1%
	Less than 1 year	2.7%	20.7%
	1-3 years	16.7%	23.8%
	4-5 years	6.7%	19.5%
	6-9 years	17.3%	8.4%
	10-14 years	22.0%	0.0%
	15-19 years	18.0%	0.0%
	20 years or older	15.3%	11.5%

Information collection instrument

The information collection instrument was the "DigCompEdu Chek-in" questionnaire,¹⁹ an analysis instrument of the European Framework for Digital Competence in Teaching DigCompEdu validated by.²⁰ This competence framework is considered to be the most suitable for assessing the TDC of university teaching staff through expert judgement.¹³

The instrument is comprised of twenty-two items, which refer to the six areas of competence analysed by DigCompEdu and presented above: a) professional engagement (four items), b) digital resources (three items), c) digital pedagogy (four items), d) assessment and feedback (three items), e) empowering learners (three items), and f) facilitating learners' digital competence (five items).

Note that the authors of the model consider that the first area of competence refers to professional competence; the areas of digital resources, digital pedagogy, assessment and feedback and learner empowerment to pedagogical competence; and the last one, facilitating learners' digital competence, to learner competence.

Each of the items measures the different components of the competence framework: A Organisational communication; A2 Professional collaboration; A3 Reflective practice; A4 Digital continuing professional development; B1 Selecting digital resources; B2 Creating and modifying digital resources; B3 Managing, protecting and sharing digital resources; C1 Teaching; C2 Guiding; C3 Collaborative learning; C4 Self-directed learning; D1 Assessment strategies; D2 Evidence analysis; D3 Feedback and planning; E1 Accessibility and inclusion; E2 Differentiation and personalisation; E3 Active student participation; F1 Information and media literacy; F2 Digital communication and collaboration; F3 Digital content creation; F4 Well-being; and F5 Digital problem solving.

At the same time, the questionnaire asks teachers to rank their self-perceived level of digital competence: when they start the questionnaire (pre) and when they finish it (post). The ranking system is as follows: Novice (A1), Explorer (A2), Integrator (B1), Expert (B2), Leader (C1) and Pioneer (C2).

Data collection and analysis procedure

The questionnaire was carried out in digital format, through the "Eu Survey" platform. The survey was sent from January to February 2022, and the data was collected in March 2022. The anonymity of the participants was always guaranteed. All the professors of each university were invited to participate by sending an institutional email.

The reliability, discriminant validity and convergent validity of the questionnaire were calculated using Cronbach's Alpha, McDonald's Omega, Composite Reliability (CR), Average Variance Extracted (AVE) and Maximum Shared Variance (MCV) coefficients. In addition, and in order to compare these results, the method of inferential analysis between items and dimensions is used. For this purpose, the bivariate correlational analysis technique uses Spearman's ρ correlation coefficient. The construct validity of the test was obtained by means of an exploratory factor analysis (EFA). The principal components method was used for factor selection. The factors obtained are orthogonally rotated using the Varimax method with Kaiser normalisation. Once the number of factors has been determined, a confirmatory factor analysis (CFA) is performed to check whether the theoretical measures of the model are consistent by modelling diagrams and using structural equations.²¹

In other words, we test whether the data fit the hypothesised measurement model yielded by the exploratory factor analysis. The method used to test the theoretical model was weighted least squares (WLS), which provides consistent estimates in samples that do not conform to normality criteria.²¹ For the latter procedure, AMOS software was used, which is capable of revealing hypothetical complex relationships between variables through structural equation modelling (SEM). In parallel, the non-normality of the data distribution was checked by a descriptive study considering skewness and kurtosis. The Kolmogorov-Smirnov goodness-of-fit test confirmed this conclusion, with a significance (p-value) equal to .000 for all elements (non-normal distribution).

Results

Reliability and validity

The reliability of the questionnaire has been calculated using Cronbach's Alpha and McDonald's Omega coefficients. The results show excellent reliability levels in both Alpha (.972) and Omega (.960).

An exploratory factor analysis (Table 3) was used to obtain the construct validity of the test. Previously, the applicability of factor analysis has been confirmed through the KMO test, with a statistically significant coefficient of .989 and Bartlett's sphericity test, with significance (p-value) equal to .000 (factor analysis can be applied). The results, which explain 74.86% of the variance, determine the 6 theoretical factors proposed. The theoretical model that proposes exploratory factor analysis (EFA) is contrasted through confirmatory factor analysis (CFA).

Table 3.

Rotated component matrix.

Item	Professional commitment	Digital resources	Digital pedagogy	Evaluation and feedback	Empowering students	Facilitate students' digital competence
A1	0.769
A2	0.706
A3	0.785
A4	0.605
B1		0.769
B2		0.719
B3		0.678
C1			0.769
C2			0.742
C3			0.671
C4			0.758
D1				0.772
D2				0.761
D3				0.768
E1					0.711
E2					0.721
E3					0.740
F1						0.749
F2						0.748
F3						0.828
F4						0.710
F5						0.729

The factorial loads of the dimensions are between .75 and .96. These values denote high levels of correlation. In the same way, most levels of relationship between the different factors are also high, with an average of .62. The following indicators confirm the theoretical model proposed in accordance to²²: Chi-Square (CMIN = 398.128), goodness of fit index (GFI = .959), parsimonic goodness of fit index (PGFI = .785), normalized adjustment index (NFI = .960) and parsimonic normalized adjustment index (PNFI = .759).

Overall, the Composite Reliability (CR), Average Variance Extracted (AVE) and Maximum Shared Variance (MSV) coefficients are calculated. Table 4 shows the results, as well as the reference values, taken for model fitting.²³

Table 4.

Convergent and discriminant validity of the model.

Dimension	CR	AJUSTMENT	AVE	AJUSTMENT	MSV	AJUSTMENT
A	.739	CR > .7	.681	AVE > .5	.551	MSV < AVE
B	.758		.685		.554
C	.812		.652		.557
D	.810		.675		.440
E	.798		.642		.538
F	.828		.647		.405

All the figures obtained fit with the reference values. Therefore, the reliability of the model (CR), as well as its convergent (AVE) and discriminant (MSV) validity, are demonstrated.

Description of the digital teaching competences

For a correct interpretation of the results, it should be considered that the response interval ranges from 0 to 4. Regarding the mean scores, Table A1 presents the mean values and standard deviations reached by the teachers in the different questions, by dimensions and globally.

In terms of digital competence in the Autonomous Community of Andalusia, the values range from basic (1.18) to intermediate (2.33). In particular, teachers show problems (basic level) when teaching pupils to behave safely and responsibly online; using digital technologies to provide pupils with personalised learning opportunities; and analysing all available data to identify pupils in need of additional support. The competences that stand out (intermediate level) are using different digital channels systematically to improve communication; creating their own digital resources and modifying existing ones to suit their needs as a teacher; and using digital technologies to acquire and document knowledge when learners work in teams. Again, in the case of the universities of the Autonomous Community of Catalonia, the values are between the basic level (1.07), somewhat lower than in the Autonomous Community of Andalusia, and the intermediate level (2.41), somewhat higher than the maximum level in Andalusia.

In particular, the least developed competences in the autonomous community of Catalonia are using digital technologies to provide learners with personalised learning opportunities; teaching learners to behave safely and responsibly online and analysing all available data to identify learners in need of additional support. On the other hand, the competences that stand out are systematically using different digital channels to improve communication with learners and peers; participating in online training courses and creating their own digital resources and modifying existing ones to adapt them to the needs of teaching. It is very significant that in both Communities the competences that stand out are in the first two areas (technical mastery), while the least developed competences correspond to areas in which evaluation of the teaching-learning process (area 4), attention to diversity (area 5) and the development of digital citizenship (area 6) predominate.

Specifically, the mean value achieved in the overall instrument for the Autonomous Community of Andalusia is 1.95 points with a standard deviation of 0.69. This indicates that, in general, the level of competence is basic-intermediate. In the case of Catalonia, the mean value achieved for the instrument as a whole is 1.87 points with a standard deviation of 0.68, a result very similar to that of Andalusia. Therefore, in general, teachers have basic levels of TDC although they consider themselves more competent in professional engagement and digital resources in both Andalusia and Catalonia. This latter assumption is very significant.

Finally, we checked whether there are statistically significant differences between the scores of the two regions. For this purpose, we applied the Mann-Whitney U test. The results can be seen in Table 5.

Table 5.

U of Mann-Whitney for the scores of the two Autonomous Communities.

	Mann-Whitney U	Wilcoxon W	Z	Sig. asymptotic(bilateral)
A	37416.000	82566.000	-0.910	0.363
B	36840.000	71031.000	-1.217	0.224
C	36636.000	70827.000	-1.317	0.188
D	36703.000	70894.000	-1.293	0.196
E	33427.000	67618.000	-3.006	0.003
F	38073.000	72264.000	-0.564	0.573
TOTAL	36508.000	70699.000	-1.380	0.168

The results confirm that there are no statistically significant differences in the overall level of Digital Competence in Teaching between the two Autonomous Communities. The same phenomenon occurs in almost all dimensions. The only dimension in which statistically significant differences are found at 95% (p < .005) is "Empowering learners". In this case, the analysis of the mean rank reveals differences in favour of the Community of Andalusia. However, after calculating Cohen's D, these differences are considered very small (d = 0.1). Therefore, for a better understanding of the phenomenon, reference is again made to Table 5 (mean and standard deviation of items, dimensions and digital competence) for the specifics of each Autonomous Community.

Discussion

In our study, we have observed different phenomena. First, we have observed in the results section that the level of digital teaching competence in both autonomous communities in health sciences is basic-intermediate and that there are no significant differences among the autonomous communities.

Second, in regard to the DigCompEdu model, our study reports data validity with the structured equation modelling in Health Sciences. Thus, this model was also validated for higher education by a related study.¹⁹ Similarly, other studies performed other validations such as¹³ where DigCompEdu was tested by the peer-groups method and the discrimination capacity was validated. Another model was also validated in the case of Andalusian universities, but only reliability was validated.²⁴

Our study has served to provide an insight into the teaching profile of Health Sciences in Higher Education in the Autonomous Communities of Andalusia and Catalonia, with a comparison between the scores obtained in the DigCompEdu Check-In questionnaire.¹³

In both communities, teachers report a high mastery of technical competences, such as communication with other teachers or participation in online courses. The results coincide with those of,^25,26 who place the technical level of medical teachers above the pedagogical level.

It shows that the participating teaching staff obtain scores that place them at basic-intermediate levels of Digital Teaching Competence, as does the research by^27,28 carried out for all university teaching staff. Similar results are obtained in a more recent study by¹³ within Andalusian universities. This study recommended carrying out studies focused on specific branches of knowledge in order to adapt university teacher training plans, as is done in the present research with the Health Sciences branch.

With regard to possible limitations of the research, we could reflect on the different areas for improvement and how comparative studies between future works with similar characteristics should be approached. One of the main weaknesses lies in the types of samples used, since the intentional sampling method indicates that it is not random, therefore, the results found should be considered non-extrapolatable to other areas of knowledge. Incorporating probabilistic samples for each area to be studied in future studies will allow the collection of more representative data that can be extrapolated globally.

Funding

Design, production and evaluation of t-MOOC for the acquisition of digital skills of university teachers. Reference: US-1260616. Junta de Andalucía (Consejería de Economía y Conocimiento)