Digital competence of faculty members in health sciences measured via self-reflection: current status and contextual aspects

Introduction

As in all fields of education, the use of digital technologies (DT) might facilitate the creation of learning environments that are as effective as face-to-face instruction in the health sciences (Car et al., 2019; Kyaw et al., 2019). The efficacy of online or e-learning has been demonstrated in the literature when the digital tools are properly designed and administrators and students can meet adequate digital competency (DC) levels (Noesgaard & Ørngreen, 2015; Thalheimer, 2017). As the use and impact of digital innovations and interventions in health education grow, it is critical to improve the DC of faculty members in these fields (Basilotta et al., 2022; Cabero-Almenara et al., 2021a; Choules, 2007). Enhancing the DC of faculty members should be a key focus for institutions, and may help meet the rising demand for higher education as well as the changing needs and expectations of today’s students (Spante et al., 2018).

In their systematic review, Batanero et al. (2020) found that the majority of studies demonstrated a lack of information and communication technologies (ICT) skills and regarding training for academicians. Similarly, Akram et al. (2021) used a technological pedagogical content knowledge (TPACK) model to assess the technology, pedagogy, and content knowledge of faculty and found that while other two sub-domain knowledge levels were adequate, the technological domain was low.

From (2017) called the digital skills of faculty members a “pedagogical digital competence” (PDC). These skills are defined as “teachers’ ability to use ICT in their teaching practice”. Guillén-Gamez et al. (2021) added that PDC includes skills as well as the attitude to improve in the personal and academic use of ICT in learning.

The broader definitions of DC were constructed in more comprehensive frameworks because the competency of a faculty member may be measured in multiple but interconnected sub-skills in task-related sub-domains of their profession (International Society for Technology in Education Standards (ISTE), 2021; UNESCO, 2011). Consequently, assessing DC for an academician is a challenging process. For this purpose, the DigComEdu framework (DigCompEdu, 2021) was constituted by the European Research Center in 2017 with both a definition and a self-reported assessment tool for DC (Redecker, 2017). According to the DigCompEdu framework, DC is:

“… an ability to use digital technologies not only to enhance teaching, but also for their [educators’] professional interactions with colleagues, learners, parents and other interested parties, for their individual professional development and for the collective good and continuous innovation in the organization and the teaching profession” (Redecker, 2017, p. 19).

In the DigCompEdu framework, the DC of educators was defined in six areas: professional engagement, digital resources, teaching and learning, assessment, empowering learners, and facilitating learners’ DC (Redecker, 2017). The framework also provided a DC assessment tool in a self-reflection questionnaire to portray the DC of faculty members in those six areas. More details about that tool are given in the Data Collection section.

It is important to determine the DC level of faculty members, yet this alone is not robust enough for the successful design and implementation of proper DC development. Understanding the related factors influencing DC levels can help in providing effective and tailored support mechanisms. In a number of studies, variables such as gender, age, generation, academic level (Guillén-Gamez et al., 2021; Jansone-Ratinika et al., 2021; Jorge-Vazquez et al., 2021), teaching experience (Batanero et al., 2021; Cabero-Almenara et al., 2021b), digital technology experience in teaching (Ghomi & Redecker, 2019; Hatlevik, 2017; Lucas, Dorotea & Piedade, 2021; Tondeur et al., 2018), and availability and accessibility in technological infrastructure with support (Cattaneo, Antonietti & Rauseo, 2022; Mohan et al., 2020) are found to be related in DC.

Moreover, the experiences during compulsory distance education during the COVID-19 pandemic could be effectual in evaluating the DC of faculty members. During that time teachers were forced to utilize many new technological tools and pedagogical strategies imposed by not only their institutions but also the facilities that were available at their workplace and home (Gonzalez, Ponce & Fernandez, 2023). They may have had to gain new DCs about using ICT to survive in teaching.

To improve the digital competence of faculty members in health sciences in higher education, an Erasmus+ Project, called DITEPRACT (Digital and Hybrid Teaching and Learning of Practical Skills in Higher Education), was initiated by six partner universities from Finland, Latvia, Lithuania, Portugal, Sweden, and Türkiye in 2021 (Jansone-Ratinika, Surakka & Wikstrom-Grötell, 2023). The objective of the project was to discover the best DT experiences or tools utilized, especially in subject-specific practical teaching in the disciplines. DCs are gained from DT, therefore, it was important to make a valid assessment of DC levels of faculty members with related contextual aspects.

Accordingly, this study aims to assess the DC of faculty members teaching practical skills in the health sciences and to understand the contextual aspects of their competencies. The two research questions were as followings:

1. What is the DC level of the faculty members?

2. Do the demographic characteristics and teaching experience of the faculty members affect their DC level?

Materials and Methods

The primary aim of this descriptive study is “to provide accurate description of the status or characteristics of a situation or phenomenon” (Johnson & Christensen, 2004, p. 347). A two-part survey was administered to faculty members from six universities in the DITEPRACT project to answer our research questions.

Participants

The participants were faculty members from six partner universities who taught practical skills during the 2020–2021 academic year. A total of 306 faculty members from diverse departments in health sciences participated in the study after being selected using a convenience sampling strategy. The average age was 46.7, and the average teaching experience was 14.6 years (Fig. 1).

Data collection instruments and process

For the first research question, a “DigCompEdu CheckIn Self-Reflection Tool” questionnaire (DigCompEdu, 2021) was used to assess the DC of faculty members. The questionnaire was first published in English in 2017 by the “European Commission’s Joint Research Center” as “European Framework for the Digital Competence of Educators: DigCompEdu” (DigCompEdu, 2021). The significance of the DigCompEdu is that “the focus is not on technical skills, rather, the framework aims to detail how digital technologies can be used to enhance and innovate education and training.” (DigCompEdu, 2021., p.1). The questionnaire includes 22 competency statements in six domains: “professional engagement, digital resources, teaching and learning, assessment, empowering learners, and facilitating learners’ DC” (DigCompEdu, 2021). There are five options in each question scored from 0 to 4 points. The higher score in a question means higher competence described in the question. The maximum score is 88 when a participant marks highest-competency statement in each question. The total score of a participant indicates his or her DC level according to predefined six competence levels, as explained in Fig. 2.

At the beginning of the questionnaire, the participants were given a brief description of competency levels from A1 to C2 and then asked to rate their DC level from this range (pre-self-assessed DC score). Next, they responded to all items and were asked again to rate their DC from the same range (post-self-assessment score). Once the participants completed the evaluation tool, the researchers calculated the DC points of faculty members based on their responses (scale-based DC score). According to the DigCompEdu Framework (DigCompEdu, 2021), each participant was assigned to one of the following competency categories: newcomer (A1), explorer (A2), integrator (B1), expert (B2), leader (C1), or pioneer (C2).

At the beginning of the study, written permission was obtained via email from the DigCompEdu research team. The original questionnaire was in English, with additional versions available in some European languages (DigCompEdu, 2021). Two partners used the English version and others preferred a version in their local language. The Cronbach’s α coefficient of questionnaire was calculated as 0.915 for the study.

For the second research question, a follow-up was conducted to inquire about the demographic and descriptive characteristics of the participants, including academic title, age, duration of teaching experience, time spent using DT, and digital tools used for learning. Likert questions about personal use patterns of DT and about the institutional facilities were also used in this section as described in “DigComEdu CheckIn Self Reflection Tool” (DigCompEdu, 2021). Data were collected from participants online using Google® Forms. Participation was voluntary.

Data analysis

After data collection, the data from six universities were accumulated into a single dataset. For categorical variables, like demographics and having available DT at work or at home, frequency distributions were used; for numerical variables like DC scores, descriptive statistics (means and standard deviations) were calculated. Multiple linear regression analysis was used to determine the influencing factors such as age, teaching experience, perception of the work environment, and having experience teaching a fully- or partially- (in blended or hybrid form) online course.

A paired t-test was used to determine the difference between perceived (pre- and post-) and real (scale-based) levels of DC according to the self-assessment questionnaire. Statistical analyses were carried out using the SPSS (v. 26) software at a significance level of 5% (α = 0.05).

To answer the second research question, the following nine hypotheses (Fig. 3), were tested:

To test the first six hypotheses (H₁, H₂, H₃, H₄, H₅, H₆), an explanatory multiple linear regression model was proposed, as illustrated in Eq. 1, below.

(1) $$DC={\beta }_0+{\beta }_{1}Age+{\beta }_{2}TE+{\beta }_{3}DTE+{\beta }_{4}PWE+{\beta }_{5}FTEBC+{\beta }_{6}PTEBC+{\epsilon }_{\dot{I}}$$

DC_i:

Digital competency score of faculty members

Age_i:

Age of faculty members in years

TE_i:

Teaching experience in higher education of faculty members in years

DTE_i:

Using DT in teaching of faculty members in years

PWE_i:

The mean of perception of the work environment of faculty members

FTEBC_i:

Experience in teaching (in years) a fully online course (in blended or hybrid form) before COVID-19

PTEBC_i:

Experience in teaching (in years) a partially online course (in blended or hybrid form) before COVID-19

ε_i:

The error (residual) term in the regression model

To test the remaining hypotheses (H₇, H₈, H₉), paired sample t-tests were employed.

Results

Table 1 shows the descriptive characteristics of the participants. The average age was 46.7 ± 11.7 years, with 72.5% of participants falling between the ages of 25 and 55. The average teaching experience of participants in higher education was 14.6 ± 10.3 years. The average time spent using DT in teaching was 10.7 ± 7.6 years. For 85% of the participants, the main profile of the students was undergraduate.

Table 2 summarizes the availability of regular or sophisticated DT for teaching for the participants. The majority of participants stated that they used presentations (98%) and online communication tools (96.4%) for teaching. The least used digital tools were wikis and blogs (9.5%). In terms of the university’s provision of DT for teaching/learning, 96.1% of the participants had synchronous communication tools and 90.2% had asynchronous communication tools. More than half of them (55.9%) stated that they had sophisticated or field-specific software. The least owned advanced tools are augmented/virtual reality tools (24.8%).

Table 3 summarizes the participants’ private use of DT as well as the criteria met by their work environment. A total of 86.6% of participants stated that they used the Internet extensively and competently, and 88.3% found working with computers and other technical equipment to be simple. In the fourth statement, 62.4% of participants said they subscribed to various social networks. Considering the participants’ assessments of the work environment, 82.3% believed that their department invests in updating and improving the technical infrastructure. Moreover, 73.9% of the participants found the department’s internet connection to be reliable and fast in all criteria evaluations.

Figure 4 summarizes the DC of faculty members based on scale-based scores (SBC). In total, the integrator level (B1, 44.1%) had the largest proportion following the expert level (B2, 29.7%). The lowest (A1) and highest proficiency (C2) categories had the smallest number of participants, at 1% and 0.7%, respectively.

In Table 4, it is shown that the regression model was overall significant at α = 0.05 level and the coefficients of age, DTE, PWE, FTEBC, and PTEBC were significant (95% confidence interval), except for TE. The signs of the regression models’ coefficients should be checked to thoroughly examine the directions of the impacts of independent variables on DC. The signs of all significant coefficients were consistent with the research hypotheses, according to the results in Table 4. The R² of the multiple linear regression model was 0.164% and, 16.40% of the variance for the endogenous variable (DC) was explained by exogenous variables (age, DTE, PWE, FTEBC, and PTEBC). In addition, the model guaranteed basic regression assumptions (normality, multicollinearity, heteroscedasticity etc.), and the results promoted the statistical validity of the analysis.

To determine any statistically significant differences between pre- and post-assessment means, a paired sample t-test was employed, as shown in Table 5. Results showed that there were statistically significant differences between pre- and post-self-assessment (p < 0.05) for the first pair, pre- and scale-based assessment (p < 0.05) for the second pair, and post and scale-based assessment (p < 0.05) for the third pair.

Discussion

According to the scale-based DC assessment results (Fig. 4), the DC of most participants was intermediate (B1 (44.1%) and B2 (29.7%)). Redecker (2017) defines integrators (B1) as educators who are aware of the potential use of DT but are still experimenting with them in various environments for their own needs. They need consistent practice in their authentic teaching and learning environments to make accurate decisions about DT. For experts (B2) (Redecker, 2017) claims that they have their own DT repertoire ready to be utilized in different situations based on their own experiences and knowledge of DT.

The accumulation of the majority of the DC scores (%73.8) at intermediary levels (B1 or B2) was similar to the other resent observations regarding higher education (Guillén-Gamez & Mayorga-Fernandez, 2020; Jorge-Vazquez et al., 2021). Considering the high rate of DT usage in their private lives and the abundance of DT facilities in the workplace (Table 3), result of integrator or expert is not a surprise.

Participants responded that the most frequently DT they used were presentations, video and audio, and online/virtual learning environments (Table 2), commonly for information presentation or content-sharing purposes, where the teacher’s role was similar to that in traditional face-to-face instruction. Conversely, other tools in Table 2, such as creating video and audio, posters, mind maps, planning tools, online quizzes or polls, and blogs or wikis received relatively low usage. Those tools are commonly used for increasing interaction, communication, and active student participation (Beldarrain, 2006). A new faculty development program introducing and highlighting the pedagogical use and benefits of those tools might be beneficial.

Moreover, it was found that the use of sophisticated DTs, like augmented/virtual reality and simulations, came in last and was used at a low level. Although integrating these tools into education requires complex peripherals, a certain level of digital literacy and tailored scenarios, given their valuable pedagogical benefits in higher education compared to other tools, more institutional support is recommended to provide them for faculty members since these technologies are not affordable or easily accessible individually to possess or develop most of the time.

Only 14.4% of participants’ DC level achieved a high proficiency status (leader 13.7% and pioneer 0.7%) (Fig. 4). These participants seemed to be competent to lead others and create innovative approaches with DT. Studies with similar results have highlighted that an increase in the DC of faculty members requires not only accessing DT in work environment and daily life, but also in customized, continuous, and convenient personal development opportunities (Zakharov et al., 2022; Lucas et al., 2021).

A small percentage of participants were in lower (newcomer 1.0% and explorer 10.8%) levels (Fig. 4). These participants may have a low usage of DT in their private life and/or were employed in an inadequate work environment. Further research and comprehensive need analysis for those staff could provide better and tailored faculty development strategies.

Based on the result of the multiple linear regression model (Table 4), all hypotheses, except the second one, were accepted. In the first accepted hypothesis, it was found that age had a negative effect on DC which is in line with many studies (Cruz & Díaz, 2016; Gallardo-Echenique, Poma & Esteve, 2018; Gonzalez, Amaro & Martinez, 2019). Given the high private use ratio of DT (Table 3), younger participants may have higher self-confidence. Lucas et al. (2021) claimed that undergraduate or pre-service programs of younger faculty members have more DT courses and a higher likelihood of using DT in training.

By rejecting the second hypothesis of “teaching experience in years” (TE) and accepting the third hypothesis, “using DT in teaching” (DTE), it is possible to claim, like Guillén-Gamez et al. (2021), that DC is more related to using DT for teaching purposes. While some studies claim that lack of TE would be a barrier to proper technology integration (Batanero et al., 2021), others argue that TE may be effective in sub dimensions of DC such as “communication and collaboration, digital content creation, security and problem solving” (Zhao et al., 2021, p.11).

The “perception of how the work environment meets the requirements for using DT” (PWE) seems to affect DC positively, as Cattaneo, Antonietti & Rauseo (2022) have found. In the study, there appeared to be a wide range of online communication, content management and presentation tools, but a limited number of field-specific tools were provided by the participants’ institutions. This would also help to understand faculty members’ intermediate DC levels. It is difficult to use and gain field-specific DT experience for faculty members if they do not have institutional support. In this respect, institutions should be responsible for providing a proper need-analysis, teacher training and continuous support mechanisms.

Furthermore, even though this study tried to obtain experiences of faculty members in online teaching at pre COVID-19 times, it might not be possible to eliminate the effects of compulsory distance education period (Shagiakhmetova et al., 2022). In those teaching efforts, faculty members may have needed to use various synchronous and asynchronous communication software, learning management systems and related ICT tools intensely.

The next two accepted hypotheses (H₅ and H₆) showed that the experience participants had in teaching in fully- or partially-online experiences (FTEBC and PTEBC, respectively) had a positive effect on DC levels, as DTE had. Based on the assumption that any competence improves with practice (Cattaneo, Antonietti & Rauseo, 2022), faculty members with experience in implementing online courses are often expected to be more competent in DT. As online teaching involves intense use of DT (Erlam et al., 2021), experience in such a teaching environment may be a sound indicator of the DC of faculty members (Thalheimer, 2017).

By accepting the last three hypothesis (H₇, H₈, and H₉ in Table 5), it is possible to claim that the participants underestimated their DC capacity. This result supported the idea that an individual’s self-perceived level of knowledge may not reflect their actual level since they were unaware of it (Alba & Hutchinson, 2000). Furthermore, Moore & Cain (2007) claimed that this fault estimation is more in difficult skill-based tasks. Although the DC levels assigned by DigCompEdu framework (DigCompEdu, 2021) were explained during the self-assessment process, it was discovered that inquiring about their actions, attitudes, and roles in using DT provided a more accurate DC assessment.

Conclusions

The study showed that the majority of the faculty members teaching in health sciences have intermediate (B1 or B2) levels of DC. According to the DigCompEdu (2021) framework, this intermediate level means they are capable of using several DT in teaching and overcoming their own problems. Others, with small percentages, are at the upper (about 14%) or lower (about 11%) ends of the DC level continuum.

In order to understand what features would have affected the measured DC level, a regression analysis provided influencing factors such as age, digital teaching experience, perception of work environment, and having teaching experience in online/partially online courses were identified as influencing factors for DC in the current study. These findings can be used to develop new strategies for developing the digital competencies of faculty members in higher education.

One recommendation for policy makers is to plan and implement tailored faculty development training programs to enhance DC. The design of training programs should be informed by an understanding of the existing levels of DC among the faculty members, with the aim of providing a diverse range of learning scenarios that cater to the varying needs of faculty members with different DC levels. Rather than adopting a one-size-fits-all approach, training programs should be tailored to the specific requirements of different groups of faculty members. Enhancing DC with supportive strategies for faculty members would increase self-confidence and their autonomy in overcoming obstacles like the outbreaks in the pandemic (Chang, Gaines & Mosley, 2022).

Institutions with a significant number of upper-level faculty members have an opportunity to arrange for and encourage the collaborative knowledge, and information and experience exchange activities among academicians with different levels of DC. Peer learning and collaborative learning would motivate faculty members to participate more readily in such personal development training activities (Langset, Jacobsen & Haugsbakken, 2018).

Another recommendation is to enrich the learning environment with proper DTs designed for daily needs, like information access and communication, as well as for field-specific and sophisticated pedagogic outcomes. As this study showed, training about DC should be supported with sustainable and continuous facilities for faculty members for successful DT integration in teaching.

Supplemental Information