Evaluation study of effect of virtual care education on healthcare providers’ knowledge, confidence, and satisfaction

Introduction

Virtual care refers to the delivery of healthcare services using telecommunication technologies, facilitating healthcare from a distance (Curran, Hollett & Peddle, 2023a). Offering virtual care can make healthcare access more efficient for patients by reducing wait times (Appireddy et al., 2019; Rabinovitch et al., 2022; Young, Gupta & Palacios, 2019), reducing appointment duration without compromising patient satisfaction (Appireddy et al., 2019), and allowing patients to spend less time and money on travel (Welk, McArthur & Zorzi, 2022). The benefits of virtual care were further highlighted by the COVID-19 pandemic when in-person healthcare provision was restricted (Curran, Hollett & Peddle, 2023b; Heyworth et al., 2020).

However, it is also crucial to consider the healthcare provider’s perspective on offering virtual care services. Research has shown that healthcare providers prefer virtual care in certain contexts. For example, a survey found that physicians favored telemedicine for follow-up visits, medication refills, urgent care, behavioral health, dermatology visits, and chronic care management (Nies et al., 2021). Similar trends have been reported in the United Kingdom and Australia, where clinicians cite convenience and improved continuity of care as key benefits of virtual delivery models (Greenhalgh et al., 2020; Snoswell et al., 2020). Healthcare providers generally have a positive impression of virtual care (Kim et al., 2022), appreciating its convenience and efficiency (Li et al., 2022).

Despite the increasing relevance of virtual healthcare delivery, many healthcare providers receive little to no formal training in providing these services. This lack of training could explain the lower confidence that results in providers hesitating to provide virtual care services (Curran, Hollett & Peddle, 2023c), as they struggle to adapt their in-person clinical skills to meet the needs of a virtual environment (Teichert, 2016). This paucity in training begins early in medical education with medical learners having historically received insufficient training in virtual care techniques (Stovel et al., 2023; Strong, Dossett & Sandhu, 2020), contributing to providers’ downstream lack of confidence. Formal education that puts the theory of virtual care into action is crucial for healthcare providers to feel competent in offering this important service.

Virtual care delivery has become a widely adopted model of healthcare delivery in Canada and internationally, particularly following the COVID-19 pandemic (Canada Health Infoway, 2020; Webster, 2020), during which 71% of Canadian seniors reported having a virtual appointment, the highest proportion among surveyed countries (mean 39%; CIHI, 2022). Additionally, post-pandemic, 98% of primary care providers offer virtual care, most of which involves telephone appointments (CIHI, 2022), demonstrating a clear place for these services in Canadian healthcare delivery. As its use expands across diverse clinical contexts, it is essential to equip healthcare providers with appropriate training to ensure the delivery of safe, effective, and high-quality care. However, few studies have evaluated continuing professional development (CPD) training programs on virtual care for healthcare providers, and existing interventions vary widely in format, content, and assessment methods (Felker et al., 2022; Khan et al., 2021). Few programs integrate both objective knowledge testing and self-perceived confidence measures, and even fewer explicitly ground their design in established learning theory (Edirippulige & Armfield, 2016). Additionally, most published evaluations report only post-intervention satisfaction without a pre-post comparison, which limits inferences about actual learning gains (Waseh & Dicker, 2019).

The CPD module evaluated in the present study was designed to address these limitations by combining (a) theory-informed instructional design grounded in constructivism, (b) application-based case scenarios, and (c) structured pre-/post-assessment of both knowledge and confidence. Given these gaps, this study evaluates the effectiveness of a newly developed virtual care CPD module by examining learner satisfaction and changes in knowledge and confidence, providing one of the few theory-informed, pre–post analyses of virtual care education in a healthcare provider population.

Materials and Methods

Learning module

The Office of Professional & Educational Development (OPED), a part of Memorial University of Newfoundland’s Faculty of Medicine, hosts a number of online, asynchronous CPD learning modules. In February 2024, the OPED launched their first module on the topic of virtual care, titled Effective Virtual Care: Elevating Primary Care through Virtual Practice. The module’s design was informed by the constructivist learning theory which states that learners actively construct their understanding and knowledge through experience and reflection (von Glasersfeld, 1991). As a result, in designing the module, the goal was to help learners construct what they learn and understand through active participation (Rufii, 2015). To achieve this, learners were first given information on several topics, including an introduction to virtual care and related considerations of privacy/confidentiality, regulatory policies, and technical logistics; the integration of virtual care into an existing practice, including choosing the appropriate patients for this medium; and conducting a virtual appointment itself, such as adapting one’s clinical skills to this delivery method. The incorporation of a case study in the module helped to conceptualize the concepts in a meaningful healthcare context which aligns with constructivism (von Glasersfeld, 1991). To promote active engagement, learners completed pre- and post-test objective knowledge surveys. Throughout the module, there was a focus on providing external resources to enable learners to access further information on any particular topic. An example of the type of information provided, along with its format, is shown in Fig. 1. Additionally, a short case study was incorporated into the module.

Theoretical support

The evaluation is guided by the Kirkpatrick model, a widely used framework for assessing training programs (Alliger & Janak, 1989). Adapted for medical education, the modified model includes four levels: Learner satisfaction (Level 1), change in opinion (Level 2A), change in knowledge and skills (Level 2B), change in behavior (Level 3), and change in healthcare outcomes (Level 4) (Fig. 2). Based on the available evaluation data, this study focuses on levels 1, 2A, and 2B to assess early outcomes of the CPD module.

Results

Confidence

For the measures of confidence, the mean pre-module score (n = 21) was 16.48 out of a possible maximum of 24. The mean post-module score (n = 20) increased to 21.30. The distribution of scores at each time point can be viewed in Figs. 3 and 4, respectively.

A Wilcoxon signed-rank test demonstrated a statistically significant improvement for each of the following matched pre- and post-module questions: Question 7 (p < 0.001), 8 (p < 0.001), 9 (p = 0.003), 10 (p = 0.002), 11 (p = 0.002), and 12 (p < 0.001).

Knowledge

Within-module multiple choice questions

The first measure of knowledge was via a multiple-choice questionnaire at the beginning and end of the training module. The mean pretest score (n = 20) was 5.18 out of a possible maximum of 6. The mean post-test score (n = 22) increased to 5.73. The distribution of scores at each time point can be viewed in Figs. 5 and 6, respectively.

A Wilcoxon signed-rank test was performed, which demonstrated statistical significance (p = 0.046).

Within-module short answer questions

The within-module short answer questions showed a range of response accuracy (Table 5). Scores ranged from 0 to 5, with two questions having a mean score below 3: Question 2 (2.44) and Question 3 (3.94). The remaining questions had higher scores, with the highest being question 4 (4.56). Inter-rater reliability was assessed as the proportion of answers for which the coders’ scores were within one point of each other. Of the 94 answers evaluated, 65 met this criterion, yielding a percent agreement of 69.1%.

Table 5:

Performance on within-module short-answer objective knowledge questions.

Participant scores for eight short-answer questions embedded within the module, reporting mean scores, score distributions, and the total possible score based on completeness, specificity, and relevance.

Question	n	Mean	Scores range
			<1	1–1.5	2–2.5	3–3.5	4–4.5	5
Q1	17	3.53	0 (0%)	1 (5.9%)	2 (11.8%)	6 (35.3%)	6 (35.3%)	2 (11.8%)
Q2	9	2.44	0 (0%)	3 (33.3%)	4 (44.4%)	0 (0%)	2 (22.2%)	0 (0%)
Q3	9	2.94	0 (0%)	0 (0%)	3 (33.3%)	5 (55.6%)	1 (11.1%)	0 (0%)
Q4	9	4.56	0 (0%)	0 (0%)	0 (0%)	0 (0%)	5 (55.6%)	4 (44.4%)
Q5	10	3.35	0 (0%)	1 (10.0%)	2 (20.0%)	3 (30.0%)	3 (30.0%)	1 (10.0%)
Q6	11	3.41	0 (0%)	0 (0%)	1 (9.1%)	8 (72.7%)	2 (18.2%)	0 (0%)
Q7	10	3.15	0 (0%)	0 (0%)	5 (50.0%)	1 (10.0%)	4 (40.0%)	0 (0%)
Q8	10	4.20	0 (0%)	0 (0%)	2 (20.0%)	1 (10.0%)	0 (0%)	7 (70.0%)

DOI: 10.7717/peerj.20414/table-5

Note:

Short answer questions were evaluated based on three criteria: Completeness (two points); specificity (two points), and relevance (one point), for a total possible score of 5.

Discussion

This study evaluated a newly developed CPD module on virtual care and found statistically significant improvements in learner confidence and objective knowledge, as well as high overall satisfaction. These findings suggest that such modules can be effective in addressing educational gaps in virtual care.

One key finding of this study was the high level of satisfaction. The majority of participants rated the course content, navigation, and interactive components favorably though some aspects of the interactive elements could be improved. Specifically, participants requested more interactive features, such as case-based learning and educational videos. Prior research has shown that the implementation of these elements can enhance student engagement and learning outcomes (Wang et al., 2025). Nunohara et al. (2020) found that videos and cases exert different influences on learners’ clinical decision-making, with video-case learning prompting greater attention to psychosocial aspects of care, whereas paper-based cases encouraged a stronger focus on biomedical elements. Given that virtual care demands a delicate balance between interpersonal abilities and medical knowledge, incorporating additional case-based activities and educational videos could improve learners’ capacity to integrate both domains, fostering a holistic and patient-centred virtual practice.

The preferences for increased interactivity reflect core principles of constructivist learning theory, which emphasizes that adult learners construct knowledge most effectively when actively engaged with realistic, problem-based scenarios (Vygotsky, 1978; Tam, 2000). This module’s design features structured content, learner autonomy, and embedded assessments, which align with best practices in instrumental design, particularly the use of multimedia learning principles and self-paced, modular structures to enhance cognitive processing and retention (Mayer, 2009). Expanding the module to include simulation- or video-based learning could further reinforce application of skills, consistent with emerging evidence in medication education (Wang et al., 2025). A recent randomized controlled trial demonstrated that interactive video-based case-based learning resulted in significant improvements in critical and systematic thinking abilities, as well as knowledge and satisfaction (Wang et al., 2025). This is particularly important in the context of virtual care training which must go beyond factual content to cultivate relational skills, empathy, and flexibility in practice (Nowell et al., 2025). Core framework principles—connection and interaction, compassion, empathy, care, vulnerability, client-centred focus, inclusivity, accessibility, and flexibility—map directly onto the gaps learners identified in our module (need for more case-based and interactive content) (Nowell et al., 2025). Incorporating interactive video cases and virtual simulations would therefore operationalize these principles by providing learners with safe, scaffolded practice in building rapport and interpreting nonverbal cues via video or phone.

Participants also demonstrated significant improvement in their post-test self-assessment scores, indicating increased confidence in applying their knowledge. Objective knowledge was also statistically significantly increased, measured via the post-test MCQ performance. These findings align with prior research, which shows that structured, interactive, and clinically relevant e-learning is more likely to improve learner satisfaction and knowledge retention (Cook et al., 2010; George et al., 2014). The intersections among knowledge improvements, confidence, and satisfaction increases demonstrate that module design features, such as interactivity and case-based learning, play a critical role in learner outcomes (Lawn, Zhi & Morello, 2017).

The rapid shift towards virtual care following the COVID-19 pandemic (Müller et al., 2021) left many providers unprepared to deliver quality virtual care (Fong, Baumbusch & Khan, 2024). As a result, the need for training modules like the one in our study is paramount. Curran et al. (2024) demonstrate that the delivery of CPD via e-learning can provide timely access to continuing professional education for health care providers who may be geographically dispersed. By demonstrating effectiveness across a heterogeneous group of learners, our study suggests that asynchronous CPD modules could be a scalable approach to virtual care training, particularly valuable for rural and remote providers who face barriers to attending in-person workshops (Calleja et al., 2022).

This study has a few key limitations. The small sample size evidently limits the generalizability of these findings; however, the population that was included is heterogeneous in several ways. Therefore, despite the small sample size, this study captured a variable group of participants and represented several perspectives. There are also limits to self-report measures; self-reported confidence gains may not fully reflect improvements in competency, a concern echoed in faculty development literature (Steinert et al., 2006). Additionally, the survey instruments were not previously validated, which may affect the generalizability of findings. However, the alignment with existing frameworks and expert review mitigates some concerns regarding reliability and validity. Finally, this evaluation study did not assess the higher modified Kirkpatrick model levels, behavior change (Level 3) or healthcare system outcomes (Level 4). Long-term follow-up could assess sustained practice change, which has been identified as a gap in CPD evaluation (Steinert et al., 2006). Therefore, the impact of this module on these more consequential outcomes is unknown at this time.

The findings of this study present several opportunities for fine-tuning future virtual care training opportunities. Firstly, interactive elements should be enhanced in order to address the neutral feedback from the survey. Additional case-based learning should also be incorporated to align with the qualitative feedback, an approach shown to improve application of knowledge (i.e., rather than just fact retention) in clinical settings (McLean, 2016; Wittich et al., 2010). Finally, assessment strategies should be expanded to include more objective knowledge measures, beyond MCQ and short answer questions, to better capture practical application of knowledge (Epstein, 2007).

Conclusions

This study is one of the first to evaluate a virtual care CPD module aligned with the Modified Kirkpatrick model. The findings demonstrated statistically significant improvements in objective knowledge and self-reported confidence, as well as overall positive satisfaction with content, navigation and interactive components. Participants valued structured content but expressed interest in more interactive and case-based learning. These insights highlight the potential of asynchronous e-learning for skill development in virtual care, while also identifying opportunities for improvement. Future versions of the module should prioritize the inclusion of case-based scenarios to improve practical relevance. Virtual care will continue to be a crucial part of healthcare provision, however, adequate training must be incorporated so this delivery modality is advantageous for both healthcare providers and patients.

Supplemental Information