Digital technology for orthognathic surgery training promotion: a randomized comparative study

The traditional orthognathic surgery training method

To achieve satisfactory therapeutic effects in both oral function and facial appearance, preoperative measurement and analysis and the application of surgical guide plate are of great significance in orthognathic surgery. In the traditional training method (Fig. 1A), residents are first taught by instructors in the form of lectures. Subsequently, they review the knowledge and finish their homework to obtain a deeper understanding of orthognathic surgery. Third, they complete surgery simulation with a plaster model and produce the surgical guide plate using self-curing resin. Lastly, they assist the instructors with an actual surgery to gain practical experience.

The digital training system for orthognathic surgery

We have systematized a digital training system to test the residents’ learning efficiency assisted by digital technology, as shown in Fig. 1B. It is a standardized process consisting of three parts: data collection and integration, surgery simulation, and guide plate preparation. Briefly, first, the facial and oral anatomy information is collected by facial scanning (3dMD Modular Camera Unit Model, MCU 1−3.3) and dental mold scanning (TRIOS intraoral scanner system, T12A). The 3D images of the oral cavity can clearly reflect the details of teeth structure and dentofacial deformities of patients. At the same time, the Digital Imaging and Communications in Medicine (DICOM) data of patients can be obtained by CT scanning (Meyer CBCT, SS-X9010DPro-3DE). After the DICOM data is reconstructed in a 3D structure, they can be combined with the 3D images of the oral cavity to get a complete dentofacial mold. Second, surgeons determine the cutting line, the amount of bone to be removed, and the operation process in the simulated surgery, and the guide plate can be designed in this process. The residents can practice the surgery on the virtual maxilla-mandibular model multiple times. Third, the guide plate is established based on the simulated surgery and produced by 3D printing (Stratasys, Objet Eden 260V) for the actual application. Residents can check the guide plate during the operation and find out the inconsistencies between the simulated surgery and actual conditions. All participants could operate and design independently after training and practice.

Research design

This study was conducted according to the guidelines set forth by the Declaration of Helsinki and approved by the Institutional Review Board of the West China Hospital of Stomatology, Sichuan University (No. WCHS-IRB-CT-2019-221). Written informed consent was obtained from all participants. The study was based on the orthognathic residents’ training course set up by the Department of Oral & Maxillofacial Surgery, West China Hospital of Stomatology, Sichuan University. A total of 24 first-year orthognathic residents participated in the study as part of their professional skill training. A randomized control trial was performed to obtain an average grouping. The random setting is as follows: a total of 34 first-year orthognathic residents were divided into two groups (male/female) firstly according to gender. Then 12 residents were randomly selected in each of two packets using a random-number table consisting of 24 participants ultimately, which can draw random sampling scientifically (Zhang et al., 2020). We randomly assigned participants into two groups (n = 12): one group was trained by the traditional orthognathic surgery training method (Control group), and another group was trained with the digital training system (Experimental group). The age distribution of the participants was between 23–27 years. None of the participants had received orthognathic surgery courses and had no exposure to the orthognathic surgery before this study. The experiment started at the beginning of the semester, lasted for one year, and was evaluated at the end of the semester. The total class hours are the same for all participants. Each lesson was 2 hours long once a week and taught by instructors with more than 10 years of teaching experience. The theoretical content covered preoperative preparation, basic surgery principles, applied anatomy, rigid internal fixation, and operative procedure, and is based on Orthognathic Surgery (People’s Health Publishing House).

To evaluate the learning efficiency of residents through the traditional and digital training system, two objective tests (the theoretical exam and practical exam) and one subjective judgement (instructors’ evaluation of residents’ performance) were included in the analysis. The whole evaluation process is shown in Fig. 1C. Test 1 was in the form of a paper exam. The instructors prepared questions based on three sections: the basic knowledge of anatomy, complications, and surgical points involved in the teaching. Including 20 multiple-choice questions, 20 blank filling questions, and 5 short answer questions, with a total score of 100. We prepared the questions refer to the Standardized Training and Assessment Outline for Residents of Oral and Maxillofacial Surgery (National Health Commission of the PRC) and Accreditation Standards for Advanced Dental Education Programs in Oral and Maxillofacial Surgery (American Dental Association).

Test 2 was a practical exam conducted on imitation head mold. The content of the exam is the LeFort I osteotomy for maxillary with sagittal split ramus osteotomy for mandibular on the bony Class III malocclusion imitation head mold. The combined use of the two surgical approaches is the most common procedure in clinical practice. Bimaxillary surgery can better examine the residents’ understanding of the relationship between the spatial anatomical structure of the jaw. Test 2 is divided into five items: design of surgical incisions, design of osteotomy lines, use of guide plate, placement of bone blocks, and fixation of bone blocks. Each item’s operation, presentation, and time consumption were examined, and each item was scored from 0 to 20 out of 100 points. Since the two groups of residents learned different methods of making the guide plate, we used a uniform guide plate in the operation test, and the test mainly examined the residents’ understanding and proficiency of the main points of the operation. After completing the exams, the scores were calculated, and the two groups’ results were compared.

Simultaneously, three instructors with more than ten years of teaching and clinical experience who were not involved in this teaching activity gave subjective evaluations by observing the whole process of residents’ practical examinations and asking random questions. Prior to participating in the evaluation, the instructors developed uniform scoring criteria referred to the Detailed Rules for Assessment and Scoring of Standardized Training for Residents (National Health Commission of the PRC). In their marks, A, B, C, and D represented excellent, good, general, and bad, respectively.

Questionnaire

To collect residents’ feedback about two different learning systems, a short questionnaire consisting of ten questions was distributed to all residents in Control group and Experimental group at the end of the course. The questionnaire contains 10 questions. Each question used a 5-point Likert scale, in which 1 represents “strong disagreement” and 5 represents “strong agreement”. Likert scale is a close-ended, forced-choice scale used in a questionnaire that provides a series of answers that go from one extreme to another. Likert scales are widely used in medical education research (Horan, Murphy & O’Brien, 2020; Manalo et al., 2021; Yang et al., 2021). As shown in Table 1, each question had a particular focus. Q1 and Q10 assessed respondents’ satisfaction; Q2 and Q9 assessed learning interest; Q3 and Q4 assessed academic outcomes; Q5 and Q8 assessed respondents’ confidence for engaging in orthognathic surgery; and Q6 and Q7 assessed the ability of handling practical operation. These aspects can be used as direct or indirect indicators of teaching effect evaluation (Alamrani et al., 2018; Kouz et al., 2020).

Data analysis

SPSS 26.0 Statistics was used to do data analysis. Independent-samples T-tests were conducted and analyzed to determine statistically significant differences between the Experimental and Control groups of scores of three sections related to anatomy, operation theory, and complication in Test1 and scores of practical operations in Test2. Non-parametric tests Mann–Whitney U and Wilcoxon rank-sum test were performed to analyze the qualitative data obtained from subjective evaluation and the questionnaire. The significance level was set as p < 0.05.

Theoretical tests

The scores of the theoretical test were collected separately and analyzed by t-test to investigate the difference between the Control group and the Experimental group as demonstrated in Table 2. For orthognathic surgery related anatomy, residents’ scores were 73.72 ± 6.05 and 82.33 ± 5.45 in the Control group and the Experimental group, respectively, and the scores of operation theory were 70.67 ± 6.37 and 83.42 ± 7.18, respectively. The results of the t-test indicated that residents of the Experimental group mastered the knowledge of anatomy and operation much better (P = 0.002 for anatomy, P = 0.000 for operation theory). Regarding complication understanding, residents’ scores were 78.42 ± 5.73 and 77.75 ± 5.34 in the Control group and the Experimental group, respectively, where P = 0.771 indicated that no statistically significant difference was found between the two groups.

The test on the practical operation and subjective judgments from instructors

As shown in Table 3, the test scores on the practical operation were 70.83 ± 4.63 and 85.67 ± 5.00 for the Control group and the Experimental group, respectively, with a maximum score of 94 in Experimental group. The statistical analysis showed that significant differences existed (P = 0.001) between the Control group and the Experimental group on the practical operation, implying residents from Experimental group could perform better in the clinical practice. Meanwhile, the results of feedback from instructors were summarized in Table 4. Significant differences also existed between the two groups (P = 0.013), indicating the instructors gave more recognition to the Experimental group’s practical operation than the Control group, which was in accordance with the test result on the practical operation.

Questionnaire

A questionnaire containing 10 questions was distributed to both groups of residents anonymously. The descriptive statistics of the questionnaire was shown in Table 5. The mean values of each evaluated factor provided by the Control group residents, who experienced conventional training, were lower than those provided by the Experimental group residents, who experienced the digital training. In particular, the difference in mean score for Learning Interest between the two groups was 12, indicating that the Experimental group enjoyed the learning process much more than the Control group (P = 0.001). Meanwhile, residents in the Experimental group could understand the anatomical structure of the surgical area and orthognathic surgery procedures better, as shown in Academic Outcome (P = 0.001), which was coherent with the outcome from theoretical tests. For the operation part, residents in the Experimental group had much more confidence in clinical work such as guide plate preparation or first assistant duty (P = 0.001), which is consistent with the previous test result on the practical operation and subjective judgments from instructors. Therefore, residents in the Experimental group were more satisfied with the training (P = 0.002) and held more confidence (P = 0.001) in the clinical practice than those in Control group.