Validity and reliability of a teledentistry survey among dental practitioners in Saudi Arabia

Introduction

Teledentistry uses information technology and telecommunications for oral healthcare delivery, consultation, and education (Di Cerbo & Morales, 2015; Scott et al., 2018; Yang et al., 2015; Daniel & Kumar, 2014). It includes diagnosing and assessing the severity of dental conditions, treatment planning and scheduling, providing oral health education and reviewing cases (Kumar et al., 2019; Jampani et al., 2011). This technological approach allows underserved patients in rural areas to connect and communicate with dental service providers, granting them access to general dentists and specialised dental treatments if needed (Daniel, Wu & Kumar, 2013; Estai, Kruger & Tennant, 2016a, 2016b). Teledentistry has shown potential in fields such as oral medicine, orthodontics, dental trauma, periodontology and caries diagnosis in pediatric dentistry (Haron et al., 2017; Batham, Pereira Kalia & Dilliwal, 2014; Marino et al., 2017; Bhambal, Saxena & Balsaraf, 2010; Morosini et al., 2014). This cybernated technology has gained significant traction since COVID-19 pandemic onset. Teledentistry is one of the practical methods used extensively to recompense the difficulties of dental care accessibility. However, its application and practitioner interest in this regard have been subject to scrutiny (Aboalshamat, 2020; Giraudeau, 2021; Macapagal, 2020; Abbas et al., 2020; Sycinska-Dziarnowska et al., 2021; Imran Farooq, Moheet & AlHumaid, 2020; Chaudhary et al., 2021). Some studies state the positive reliability, validity, and efficacy on the usage outcomes (Tiwari et al., 2022; Estai, Kruger & Tennant, 2016c; Aboalshamat et al., 2022; Alsharif & Al-harbi, 2020). However, there are some factors such as information confidentiality related to patient data, informed consent, the risk of misdiagnosis, ethical factors, and financing problems that play a role in its impediment (Aboalshamat et al., 2022). To assess dental practitioners’ perceptions of teledentistry, Mandall, Quereshi & Harvey (2005) developed the Teledentistry Survey (TDS) based on interviews with eight general dental practitioners (GDPs). The survey questions, which were derived from these interviews, focus on four factors: efficiency, usefulness, GDPs’ perspectives and concerns. Although this survey has been widely used to study dental practitioners’ perceptions of teledentistry in Australia, Canada, the United Kingdom, Saudi Arabia and Indonesia, limited evidence exists regarding its validity (Estai, Kruger & Tennant, 2016a; Aboalshamat, 2020; Tiwari et al., 2022; Estai, Kruger & Tennant, 2016c; Aboalshamat et al., 2022; Alsharif & Al-harbi, 2020; Mandall, Quereshi & Harvey, 2005; Soegyanto et al., 2022; Patel & Antonarakis, 2013). Although, teledentistry has been extensively covered in the region, this study will provide a unique perspective by focusing on the psychometric analysis of the TDS that hasn’t been explored comprehensively in diverse contexts. Notably, the Saudi dental community comprises a diverse spectrum of dental professionals with expertise spanning many aspects of oral healthcare. The diversity of the dental field in Saudi Arabia enables our study to produce results that can be compared effectively with those of other countries using similar survey instruments and methodologies. Comparative research adds value to our work since it allows us to draw meaningful insights, benchmark our findings against international standards, and gain a deeper understanding of dental practices and trends globally. To establish the generalisability of the (TDS-24) in terms of assessing dental practitioners’ perception of teledentistry, this study examined the factorial structure and psychometric properties (validity and reliability) of the TDS-24 among dental practitioners in Saudi Arabia. In our exploratory factor analysis of the TDS-24 survey, we aimed to measure its psychometric properties. This involved evaluating the factor loadings of each item to determine their strength and direction of association with the underlying factors. By identifying these factors and their corresponding items, we intended to provide insight into the latent constructs or domains within the survey and explore the potential reduction of the scale to a more concise and psychometrically sound version.

Materials and Methods

Results

Psychometric properties of the TDS

The factor analysis conducted for the entire dataset revealed that the KMO measure of sampling adequacy was 0.900. The dataset revealed a correlation matrix of many coefficients of ≥0.3 and Bartlett’s test of sphericity was significant (χ²(df) = 3,863.94 (253); p ≤ 0.001), supporting the suitability of the data for factor analysis. The square root of the average variance extracted (AVE) was compared with correlations between the constructs under study to ensure discriminant validity. For each construct, the square root of AVE was larger than correlations with other factors. A four-factor solution accounting for 47.34% of variance was obtained from the factor analysis of the TDS. The slope of the Cattell’s Scree plot test explicitly demonstrated the existence of these four factors (Fig. 1).

As shown in Table 3, most items had loading values higher than 0.50 on all factors. However, four items of the proposed TDS were dropped in this dataset analyses, namely items 12, 16, 21 and 24 (Table 3). Interestingly, some items demonstrated significant loading values on factors different from those originally assigned. For instance, items 8, 9 and 10 of factor II had loading on factor I (0.763, 0.702 and 0.671, respectively), and items 13, 15, 17 and 18 of factor III exhibit loading on factor I. Similarly, item 6 of factor I had loading on factor II (0.673); items 14, 16, and 19 of factor III had loading on factor II; items 2, 4 and 5 of factor I had loading on factor III (0.637, 0.621 and 0.681, respectively) and items 20, 22 and 23 of factor IV showed loading on factor IV. Some items were loaded on different factors from those originally considered. The Cronbach’s alpha for all factors was 0.85, and it was 0.86 for the efficiency factor, 0.70 for the cost factor, 0.52 for the capability factor and 0.57 for the security factor (Table 3). The Cronbach’s Alpha if items were deleted was >0.80 for each individual item, ranged from 0.841 to 0.859. For the corrected item-total correlation, most items had values higher than 0.50. More details are presented in Table 3.

Table 3:

Reliability and validity psychometric analyses of the teledentistry survey.

Teledentistry survey factors	Corrected item-total correlation	Cronbach’s alpha if item deleted	Factors loadings
			Factor 1	Factor 2		Factor 3	Factor 4
Efficiency in patient care
9. Improve communications between dentists	0.565	0.842	0.763
8. Make the referral of new patients more efficient	0.579	0.841	0.702
10. Enhance guidance and advice	0.537	0.843	0.671
7. Enhance clinical training and continuing education	0.526	0.843	0.665
3. Save time as compared with a referral letter	0.560	0.842	0.673
17. Help with patient information and education	0.522	0.844	0.586
7. Provide adequate diagnostic information	0.530	0.842	0.541
15. Helpful in monitoring a patient’s condition	0.506	0.844	0.568
13. Useful for patients in distant or rural locations	0.572	0.841	0.563
18. Improve interaction and communication with patients	0.527	0.843	0.538
Cost reduction
6. Reduce costs for dental practices	0.407	0.847		0.673
19. Save money for patients	0.394	0.847		0.599
16. Help reduce unnecessary travel to hospitals	0.545	0.842		<0.50
14. Convenient for patients and well received by patients	0.542	0.842		0.546
11. Help shorten waiting lists	0.488	0.844		0.559
Capabilities to improve practice
5. Increase surgery time spent with the patient	0.227	0.855				0.681
4. Necessitate an additional appointment for taking photographs	0.266	0.853				0.621
2. Too expensive to set up	0.095	0.859				0.637
12. Diagnosis based on intra-oral images is as accurate as in a traditional clinical setting	0.356	0.849				<0.50
Security and confidentiality
20. Reliability of equipment	0.179	0.855					0.671
21. Technical incompatibility	0.517	0.843					<0.50
22. Patient confidentiality when images are sent online to the hospital	0.301	0.851					0.684
23. Potential for tampering with computer images	0.137	0.858					0.739
24. Gaining patient consent for referrals via email	0.517	0.843					<0.50

DOI: 10.7717/peerj.16834/table-3

Note:

The bold values indicate the loading rating, with 0.55 being categorized as good, 0.63 as very good, and 0.71 as excellent.

Discussion

This is the first study to assess the factorial structure and psychometric properties of the TDS-24 (Estai, Kruger & Tennant, 2016c) using EFA (a data-driven method) based on a diverse sample of dental practitioners in Saudi Arabia. In comparison with Estai’s proposed factors, our EFA found a well-separated four-factor structural model, the items of the four-factor model have greater loadings for their corresponding factor and almost half of the items can be explained by a single factor. It is a shortcoming of factor IV that it has only three items; however, according to the guidelines, one factor should have more than two items if possible (Crawford, 1975; Zwick & Velicer, 1986). The phenomenon in which some items show loadings on factors different from their intended ones is a common occurrence in factor analysis and can be attributed to ambiguity in item wording, item redundancy, conceptual overlapping or, in some cases, factors that are not strongly distinct. Also, the underlying structure of the data may not be perfectly aligned with the proposed factor model. On the other hand, the characteristics of the sample used in the factor analysis can influence the results. If the sample has unique traits or experiences that differ from the population the scale was designed for, this may lead to unexpected item-loadings in the factor analysis. Thus, researchers should explore the theoretical underpinnings of the scale, review the wording of ambiguous items and evaluate the conceptual overlap between factors. It might also be beneficial to conduct further analyses, such as confirmatory factor analysis, to test the stability and validity of the factor structure with an independent dataset. Ultimately, the goal is to ensure that the factor structure accurately represents the underlying constructs being measured and that items are appropriately assigned to the intended factors.

Four items (items 12, 16, 21 and 24) were excluded, and the exclusion of these items could have been due to their weak correlation with the underlying factors or their redundancy with other items. Thus, the exclusion of these items should be carefully considered when interpreting the results.

The EFA of the scale yielded 20 items supporting a four-factor structure as follows: factor I (10 items), factor II (four items), factor III (three items) and factor IV (three items). The differences in item loadings were anticipated. It is not realistic to expect that the factor loadings will be identical across countries. Dental practitioners from Saudi Arabia have perceived and reported items differently from those in other countries, which resulted in slight variability in the TDS loadings and, as such, factor naming. All the extracted factors showed acceptable or good internal consistency (Cronbach’s α). Using such a model ensures sample adequacy and the validity and reliability of the interpreted data.

The EFA confirmed that the TDS-20 had good model fit. As this study was the first to assess the factorial structure and psychometric properties of the TDS-23, comparisons of the findings of this study with those of other studies were not possible. The 20 items loaded under the four-factorial structure were suggested to be related to the following factors: ‘efficiency in patient care’, ‘cost reduction’, ‘capabilities to improve practice’ and ‘security and confidentiality’. However, Estai’s proposed four factors for the perception of TDS were as follows: ‘the dentist’s perceived usefulness of Teledentistry’, ‘the efficiency of Teledentistry’, ‘the benefits of Teledentistry for patients’ and ‘the practice-related use of information technologies’.

In the context of teledentistry, the theoretical technology acceptance model (TAM), proposed by Davis (1989), plays a crucial role in understanding the factors influencing the acceptance and adoption of technology by dental practitioners (Kamal, Shafiq & Kakria, 2020). The results of our EFA revealed that the TDS-20 questionnaire, a modified version of the TDS-24, effectively captured four essential factors related to teledentistry acceptance. These factors, namely ‘efficiency in patient care,’ ‘cost reduction,’ ‘capabilities to improve practice,’ and ‘security and confidentiality,’ align with the TAM constructs, shedding light on how dental practitioners in Saudi Arabia perceive the ease of using teledentistry and its usefulness in their practice. These results align with findings from multiple studies (Aboalshamat, 2020; Alsharif & Al-harbi, 2020; Almazrooa et al., 2021). Furthermore, the study’s findings indicate the importance of considering cultural and contextual differences when applying TAM to teledentistry. The variations in factor loadings and factor naming compared to Estai’s proposed factors highlight the need for a nuanced approach in understanding technology acceptance within specific regions and populations. This reinforces the significance of conducting cross-cultural validation studies and tailoring technology acceptance models to the unique needs and perspectives of diverse user groups.

The strengths of this study are that it is likely the first study to cross-culturally validate the TDS using a diverse sample in a large-scale cross-sectional survey. This study introduces a refined survey version named TDS-20, which is a modification of the original TDS-24 questionnaire. This adaptation stems from the results of an exploratory factor analysis, which unveiled that items 12, 16, 21, and 24 exhibited low factor loadings and were subsequently excluded from the TDS-24. These exclusions were made in TDS-20 to enhance the scale’s precision and relevance in the context of our research. This separate study goes beyond the surface by delving into the intricacies of the survey, employing exploratory factor analysis. The results unveil intriguing findings as the proposed domains in the survey underwent significant shifts when applied in the Saudi context, suggesting the emergence of new domain categories.

The limitations of the study can be summarised as follows: the information was retrieved from the participating subjects using the English language, which is the language of teaching in dental schools in Saudi Arabia. However, cross-cultural revalidation in other languages is warranted. A further limitation of this study is its exclusion of clinicians who are not active on social media networks. One should consider that the factor structure of the TDS could have been influenced by the present sample data (i.e., those who were interested in the study participated). Therefore, revalidation studies using different populations and random samples are needed to overcome sampling bias and confirm the latent variable structure. Participants’ self-reported perceptions were not verified with objective measures and, as such, were subject to recall bias. Finally, this study’s sample had socially advantageous backgrounds. Saudi Arabia is classified as a high-income country, but the revalidation of the TDS in low- and middle-income countries should be undertaken. Our study was also limited by the cross-sectional design and use of an anonymous online questionnaire, which made it challenging to assess test-retest reliability.

In conclusion, the TDS was found to be valid and reliable in assessing teledentistry perceptions among dental practitioners in this study sample. This study proposes a modified version of the TDS-24 survey as the TDS-20. Considering the findings from this study, it is recommended that the modified TDS-24 survey can guide further development of the survey to better suit resemble context. The results of the factor analysis provided support for the underlying structure of the proposed TDS. While most items demonstrated satisfactory loading values, the exclusion of three items and the cross-loading of others highlighted potential areas for improvement. The factors of efficiency and cost exhibited good internal consistency, whereas the capability and security factors showed room for improvement. The findings derived from this factor analysis contribute to our understanding of the proposed TDS and its potential applications. However, this scale can be improved if the items are adjusted via further studies. Further construct validation and construct convergence validation of the TDS with unexplored factors relevant to teledentistry use (e.g., technology-related items) should be explored in future research to enhance scale validation.

Supplemental Information