The application of deep learning in early enamel demineralization detection

Trial design

All procedures involving digital photographs of human subjects received approval from the West China Hospital of Stomatology Ethics Committee (WCHSIRB-CT-2022-044). Only patients who consented to the use of their intraoral images for research were included in this study. The photographs were taken by a nurse specializing in oral photography. A total of 237 adult patients who underwent oral examinations at a single institution, from January to December 2022, were selected. Each patient had one digital photograph taken in the frontal occlusal position, one in the left occlusal position, and one in the right occlusal position. The photographs were divided into training, validation, and test sets in a ratio of 7:2:1. Two senior dentists reached a consensus through clinical examinations and patient history discussions, marking tooth contours and demineralized lesions on the photos, which served as the required samples for the study. Dental photos were taken using a professional DSLR camera, using macro flash after tooth cleaning and drying (Kühnisch et al., 2022). All images were stored in jpeg format with RGB colour for use in this research project.

Inclusion criteria for photos: (1) permanent dentition; (2) presence of only early enamel demineralization issues; (3) all three images clear and unblocked. Exclusion criteria: (1) obscured dentition; (2) extensive tooth loss or previous treatments like dental restorations or implants.

Table 1 displays the demographic and clinical characteristics of the patients.

The dental photos were retrospectively and randomly used for enamel demineralization detection by dentists with or without the DL assistance. Two senior dentists (Juan Li, Rongxiu Zhang), two junior dentists (Ketai He, Keyue Tian), and one computer scientist(Jiajun Wang) participated in the experiment. The computer scientist created a DL model based on the annotated dataset, the two senior dentists supplied example annotations. The two junior dentists subjectively assessed the photos with and without the assistance of DL. Finally, the recognition of two junior dentists without DL assistance was compared with that with DL assistance.

An overview of the study is presented in Fig. 1.

The demineralization index, representing the percentage of demineralized area relative to the total tooth area, was calculated using DL on the training set. The maxillary and mandibular anterior teeth were calculated in the frontal occlusal photos and the respective posterior teeth (excluding the second and third molars) were calculated in the lateral photos on each side.

Sample annotation

Labelme is a Python-based image annotation tool with a graphical interface built using Qt application framework. We utilized version 4.5.13 to annotate tooth contours, marking early enamel lesions when present, and outputting annotations in JavaScript Object Notation (JSON), a lightweight data-interchange format.

In our study, two senior dentists with over 10 years of experience examined patients and conducted visual inspections based on their medical histories. Subsequently, they marked tooth contours and corresponding demineralized areas on digital photos. The annotation rules are outlined in Table 2, and an illustration is provided in Fig. 2. The dentists were blinded to each other’s responses. To minimize potential subjective impacts on the results, the two dentists underwent testing before the experiment, resulting in a Cohen’s κ > 0.8(0.858), indicating consistency. Only images with consistent annotations from both senior dentists were included in the database, which is critical for the study’s validity.

Preprocessing

Out of the initial 711 digital photos, 87 were excluded. Ultimately, a total of 624 digital photos were included in the study. The digital photographs were randomly divided into three groups: training, testing, and validation. The data was divided in a ratio of 7:2:1, ensuring that each patient’s photos belonged to only one subset. For the purposes of training, validating, and testing the model, the input picture resolution was standardized. The training set comprised 444 images from 148 patients. The validation set included 120 images of 40 patients, and the test set had a total of 60 photos of 20 patients for testing.

DL model for enamel demineralization detection and area calculation

Data augmentation is a data-space solution to address the limited data problem. Current deep convolutional neural networks rely on large datasets to avoid overfitting. In the medical image analysis domain, obtaining a substantial number of similar images is challenging due to ethical constraints. Therefore, data augmentation techniques that enhance dataset size and quality become a viable solution. When processing the secondary database, the team employed three data augmentation methods: cutout, cutmix, and mixup. Additionally, the project utilized an improved Mask R-CNN network for demineralized area segmentation, with the ConvNext network serving as the backbone for the enhanced Mask R-CNN network. Subsequently, the segmented demineralized area was used as input for the Segment Anything network to segment teeth, ensuring precise tooth segmentation and area calculation. This method significantly enhanced recognition efficiency. Pixel area calculations were then used to determine the percentage of the demineralized area relative to the total tooth area, providing input for subsequent scoring.

The team conducted area calculations based on the segmented mask information, and the results were automatically recorded in the experimental records (e.g., Excel spreadsheet).

We call the result of this calculation the demineralized index. In other words, the index is the demineralization area divided by the total tooth surface area. It can reflect the proportion of tooth demineralization, and thus more accurately reflect the severity of tooth enamel demineralization. This index was calculated for specific tooth positions in the training set.

Statistical analysis and evaluation criteria

Excel and SPSS 26.0 were used for all statistical analyses. We assessed the strategy’s effectiveness using metrics like sensitivity (SEN), specificity (SPEC), F1-score, positive predictive value (PPV), negative predictive value (NPV), receiver operating characteristic (ROC) curves, and area under the ROC curve (AUC). These measures were computed for each individual, each picture, and each tooth (Li et al., 2022). The following formulas was used to calculate the metrics:

$$\mathrm{N}\mathrm{P}\mathrm{V}={\displaystyle \frac{\mathit{T}\mathit{N}}{\mathit{T}\mathit{N}+\mathit{F}\mathit{N}}}$$

$$\mathrm{P}\mathrm{P}\mathrm{V}={\displaystyle \frac{\mathit{T}\mathit{P}}{\mathit{T}\mathit{P}+\mathit{F}\mathit{P}}}$$

$$\mathrm{S}\mathrm{E}\mathrm{N}={\displaystyle \frac{\mathit{T}\mathit{P}}{\mathit{T}\mathit{P}+\mathit{F}\mathit{N}}}$$

$$\mathrm{S}\mathrm{P}\mathrm{E}\mathrm{C}={\displaystyle \frac{\mathit{T}\mathit{N}}{\mathit{T}\mathit{N}+\mathit{F}\mathit{P}}}$$

$$\mathrm{F}1\text{-}\mathrm{s}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{e}={\displaystyle \frac{2\ast \mathit{P}\mathit{P}\mathit{V}\ast \mathit{S}\mathit{E}\mathit{N}}{\mathit{P}\mathit{P}\mathit{V}+\mathit{S}\mathit{E}\mathit{N}}}$$where TP, FP, TN, and FN represent true positive, false positive, true negative, and false negative, respectively. Positive/negative indicates the model predicting the presence/absence of demineralization, and true/false indicates correct/incorrect predictions. The F1-score, representing the harmonic mean of PPV and SEN, is a commonly used evaluation metric for classification models in DL. It balances these two metrics and is a comprehensive measure of model performance, especially when there is an imbalance between positive and negative samples. In this study, F1-score was used to evaluate the accuracy and reliability of the DL model and junior dentists in detecting demineralization. Perfect PPV and SEN are indicated by the maximum F1-score of 1, which is 1, and zero for either PPV or SEN. The range of all metric values is 0 to 1 (Li et al., 2022).

Subjective assessment and DL-assisted evaluation

The DL model underwent evaluation on a validation set comprising 120 images from 40 patients. To compare the performance under the guidance of DL, two junior dentists with 3 years of experience independently assessed the same set of 60 images from 20 patients, and the DL model also assessed this test set. These junior dentists relied solely on their clinical experience without additional training. Both junior dentists underwent a consistency assessment before the experiment (Cohen’s κ > 0.8). For these photos, they assessed the concordance between the DL screening outcomes and their subjective visual evaluations. If the results were inconsistent, the dentists made final decisions based on their judgments, referring to the DL’s results (Li et al., 2022).

Experimental setup

Experiments were conducted using Python 3.8 and PyTorch 1.10 on a single NVIDIA V100 GPU to ensure sufficient computing power to handle DL tasks. The ConvNext-T network was used as the backbone for the Mask R-CNN network for demineralized area segmentation. The ViT-H SAM network served as the tooth area segmentation network. Before training, all images were standardized to ensure consistent data distribution. During training, we employed five-fold cross-validation to obtain better model parameters. Regarding hyperparameter configuration, we used the SGD optimizer with an initial momentum parameter of 0.9, weight decay parameter of 0.0001, learning rate of 0.004, batch size of 8, and a total of 30 training epochs. The model achieved optimal performance at the end of the 26th epoch. This model is then used on the test set.

Model performance

On the validation set, the model demonstrated a SEN at 0.935, SPEC at 0.789, PPV at 0.968, NPV at 0.991, and an F1-score at 0.856 at the tooth level (refer to Table 3). The model’s ROC curves, depicting AUC values at the individual, image, and tooth levels, are presented in Fig. 3. On the test set, the model also showed a strong capacity to learn features from digitally annotated photos. Compared to the junior dentists, it performed well in detecting demineralization (see Table 4). Specifically, for different tooth positions, the model performed best in recognizing upper premolars, with F1-scores of 0.963 on the test sets. The lowest performance was observed in recognizing lower molars, with F1-scores of 0.727 (see Table S3 for detailed dentition test results). Figure 4 illustrates the DL model’s performance in detecting enamel demineralization.

Comparison of DL-assisted and dentist-only detection

We contrasted the annotation performance and efficiency of junior dentists with and without DL assistance in detecting demineralization on digital pictures to evaluate the model’s applicability thoroughly. Our data show a significant improvement in the performance of junior dentists with DL assistance compared to without, both on a per-image and per-tooth basis: Dentist 1 (Images: p = 0.099; Teeth: p = 0.019), Dentist 2 (Images: p = 0.026; Teeth: p < 0.01). The detection capability of two junior dentists for dental enamel demineralization improved significantly, with the F1-scores for demineralized teeth increasing from 0.713/0.689 to 0.897/0.949 respectively. The performance in SEN and SPEC was equally remarkable. However, the standalone model and junior dentists without assistance demonstrated lower sensitivity and higher specificity in demineralization identification. The primary challenge for junior dentists was the tendency to misclassify, which was effectively addressed with DL assistance, outperforming either of the two alone.

In addition, with the help of DL, two junior dentists made satisfactory improvements in the identification of demineralized areas. Regarding the demineralization area, the accuracy of their annotations improved with DL assistance, as indicated by the Intersection over Union (IoU). Their IoU scores increased from 0.571/0.527 to 0.704/0.682, respectively.

Basis of demineralization index

Figure 5 shows that the tooth contours outlined by the DL model closely match the real contours. Based on the test set, the IoU of the tooth contours with the gold standard reached 90.3%. This allows the DL-drawn contour area to be directly used as the denominator when calculating the percentage of demineralized area for individual teeth. Thus, this lays the foundation for the automated calculation of the demineralized index. We found that the average demineralization index was highest for upper molars (0.243) and lowest for lower incisors (0.104), precisely in line with clinical patterns of caries occurrence. This indicates the index’s practical value to some extent.