Detection and diagnosis of diabetic eye diseases using two phase transfer learning approach

Novelty of this research

The most significant contributions of this research are,

• This research proposes a two-phase transfer learning approach for classifying and segmenting multi-class DED diseases.

• By removing undesired abnormalities, image pre-processing techniques improved the quality of the images. The unwanted noise could be removed from the grayscale image using median filtering. The image contrast was enhanced by utilizing the Contrast-Limited Adaptive Histogram Equalization (CLAHE) method.

• Effective classification is achieved using the transfer learning-based Modified ResNet-50 model, which classifies the fundus efficiently, including diabetic retinopathy, glaucoma, DME, and cataracts.

• After classification, it is challenging to identify diabetes-related eye disorders using machine learning-based segmentation. The transfer learning-based DenseUNet model is used in the proposed system for the localisation of disease regions. The segmentation results provide precise identification of the affected areas that guarantee accurate disease identification effectively.

• The proposed approach is assessed using publicly accessible datasets. The Python platform is utilized to evaluate the proposed approach. According to the experimental results, out of all the approaches, the proposed approach outperforms the state efficiency.

Research gaps of existing models

• Most existing deep learning-based models fail to detect and characterize small lesions or early-stage pathologies in fundus images, as early detection is crucial for effective treatment.

• For multi-class classification of diabetic eye diseases, several prior models require high computing resources and are not applicable for accurate word settings.

• Most deep learning models for fundus image analysis are considered black boxes. Research is needed to make these models more interpretable, especially for clinical adoption.

• The performance results of the segmentation for accurate lesion segmentation are decreased in most of the existing deep-learning models.

To solve the research gaps in the existing deep learning models, an efficient combined deep learning-based classification and segmentation model is proposed in this research. In the proposed approach, a reliable and effective two-phase transfer learning model is developed in this research.

Solutions of proposed research

The research employs a two-phase transfer learning approach to address the issues of class imbalance and data over-fitting. By training on high-resolution data with fewer resources and utilizing a smaller batch size, the issue of high computational resource requirements for training on high-resolution data is resolved in this research. Image pre-processing using an effective filter and contrast enhancement technique was employed to categorize low-intensity and noisy data images; it generates precise segmentation and classification outcomes. Effective pre-processing filters and hyper-parameter tuning are used in this research to overcome the issue of an unstable network model, which can significantly enhance the model’s performance and dependability.

In the segmentation network model, we used a dropout layer. The dropout layer randomly sets a fraction of the neurons to zero during each training iteration, which helps prevent the co-adaptation of neurons and reduces the over-fitting problem. We used a minimum number of layers in the classification and segmentation models; the network over-fitting is avoided based on this smaller number of layers. We set the batch size of 20 for training; smaller batch sizes introduce more noise into the training process, which can help prevent over-fitting. We used effective hyper-parameter optimization for network training to avoid over-fitting the network. We used transfer learning with pre-trained models for classification and segmentation; it is used to enhance the generalization abilities of the models and solve the over-fitting problem.

Dataset collection

The open-source datasets were gathered from DRISHTI-GS, Messidor-2, Messidor, and Kaggle cataract datasets. Despite their modest size, the images in the Messidor dataset were of excellent resolution and had precise categorization. A similar public dataset, Messidor-2, was used by others to analyze the output of the DED algorithm. A total of 1,748 images representing 874 subjects make up the data. Messidor-2 differs from Messidor’s original 1,200-image data set for each eye. Glaucoma and cataract datasets were acquired from the retina dataset, and Kaggle was used to obtain this dataset. Glaucoma and cataracts were each represented by 100 images in this dataset. The dataset distribution is shown in Fig. 2.

Data pre-processing

Classification using modified ResNet-50

The classification of the DED dataset was incorporated by employing pre-trained CNNs in this research. The specific feature representations were obtained by converting a feature vector with a specified weight matrix in deep CNN without any gaps in the spatial arrangement’s information. The idea applies features discovered while working on the original task to the targeted jobs. Research areas using significant data and computing resources benefit from transfer learning. The details of the pre-trained models are presented in this section.

We propose a new pre-trained CNN architecture based on a modification to the ResNet50 architecture. In the ResNet50 model architecture, additional layers are added to better fit the fundus image collection. Low-resolution fundus images are taken, and the height-to-width ratio may vary. To implement a similar approach in the developed model architecture, the images in the dataset are resized into 224 × 224 × 3 for training and testing.

As ResNet could achieve greater accuracy and was relatively simple to improve, it has an excellent deep-learning architecture. Furthermore, the network’s skip connections resolve persistent vanishing gradient problems. The network’s time complexity increases proportionately to the deep network architecture’s layer count. To lessen this complexity, consider using a bottleneck design. As a result, we chose to construct our framework using the ResNet 50 model and ignored other networks with more layers. The architecture is explored in more detail below in Fig. 3.

The ResNet50 architecture has been modified to classify retinal diseases and achieve adequate performance. The last three layers of the ResNet50 architecture, including the classification layer, softmax layer, and fully connected layer, were first modified for our classification process. Our modified network used The fully connected layer instead of the convolution layer. The output size is represented based on the five classes: regular, diabetic retinopathy, glaucoma, DME, and cataracts.

The three layers, convolution, batch normalization, and Activation Relu, were added next in the basic ResNet-50 model. These three layers were used in fundus images to extract robust features automatically. These three layers are added using the procedures listed below.

• The ‘activation_49_relu’ layer is connected to the newly added ‘Conv’ layer and disconnected from the ‘avg_pool’ layer.

• The recently added ‘activation_relu’ layer is connected to the ‘avg_pool’ layer.

• The classification, softmax, and fully connected layers are the last three recently added layers; these layers are mentioned after the “avg_pool” layer in the sequence of appearance.

After injecting the new layers, the ResNet-50 network model’s modified architecture appears in Fig. 3. In the dataset, the image features were obtained by passing the pre-processed images to layers of this modified network. Then, the classification layer of this network classified the fundus images into typical diabetic retinopathy, glaucoma, DME, and cataract. The proposed model was developed to classify a variety of retinal disorders.

Segmentation

This research developed a transfer learning-based DenseUNet network model to more precisely segment DED diseases. To segment semantically, we add U-Net to the denseness model. Figure 4 shows the design of the DenseUNet model. The segmentation network model combined the FCN and u-net model’s encoder-decoder design concepts (Kaur & Kaur, 2022). This included an up-sampling path and a down-sampling path. The Input image’s semantic information was extracted by downsampling path using down-sampling and continuous convolution operations layer by layer.

Up-sampling and de-convolution were used to increase the feature map’s resolution until the original resolution of the input pictures was reached. The transition layer and dense block comprised the down-sampling and up-sampling paths. These paths connected adjacent dense blocks. The thick blocks and four transition layers are included in both down-sampling and up-sampling paths. The exact resolution for n consecutive convolution layers was present in the dense block, followed by BN, ReLU, and dropout layers.

Each layer’s input yielded the output feature maps from all previous layers. It is known as dense concatenation. With the same feature size, the layer’s feature maps must contain dense blocks for this process. The proposed DenseUNet architecture consisted of 10 dense blocks. In both the dense down-sampling up-sampling paths, five dense blocks are used. Four densely connected layers were present in each dense block with the same feature size.

They are introducing a transition block for the layer transition. The five layers are used to form the dense down-sampling path. The thick and transition blocks are presented in the dense down-sampling path. The up-sampling layer, merger operation, and thick block operation reconstruct the high-resolution images in the dense up-sampling path. Five layers made up the path, and these layers localised the regions, which also recovered the entire input resolution.

The output of the $l^{th}$ layer is defined by $x_l$. From the previous layer output $x_{l-1}$, transformation $H_l\left(x\right)$ computes the $x_l$,

(3) $$x_l=H_l\left(x_{l-1}\right).$$

A sequence of operations is represented by $H_l\left(x\right)$ convolution (Conv), pooling, BN or ReLU, etc. To avoid vanishing gradients, skip connections were introduced to improve the network's information flow and training process. From the previous layer, the feature map's identity mapping is integrated with the output $H_l\left(x\right)$ of these skip connections:

(4) $$x_l=H\left(x_{l-1}\right)+x_{l-1}.$$

Moreover, the identity function is integrated using Eq. (4) with the output, $H_l$ which can obstruct the network’s data flow.

The network introduces the dense connection to improve the data flow and enhance the skip connection.

(5) $$x_l=H_l\left(\left[x_{l-1},x_{l-2},...,x_0\right]\right)$$where (…) signifies the process of concatenation. The dropout, Relu, BN, and convolution layers comprize the nonlinear transformation function. In this network model, the transition layer contains the BN layer, the 2 * 2 average pooling layer, and the 1 * 1 convolutional layer.

Performance analysis

For the diagnosed DED, various measures were used to determine the false and true categories to analyze the effectiveness of the proposed two-phase transfer learning model. The cross-validation estimator was then employed. The confusion matrix was used to determine the classification performance. The correctly classified samples based on a specific category are represented using true positive (TP) indices. True negative (TN) indices of the confusion matrix, the extra samples that were pertinently related to certain other classes established were contained. Similarly, the false positive (FP) and false negative (FN) indices in the confusion matrix presented how many samples the classifier misestimated. These metrics’ values were determined using Eq. (6) through Eq. (8) formulas.

By dividing the total values of the confusion matrix by the ratio of true positive to actual negative values, Eq. (6) defines accuracy. The factors of the confusion matrix determined the classifier’s performance in the proposed model.

(6) $$Accuracy\left(\mathrm{\%}\right)={\displaystyle \frac{TP+TN}{TP+FP+TN+FN}}$$

(7) $$Specificity\left(\mathrm{\%}\right)={\displaystyle \frac{TN}{TN+FP}}$$

(8) $$Sensitivity\left(\mathrm{\%}\right)={\displaystyle \frac{TP}{TP+FN}}$$

(9) $$F1-Score\left(\mathrm{\%}\right)={\displaystyle \frac{2\times Precision\times Recall}{Precision+Recall}}$$

(10) $$Precision\left(\mathrm{\%}\right)={\displaystyle \frac{TP}{TP+FP}}$$

(11) $$Recall={\displaystyle \frac{TP}{TP+FN}.}$$

The Intersection of Union (IOU) and Dice score evaluate the segmentation performance.

(12) $$Dicescore=2\times {\displaystyle \frac{Precision\times Recall}{Precision+Recall}}$$

(13) $$IoU\rightleftarrows score=\frac{TP}{TP+FP+FN}.$$

The predicted and ground truth labels are used to calculate the evaluation measure.

Experimental results

Training, validation, and testing sets of images are generated from the dataset, and various fundus image types were utilized as training and validation sets. The experiment analysis is performed using 50 epochs. In training and validation, the proposed model achieved the predicted accuracy when all the epochs were complete. Combining features, the output images were constructed using the proposed two-phase transfer learning model. Then, the proposed model classified the fundus images into five significant DED disease classifications: glaucoma, cataract, DME, diabetic retinopathy, and routine. They were utilizing various performance measures. The proposed model's experimental results are shown in Fig. 5.

Table 2 presents the performance metrics derived from the confusion matrices. It demonstrates the proposed model's class-wise specificity, sensitivity, and accuracy. An accuracy of 99.02% was obtained to classify diabetic retinopathy, 98.79% for DME, 99.01% for cataracts and 99.53% for glaucoma classification. It was observed that the performance classification results had significantly improved. The proposed algorithm obtained 99.67% accuracy, 99.73% specificity, 99.54% sensitivity, and 99.02% F1-score. The class-wise performance was assessed using a small sample of the validation set's images. The proposed two-phase transfer learning model effectively performs for multiple classification DED diseases.

For the classification of multi-class diabetic diseases, the curve scores for each receiver operating characteristic (ROC) class and area under the curve (AUC) are displayed in Fig. 6. For every class, the model has outstanding AUC scores. The model is deemed acceptable and efficient if it attains the highest AUC of the ROC. The ROC curve was calculated by using the TP and FP metrics. The AUC (ROC) for the proposed methodology is 0.9516.

The proposed model’s accuracy and loss over 50 epochs of training and validation are shown in Fig. 7 for classification. Regarding the experimental results, the highest training accuracy measured was 99.67%, and the proposed model obtained 92.15% accuracy for validation. Losses for the model’s training and validation were 0.011 and 0.069, respectively. Per our experimental results, multiple ocular diseases were accurately identified and classified by proper training in the proposed two-phase transfer learning approach, including diabetic retinopathy, DME, cataract, and glaucoma.

Figure 8 shows the proposed model’s confusion matrix for classification. The X-axis indicates the proposed system outputs, and the Y-axis indicates the ground truth values.

Two evaluation metrics, IoU score and Dice score, are used to evaluate the segmentation results of each testing data for multiple classes of fundus images for the segmented maps per pixel and compared with the original ground truth. The segmentation results for multiple DED diseases using fundus images are given in Table 3. For image segmentation, it is noted that the proposed transfer learning-based DenseUNet model produced the overall highest Dice score of 95.50% and IoU score of 95.01%on the test set fundus images for different diabetic eye diseases.

Compared to datasets used to train models from scratch, datasets related to diabetic eye disease may be smaller. The proposed two-phase transfer learning reduces the chance of overfitting by improving the smaller dataset and utilizing the pre-trained model’s generalization abilities. This method frequently produces more reliable models and enables better use of the available data. Fine-tuning the network layers usually results in faster convergence during training on the diabetic eye disease dataset, as the initial layers of the pre-trained model have already learned general features from a dataset. This may shorten the amount of time needed to train the model. Images of diabetic eye diseases might differ significantly in quality, light, and patient features. Transfer learning facilitates the model’s acquisition of robust features that overcome this kind of unpredictability, improving classification performance. Images of diabetic eye diseases may differ in pathology, texture, and colour. Transfer learning allows the segmentation model to adapt to these variances and produce more accurate segmentation results across various image features by fine-tuning on a smaller dataset.

Time complexity analysis

The estimated processing time (for this study) is shown in Table 4, and it is an essential factor in the image retrieval process. Each image’s processing, training, and testing times comprise the complete time procedure. Pre-processing, classification, and segmentation processes comprise the processing time; reading the image is the first step, and segmentation is the last. Likewise, the training time for each network is the amount of time required to train the complete dataset. Each network’s prediction and vote constitute the only testing time. However, this research estimated a processing time of around 10 s, which is comparatively quicker than state-of-the-art methods. The average power consumption to compute each sample was recorded as 25.656 W.

Comparison results

Using the two evaluation metrics, including the Dice score and IoU score, the segmentation results of each testing data were compared to the original ground truth for the segmented maps per pixel, as shown in Table 5. For multi-class diabetic eye illnesses, it should be emphasized that the proposed transfer learning-based DenseU-Net model generated the highest overall dice score of 96.34%. Additionally, across many classes of diabetic eye disorders, the suggested model had the most significant overall IoU Score of 91.21%. Regarding the Dice score and IoU score, the proposed transfer learning-based DenseU-Net model performs better than the other related segmentation models (Wang et al., 2019; Wang, Li & Cheng, 2023) based on the comparative results.

The proposed method’s classification results were compared with those from prior research using fundus images. The comparison results in Table 6 were used to conduct a detailed analysis of the proposed two-phase transfer learning model in terms of specificity, sensitivity, and accuracy.

Gour & Khanna (2021) developed a Transfer learning-based VGG16-SGD model, and 96.49% accuracy was achieved by this model for multiple retinal disease classification. Demir & Taşcı (2021) provided a hybrid deep learning-based R-CNN+LSTM model for the multi-classification of retinal diseases, which classified the fundus image to DR, glaucoma, AMD, myopia, hypertension, and cataract. Nazir et al. (2021) utilized the CenterNet model and achieved 98.01% accuracy for multi-class DED disease classification. An accuracy of 81.33% was attained by Sarki et al. (2021b) for the multiclassification of normal, DR, DME, glaucoma, and cataracts using CNN. The inception v3 model was proposed by Kim et al. (2021) for multiple macular disease classification. It classified the retinal image into diabetic retinopathy (DR), retinal vein occlusion (RVO), retinitis pigmentosa (RP), macular hole (MH), rhegmatogenous retinal detachment (RRD), epiretinal membrane (ERM), and neovascular or dry age-related macular degeneration (nAMD or dAMD), it achieved 98.08% accuracy. Muthukannan (2022) proposed an FPOA-CNN approach for multi-classification of ocular diseases, which achieved an accuracy of 98.30%. The proposed two-phase transfer learning-based modified ResNet-50 and DenseUNet model achieved better results than the recent existing models. The multiple retinal diseases are correctly classified, and the detected tumour region is accurately segmented using the proposed approach. The key advantages of the proposed classification network model are that it prevents overfitting and has no detrimental effects on network performance because of the classification process.

The suggested model accurately extracts the essential features corresponding to the diversity of diseases across different scales, enhancing classification performance. He developed a deep learning-based hybrid model to classify DED disease that outperformed the state-of-the-art models. The performance of 99.73% specificity, 99.54% sensitivity and 99.67% accuracy is achieved by the proposed approach. The classification accuracy enabled automatic diagnosis and pre-screening of various DED diseases. The anomaly patterns were recognized effectively by the proposed network model, and discriminative sequences were produced, which assisted in identifying DED diseases. According to the experimental results, the fundus images for classifying various retinal diseases, the proposed two-phase transfer learning approach, are priceless tools for healthcare providers. Figure 9 shows the performance comparison graph for proposed and existing methods.

This research is subject to certain limitations. Two different resolutions for the fundus images in our study were used. The images are pre-processed to the exact resolution of 512 pixels × 512 pixels to reduce the difference and then used for AI learning. Even though the resolutions of the two types of images were identically matched by pre-processing, it is possible that this step did not entirely remove the differences between them. Secondly, while the reading time was reduced by half with the aid of AI technology, the performance did not improve as much. However, it has been demonstrated that this research can aid in the differential diagnosis of many retinal disorders using AI deep learning.