Advancing retinoblastoma detection based on binary arithmetic optimization and integrated features

Dataset and pre-processing

We used images from the American Society of Retina Specialists’ public dataset: http://imagebank.asrs.org. A dataset consisting of 2,048 photographs, 1,024 of which were healthy and 1,024 of which were ill, was acquired after the augmentation procedure. For testing, a dataset composed of 500 photos was utilized. In Dataset 1, the original dataset comprised 256 training samples, 500 test samples, and 500 validation samples, resulting in 1,256 samples. However, after applying data augmentation techniques, the training set was significantly expanded to include 1,024 samples, while the test and validation sets remained unchanged at 500 samples each, resulting in a new total of 2,048 samples. This augmentation process effectively quadrupled the size of the training dataset in Dataset 1.

Dataset 2 MRI images depict the scan’s soft tissue delineation in the orbital and extraocular regions. Hyperreflective and hyperreflective retinoblastoma are shown in Fig. 2. The CT scan reveals the calcified area and nucleates the eye via the giant tumor. The fundus picture of the patients is captured using the Retcam pediatric camera. The planned study would focus on participants aged 12 to 20. Average diagnostic wait times range from zero to seven days. The experts are used to establish a reference point for assessing reality. A total of 243 pictures of pathogenic retinoblastoma are considered in the suggested technique shown in Fig. 1.

Moving onto Dataset 2, the initial dataset consisted of 1,920 training samples, 640 test samples, and 640 validation samples, totaling 3,200 samples. Post-augmentation, the training dataset experienced substantial growth, increasing to an actual 9,600 samples. Meanwhile, the test and validation sets remained unaltered, maintaining 640 samples each. Consequently, Dataset 2’s total sample count swelled to 10,880 samples following data augmentation. This augmentation procedure greatly expanded the size of the training dataset within Dataset 2, facilitating enhanced model training and potentially improving overall model performance.

The presented method for data augmentation is based on the targeted use of texture kernels. Texture kernels are small patches of image data used to generate new images by modifying the existing ones. This technique is used to increase the size of the training dataset and thus improve the model’s performance. The new data generated through this method can be used to train deep learning models and other machine learning algorithms. Eliminating eye-light spots in images is a common problem in computer vision. The Navier–Stokes equation, which describes the motion of fluids, can be used to build a technique for eliminating these spots. Navier–Stokes serves as a form of regularization, as it helps to smooth out the image and reduce noise. This technique can be combined with other methods to improve the overall performance of computer vision models. The concept behind this is to think of the picture as a fluid and then to let this fluid flow to fill the gaps, which are the areas specified on the mask. The morphological operation dilation is first performed to prepare for applying the Navier–Stokes inpainting. This action aims to guarantee that the holes and the black portion of the pupil are directly linked to one another. Consequently, the black pixels can move freely and fill the spaces.

Accurate classification performance relies heavily on pre-processing. Data undergo this process before being sorted into categories. When training a model for classification on the “cancer data set”, pre-processing techniques are crucial. Class classification, image scaling, data augmentation, random cropping, and sliding with the crop are some of the most used pre-processing methods. All photos were scaled to the exact dimensions to ensure the proposed model was trained with consistent results. The cancer data collection underwent data augmentation. Convolutional neural networks (CNNs) generally do better when dealing with large datasets, yet amassing such datasets can be difficult. Convolution neural networks have difficulty evaluating their performance because of an overfitting problem caused by insufficient training data. The best strategy for fixing the problem is to use a data augmentation strategy. The cancer dataset was also pre-processed using a technique called random cropping using convolution neural networks. For the model to be trained appropriately, a vast number of data must be made accessible throughout the process of randomly cropping various regions of high-dimensional pictures.

Image segmentation

There is a great deal of data in the initial image, some irrelevant to the categorization process. They may also be considered background or irrelevant information since they may interfere with the classifier’s forecast. Segmentation, a sort of pre-processing, is used to strip this data from the pictures. The final result of a segmentation process is an image from which irrelevant details have been deleted. Muscles, ink marks, folded/severed tissues, and hazy areas are all background details that need editing. The RGB picture is first converted to HSV to apply hue normalization, and then the hue channel is subjected to multi-level thresholding. After that, the picture is changed back into its original RGB format. K-means clustering segmentation for K = 3 (pink, blue, and white clusters) is applied to the RGB picture. While the pink and blue groups are typical and necessary for feature extraction, the white group is unnecessary and should be discarded. The white collection may be eliminated by selecting the neighboring pixels and coloring them white. The completed product of this process is a picture that has been edited by having its backdrop removed. When using hand-crafted feature extraction approaches, segmentation is crucial. Because DLs can dynamically segment to determine the relevant area in an image, classification is unnecessary when using DLs.

Feature extraction

The Gray-Level Co-occurrence Matrix (GLCM) is widely employed in artificial intelligence and computational imaging for hand-crafted feature extraction. The occurrence of each pair of grayscale values in an image is represented by a matrix called the grayscale locus of the correlation matrix (GLCM). Its purpose is to record a picture’s texture details. The spatial connections between adjacent picture pixels are analyzed to calculate GLCM. The GLCM tracks how often two pixels at a certain distance and orientation occur together. Parameterizing the GLCM calculation enables the management of both space and direction.

After the GLCM has been calculated, several texture characteristics may be derived. Image classification, segmentation, and object identification are just machine-learning tasks that may take advantage of these attributes as inputs.

The formula for calculating the GLCM is as follows:

Let I be an image with M × N, and let d be the offset distance. Let G be the set of gray-level values in the image. The GLCM C at offset (dx,  dy) is defined as: (1)${\displaystyle C\left(i,j\right)=\left\{count\left(m,n\right)inI\mid I\left(m,n\right)\right.}$ (2)${\displaystyle \left.=iandI\left(m+dx,n+dy\right)=j\right\},fori,jinG}$ where count{(m, n) in I∣I(m, n)  =  i and I(m + dx, n + dy)  =  j} is the number of times that a pair of pixels with gray-level values i and j occurs at offset (dx,  dy) in the image.

Once the GLCM is calculated, various statistical measures such as contrast, correlation, energy, and homogeneity can be computed. These can be used as texture features for image classification and other tasks. Numerous fields have found success using GLCM-based texture characteristics, including medical imaging, remote sensing, and industrial inspection. However, it is essential to note that deep learning approaches that directly learn features from the data have essentially supplanted handmade features like GLCM-based ones. The CNN is used as the deep learning model to extract features automatically, as shown in Fig. 3.

We use a hybrid of the two techniques above for each image to jointly identify features, resulting in two sets of feature vectors. The first group comprises feature vectors derived from a combination of HC and CNN feature extraction. A 1,350-element feature vector was constructed. The second collection includes CNN feature and HC feature vectors fused. The resulting feature vector has a size of 2,118 bytes. As a result, each image has access to a massive set of features. However, not all these details factor into the ultimate categorization. Therefore, selecting meaningful characteristics from the extracted features is essential. Both the BAOA-S and BAOA-V are described here.

Binary arithmetic optimization algorithm (BAOA-S/BAOA-V)

The Binary Arithmetic Optimization Algorithm (BAOA) was initially presented to solve optimization issues using binary variables. The BAOA algorithm comes in a standard and a variable form. The BAOA-S algorithm keeps track of potential answers in binary strings. After each cycle, the algorithm applies a set of genetic operators on the current population to develop a new set of possible solutions. The best solutions are then chosen to produce the next population once their fitness has been assessed. This procedure is continued until a stopping condition is reached, such as the maximum number of allowed iterations or a predetermined quality threshold for the result. The BAOA-V method expands the BAOA-S algorithm by supporting binary strings with varying lengths. This is helpful when solving optimization issues where the ideal number of choice variables may change from problem to problem. Optimization issues in fields as diverse as feature selection, machine learning, and engineering design have benefited from both iterations of the BAOA method. Comparisons to other metaheuristic algorithms, such as genetic and particle swarm optimization, have revealed that the approach performs competitively.

In a d-dimensional search space, starting solutions, denoted by $\left[x_m^1,x_m^2,\dots ,x_m^n\right]$, are created at random. (3)${\displaystyle x_m^n=x_{min}^n+s\left(x_{max}^n-x_{min}^n\right)=\left\{1,2,\dots ,M\right\},n=\left\{1,2,\dots ,n\right\}}$ In this formula, M is the total number of individuals, x_m is the index of the mth solution, $x_m^n$ is the index of the nth dimension of the mth solution, $x_{max}^n$ and $x_{min}^n$ are the maximum and minimum values of the jth dimension of the search space, and s is a random number between 0 and 1. Matrix representations of the initial solution Y are also possible, as demonstrated by Eq. (4). (4)${\displaystyle Y=\left[\begin{array}{ccc}\hfill x_1^1\hfill & \hfill \dots \hfill & \hfill x_1^n\hfill \\ \hfill \begin{array}{c}\hfill .\hfill \\ \hfill .\hfill \\ \hfill .\hfill \end{array}\hfill & \hfill \begin{array}{c}\hfill \dots \hfill \\ \hfill \dots \hfill \\ \hfill \dots \hfill \end{array}\hfill & \hfill \begin{array}{c}\hfill .\hfill \\ \hfill .\hfill \\ \hfill .\hfill \end{array}\hfill \\ \hfill x_M^1\hfill & \hfill \dots \hfill & \hfill x_M^n\hfill \end{array}\right]}$ Math Optimizer Accelerated (MOA) function is used to decide between exploration and exploitation. (5)${\displaystyle M\left(L_i\right)=Min+L_i\times \left(\frac{Max-Min}{N_i}\right)}$ L_i represents the current iterations, the maximum number of iteration is denoted by N_i. The lowest and highest values are denoted by Min and Max. We use the dividing and multiplying operators to probe the solution space during this stage. Exploration is carried out by selecting, with equal probability, either the division or multiplication operator. A new answer is computed. (6)${\displaystyle x_m^n\left(L_i+1\right)=\left\{Best\left(x^n\right)\text{\%}\left(M+\in \right)\times \left(\left(x_{max}^n-x_{min}^n\right)\times \mu +x_{min}^n\right),s_2\right\}}$ In this exploitation stage, we are essentially doing a deep search for the ideal answer, or one that is very close to the best possible solution. As a result, operations such as addition and subtraction are utilized. The odds of getting picked as an exploitation operator are the same as during exploration. The new answers are found by, (7)${\displaystyle x_m^n\left(L_i+1\right)=\left\{Best\left(x^n\right)-M\times \left(\left(x_{max}^n-x_{min}^n\right)\times \mu +x_{min}^n\right),s_3\right\}}$

Algorithm 1: Binary Arithmetic Optimization Algorithm (BAOA-S/BAOA-V)

BAOA-S:

1. Initialize the binary string of the input number x and set the iteration count i = 0.

2. Evaluate the fitness function f(x)  andstore it as f_best  =  f(x).

3. Repeat the following steps until the termination condition is met:

   1. Select a random bit position b in x.

   2. Flip the bit at position b to obtain a new x_new.

   3. Evaluate the fitness function f(x_new).

   4. If f(x_new)  >  ff_best,  set x  =  x_new and f_best =  f(x_new).

   5. Increment the iteration count i.

4. Return the optimized binary string x and its corresponding fitness value f_best.

BAOA-V:

1. Initialize the input number  x and set the iteration count i  =  0.

2. Evaluate the fitness function (x) andstore it as f_best  =  f(x).

3. Repeat the following steps until the termination condition is met:

   1. Generate a random vector v with the same length as x.

   2. Generate a new binary x_new by performing a bitwise XOR between x and v.

   3. Evaluate the fitness function f(x_new).

   4. If f(x_new)  >  f_best ,  set x  = x_new and f_best  =  f(x_new).

   5. Increment the iteration count i.

4. Return the optimized x and its corresponding fitness value f_best.

The fitness function f(x) depends on the specific problem being optimized and is not specified in the pseudocode. The termination condition may also be based on a maximum number of iterations, reaching a desired fitness threshold, or other criteria.

Classification

After the completion of the pre-processing processes, the following step is to load the data into the model that has been proposed (EffNet-GRU). This section overviews the EffNet and GRU models and explains the learning parameters for cancer detection and classification.

The analysis module uses a recurrent and convolutional neural network to classify retinoblastoma. The CNN uses a lightweight, accurate model presented by the EfficientNetB0 architecture (Deng et al., 2009). EfficientNetB0 provides the smallest model in the EfficientNet family and can be processed more quickly than larger models. EfficientNetB0 is the best model to use when the model has to rapidly-produce a forecast since the differences in accuracy when upgrading to a higher model are small in our area. To improve the model’s performance, we employ transfer learning (Andayani et al., 2019), in which we re-use weights that have proven successful in the past when applied to the dataset’s object recognition task. Each of the nine blocks that comprise the EfficientNetB0 architecture comprises numerous layers. An overview of this architecture, including its nine significant components, is shown in Fig. 4.

Here, we concentrated on optimizing the correctness of the transferred model by adjusting the number of frozen layers. Four distinct configurations were used during training, each denoted by the number of layers constituting the EfficientNetB0 model. All layers are frozen and not trained from EfficientNetB0 in the no-training mode. Except for Block 9’s pooling, flattening, dense, and dropout layers, all other layers are kept static throughout top training. Layers from the previous MBConv block, Block 8, are likewise trained in the partial version of the model (Block 9). All layers are kept unfrozen during the whole training.

The EffNet model relies heavily on the convolution layer. It is made up of a variety of trainable filters that can extract characteristics from raw data. The layer k , is assumed to be identified as $x_b^c$, whereas the layer c-1, mth has been designated by $x_b^{c-1}$ in the feature map.

Following is the formula for determining $x_b^c$: (8)${\displaystyle x_b^c=\sum _{m\in ij}{a}_{jm}^c\ast x_m^{c-1}+y_j^c}$ where |i| is the total number of mapping features in layer c, $y_j^c$ is a biased term applied to all links leading to the j feature map, and ij is a collection of characteristic maps in layer c − 1 that are associated with unit j. The expression for ReLU Functions activation d(x) is as follows: (9)${\displaystyle d\left(x\right)=max\left(o,x\right)}$ where the range is from 0 to x. Following is a mathematical expression for the max_pooling: (10)${\displaystyle x_{b^{c+1}}{m}^{c+1},s=\begin{array}{c}\hfill max\hfill \\ \hfill 0\le b\le H,0\le m<K\hfill \end{array}{a}_{b^{c+1}}^c\ast H+i,m^{c+1}\ast w+m}$ An initial evaluation was conducted, with each configuration training for 25 epochs before being tested. Based on our analysis of the experimental training accuracy, we determined that top training was the optimal setup. To prevent information loss from earlier layers during training, the model’s weights are frozen at each layer except the last block of layers (which consists of pooling, flattening, dense, and dropout layers). Here, we merge a convolutional neural network (CNN) with a recurrent neural network (RNN) by building a GRU (gated recurrent unit) recurrent neural network on top of the EfficientNetB0 architecture. The GRU network is fed photos and tasked with determining which features disclose whether or not the eye is cancerous. The final classifier receives the GRU’s output and uses dense and dropout layers to assign a value between 0 and 1 to the malignant picture.As previously noted, EfficientNetB0 is implemented in the GRU module of Fig. 4. EfficientNetB0 provides a foundation of information, and the remaining layers are taught to apply that knowledge to classify cancer. The completed network layout is seen in Fig. 5.

GRU is superior to LSTM due to its three main gates and internal cell state. The GRU has the data locked up securely. The reset gate (g) presents the previously known information, whereas the update gate (u) provides historical and future data. Using the reset gate, the current memory gate keeps and remembers crucial data from the preceding computer state. The input modulation gate allows for the introduction of nonlinearity while maintaining the input’s zero-mean features. The mathematical formulation of the basic GRU of static and dynamically modified gates is, (11)${\displaystyle g_t=\sigma \left(A_t.W_{ag}+L_{t-1}.W_{lg}+b_g\right)}$ (12)${\displaystyle y_t=\sigma \left(A_t.W_{ay}+L_{t-1}.W_{ly}+b_y\right)}$ W_lg and W_ly are the weights, b_g and b_y are the bias.

The original picture input geometry was (48, 48, 3), having a height of 48 pixels and a width of 48 pixels in RGB and three channels. The suggested model uses a single convolutional layer to derive characteristics of the input shape, with a 64-shaped feature map as the layer’s final output. In addition, the convolutional layer has a stride of (3 × 3) and a kernel size of 1. While typical layer one padding was maintained throughout the proposed model, variability was reduced by using rectified linear units (ReLU) as the activation function. The first convolutional layer’s output was a shape with (48, 48) sized feature maps, totaling 64 features.

The suggested model’s training process is sped up thanks to the pooling layer, which reduces the training parameter to the (58, 58) size. The training parameters (58, 58, 64) were sent via a dropout layer after the pooling layer to avoid overfitting. To prevent overfitting, the convolution layer was given an initial dropout of 0.5. The training parameter was drastically reduced after each conventional and max-pooling layer, and activation function (ReLU) and dropout were added. A fully connected layer built by flatten with training parameters (42, 42) size and features map (256) must have its input data combined into an ID array once the conventional and max-pooling layers have been trained. Table 1 shows the hyperparameter settings used in the proposed model. The dropout generated 512 feature maps once the convolutional layers procedure was finished. A GRU model was built with a fully linked layer of 512 neurons to address the vanishing gradient issue. The GRU paradigm was followed by the adoption of two ultimately linked layers. The last connected layer also includes SoftMax functions.

The most commonly used method for splitting data randomly is known as random splitting. In this study, the dataset was randomly shuffled to ensure unbiased partitioning. The dataset was then divided into the desired proportions, with 70% allocated for training, 15%, for validation, and the remaining 15% for testing. This approach was employed to maintain the integrity and representativeness of the dataset during the experimentation phase. Libraries such as sci-kit-learn in Python are commonly utilized for random splitting.

The selection of optimization algorithms, learning rate schedules, and convergence criteria is crucial in achieving practical training of machine learning models. The Adam algorithm is a widely used adaptive optimization technique that integrates components from both momentum and RMSprop algorithms. The learning rates are adapted for each parameter individually. The learning rate schedule is a crucial component in the training process as it governs the dynamic changes in the learning rate. The importance of model convergence and stability must be addressed in research. After conducting the ablation study, the learning rate was adjusted to 0.0001.

Based on based on feature extraction

In Automatic Feature Extraction from the data, the results show that this approach achieved an accuracy of 88.5%, which means it correctly classified 88.5% of the instances in the dataset.

Table 4 shows that the results indicate that the model using handcrafted features achieved an accuracy of 86.2%. It performed slightly lower than the automatic feature extraction method in terms of accuracy but still provided reasonable results. The fusion or combination of automated and handcrafted features resulted in a perfect accuracy of 100% , with an F1 score, precision, and recall close to 1.0. An ideal accuracy suggests that the model made no mistakes in its predictions on this dataset.

Based on learning rate

With a learning rate of 1.0, the model achieved an accuracy of 62%. A learning rate 1.0 is typically considered very high and can result in overshooting the optimal parameter values during training, leading to slow convergence and poor generalization. A learning rate of 0.1 is more moderate. In this case, the model’s accuracy improved to 78%. This suggests that a learning rate of 0.1 provided a better balance between convergence speed and accuracy than a learning rate of 1.0. A learning rate of 0.01 is lower than the previous two rates. However, it resulted in a lower accuracy of 70%. This indicates that the learning rate was too low for the specific problem, causing the model to converge slowly and get stuck in local minima. With a further reduction in learning rate to 0.001, the model’s accuracy increased to 83%. A lower learning rate often helps the model converge more precisely towards the global minimum, leading to better results. Finally, a minimal learning rate of 0.0001 resulted in a perfect accuracy of 100%. While this might seem desirable, achieving perfect accuracy in training data can also be a sign of overfitting, where the model has memorized the training data but may need to generalize better to unseen data.

Based on dropout rate

In Table 5, each row corresponds to a specific dropout rate, and the corresponding accuracy achieved during model training is indicated as a percentage. It is worth noting that when the dropout rate is set to 0.5, the model achieved a perfect accuracy of 100%, suggesting that this particular dropout configuration worked well for this dataset and model architecture. Dropout is a regularization technique commonly used to prevent overfitting in neural networks by randomly dropping out (setting to zero) a fraction of the neurons during training. The optimal dropout rate can vary depending on the dataset and model, and it is often determined through experimentation and hyperparameter tuning.