A novel deep learning based approach with hyperparameter selection using grey wolf optimization for leukemia classification and hematologic malignancy detection

Dataset collection and preprocessing

In this research, we employed the leukemia dataset for experimental purposes. The selected dataset is publically available at https://www.kaggle.com/datasets/andrewmvd/leukemia-classification. The dataset is collected from 118 patients, and it has two classes, including normal cells and leukemia blast. The normal class contains 5,531 microscopic images, and the leukemia blast consists of 5,530 microscopic images. Each sample in the dataset has the dimension of $224\times 224\times 3$. The information on images is not clear enough to perform accurate classification. Therefore, we proposed a new technique to improve the contrast of input images. The proposed method starts by applying a hyperbolic sin function to provide a simple modification. The function is mathematically formulated as:

(1) $$\int \sinh =\frac{e^{f(x,y)}-e^{-f(x,y)}}{2}$$where $f(x,y)$ represents the low contrast input image, and $\int \sinh$ denotes the output of the hyperbolic sine function. After that, the power law function is performed on the resultant image. The power law is defined as:

(2) $${\mathrm{\partial }}_p=\rho \cdot {(\int \sinh )}^{\alpha }$$where ${\mathrm{\partial }}_p$ denotes the resultant image of the power law, $\rho$ represents the controlling parameter of brightness, and $\alpha$ controls the enhancement in the contrast. Following that, the hazing layer is removed by employing the dehazing function. The mathematical equation of dehazing is:

(3) $${\varphi }_h=\frac{{\mathrm{\partial }}_p-{\omega }_A}{{\epsilon }_t}+{\omega }_A$$where ${\varphi }_h$ denotes the output image of dehazing, ${\omega }_A$ represents the estimation of atmospheric light, and ${\epsilon }_t$ is the estimation of the transmission map. In the last phase, the resultant image is passed through the contrast stretching function to remap the pixels into their dynamic range. The function is described as:

(4) $$F_{\mathrm{o}\mathrm{u}\mathrm{t}\mathrm{p}\mathrm{u}\mathrm{t}}=\frac{{\varphi }_h-min({\varphi }_h)}{max({\varphi }_h)-min({\varphi }_h)}$$where $max({\varphi }_h)$ and $min({\varphi }_h)$ represent the maximum and minimum intensity values, and $F_{\mathrm{o}\mathrm{u}\mathrm{t}\mathrm{p}\mathrm{u}\mathrm{t}}$ is the final output of the proposed algorithm, which is visually presented in Fig. 1.

Proposed parallel inverted dual self-attention network (PIDSAN4)

CNNs are the baseline architecture for numerous computer vision tasks, including image classification, object detection, and segmentation. CNNs are structured to automatically and adaptively learn different spatial hierarchies of features from the input; the fundamental property in the standard architecture of a CNN is the convolutions; the convolutions are applied to the input via multiple filters which capture the local patterns in the input like edges, corners, textures, etc. Activation functions such as rectified linear unit (ReLU) are applied after the convolutions to introduce non-linearity to the model. The convolutional layers are typically followed by pooling layers which help to downsample the spatial resolution of the input in order to reduce computational costs, as well as fully connected layers which help the model to reason about high level representations for classification. Although CNNs can extract features locally, CNNs are limited when it comes to modelling the long-range dependencies and global contextual features. To address this, hybrid models that integrate CNNs with self-attention techniques. Hybrid models seek to capitalize on the local feature extraction capabilities of CNNs, alongside the global modelling capabilities of self-attention.

Pre-trained models can be an efficient and time-saving solution to many problems. Still, they also come with a certain set of limitations that disqualify their use in some applications. Such as, they have high numbers of parameters, translating into excessive computational resource requirements and memory usage that make them harder for devices with limited hardware capabilities, unable to effectively learn long-term dependencies in data, as they are usually intended to solve generic tasks like object recognition, and fixed architectures can limit the adaptability to learn unique features. Therefore, it is necessary to optimally design CNNs to realize these desired features to solve specific tasks. In the work, we designed a lightweight hybrid model based on parallel inverted residual blocks integrated with self-attention mechanism named with PIDSAN4. The purpose behind designing the PDSAN4 is to learn the irregular shape and large nuclei, as in leukemia cells; the nucleoli are prominent compared to normal cells. The proposed model comprises 107 layers and a total of 5.7 million parameters.

Input and initial layers: The network starts with the input size of $224\times 224\times 3$. Initially, two convolutional layers are attached with the configurations of $3\times 3$ kernel size, $2\times 2$, $1\times 1$ stride, and 32, 64 number of channels respectively.

Parallel inverted bottleneck block: After initial layers, inverted residual block has been employed with parallel branches. Each block contains one expansion convolutional configured with $1\times 1$ filter size, 64 depth, and $1\times 1$ stride, two batch normalization layers, swish activation, and one depth-wise convolutional operation with $3\times 3$ filter size, 64 group channels, and $1\times 1$ stride settings, one projection convolutional operation with $1\times 1$ filter size, 64 depth, and $1\times 1$ stride. A skip connection is established among the expansion to projection convolutional operations. The mathematical formulation of this block is:

(5) $${\varphi }_{\mathrm{e}\mathrm{x}\mathrm{p}}=\phi (T\times {\omega }_{\mathrm{e}\mathrm{x}\mathrm{p}}),{\omega }_{\mathrm{e}\mathrm{x}\mathrm{p}}\in {\mathbb{R}}^{1\times 1\times D\times D_{\mathrm{e}\mathrm{x}\mathrm{p}}}$$

(6) $$\phi (k)=k\cdot \sigma (k)$$

(7) $${\varphi }_d=\phi ({\varphi }_{\mathrm{e}\mathrm{x}\mathrm{p}}\times {\omega }_d),{\omega }_d\in {\mathbb{R}}^{3\times 3\times D_{\mathrm{e}\mathrm{x}\mathrm{p}}\times 1}$$

(8) $${\varphi }_p={\varphi }_d\times {\omega }_p,{\omega }_p\in {\mathbb{R}}^{1\times 1\times D_{\mathrm{e}\mathrm{x}\mathrm{p}}\times D_o}$$

(9) $${\varphi }_{\mathrm{s}\mathrm{k}\mathrm{i}\mathrm{p}}=T+{\varphi }_p$$where ${\varphi }_{\mathrm{e}\mathrm{x}\mathrm{p}}$ represents the expansion operation, $\phi$ is swish activation, ${\varphi }_d$, ${\varphi }_p$ represent the depthwise and projection operations, and ${\varphi }_{\mathrm{s}\mathrm{k}\mathrm{i}\mathrm{p}}$ is the skip connection. Intermediate Layers: After each parallel inverted bottleneck, max pooling activation is employed to downsample the spatial dimension with swish activation function.

(10) $${\varphi }_{⊞}=\phi (\mathrm{m}\mathrm{a}\mathrm{x}({\varphi }_{\mathrm{s}\mathrm{k}\mathrm{i}\mathrm{p}}))$$

Patch embedding and dual attention layers: After the fourth intermediate layers, the patch embedding layer is employed to convert 2D tensor into 1D sequence of patches. The number of patches is 16. After that, the patches are passed to the dual attention layers. The first self-attention layer is applied along the spatial features and second attention is employed on channel attention. The mathematical formulation is:

(11) $${\varphi }_{⊞}=\{p_1,p_2,p_3,\dots ,p_{16}\}$$

(12) $$p_i=\mathrm{F}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}(P_i)$$

(13) $$p_i=\omega \cdot p_i+b$$

(14) $${\varphi }_E=[p_1@p_2@\dots @p_{16}]\in {\mathbb{R}}^{N\times D}$$

(15) $${\varphi }_{\mathrm{a}\mathrm{t}\mathrm{t}}^s=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(\frac{Q{K}^T}{\sqrt{d_k}}\right)V$$

(16) $${\varphi }_{\mathrm{a}\mathrm{t}\mathrm{t}}^{\mathrm{\top }}=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(\frac{Q_D{K}_D^T}{\sqrt{d_k}}\right)V_D$$where ${\varphi }_{⊞}$ represents the max pooling operation, ${\varphi }_E$ is the patch embedding and ${\varphi }_{\mathrm{a}\mathrm{t}\mathrm{t}}^s$, ${\varphi }_{\mathrm{a}\mathrm{t}\mathrm{t}}^{\mathrm{\top }}$ denote the spatial attention and channel attention operations. In the final step, the dual attention is presented as:

(17) $${\varphi }_{\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l}}={\varphi }_E+\alpha \cdot {\varphi }_{\mathrm{a}\mathrm{t}\mathrm{t}}^s+\beta \cdot {\varphi }_{\mathrm{a}\mathrm{t}\mathrm{t}}^{\mathrm{\top }}$$where $\alpha$ and $\beta$ are learnable parameters with value of 0.5 for equal weights.

Final layers: After the dual attention mechanism, a 1-D global average pooling layer is utilized, followed by a new fully connected layer, a new softmax layer, and a classification layer at the end for class prediction. The loss function of the proposed model is categorical cross-entropy to minimize the training loss, defined as:

(18) $$\mathcal{L}=-\sum _{i=1}^N{y}_i\log ({\hat{y}}_i)$$

(19) $${\psi }_{\mathrm{L}\mathrm{o}\mathrm{s}\mathrm{s}}=-\sum _{\alpha =1}^N\sum _{\beta =1}^C{O}_{\alpha \beta }\log (P_{\alpha \beta })$$where N and C represent the number of samples and classes, respectively. $O_{\alpha \beta }$ denotes the $\alpha$-th sample belonging to the $\beta$-th class, and $P_{\alpha \beta }$ is the predicted probability output. Figure 2 presents the proposed model architecture.

Modified vision transformer

ViT, a deep learning architecture, draw inspiration from the Transformer model, initially designed for text data within natural language processing (NLP) settings, to tackle image processing tasks. ViT is selected to take advantage of its long-range dependency processing features and flexibility in handling different features, making it suitable for identifying complex and subtle features of leukemia and normal cells. The tiny16 variant is employed in this work due to its lightweight complexity and computational cost.

The proposed ViT is briefly described as follows: Suppose $T=\{(A_i,b_i){\}}_{i=1}^s$ is a set of $s$ images, where $A_i$ denotes a sample image and $b_i$ denotes the corresponding class label of that image, $b_i\in \{1,2,\dots ,n\}$, where $n$ represents the number of classes in set T (Bazi et al., 2021). The ViT model aims to learn a mapping from patterns of image patches to their semantic labels. The architecture of ViT is inspired by the vanilla Transformer, which follows the encoder-decoder architecture, capable of processing sequential data independently of recurrent networks (Vaswani, 2017; Holliday, Sani & Willett, 2018).

The self-attention mechanism is a key strength of the transformer model, as it acquires relationships between elements in long-range sequences. ViT extends transformers for image classification. The attention mechanism enables ViT to process different regions of an image and concatenate information obtained from the entire image sequence. The generic architecture of the ViT model comprises multiple layers, including an embedding layer, an encoder, and a classification layer.

Initially, a sample image $A_i$ is split into patches, each containing non-overlapping information of the image. Each patch is considered a token by the transformer. For an image $A_i$ of size $(d\times t\times w)$, where $d$ is the depth of the channels, $t$ is the height, and $w$ is the width, patches of dimension $(d\times p\times p)$ are extracted. A sequence of image patches is formulated as $(a_1,a_2,\dots ,a_m)$ of length $m$, where $m=\frac{t\cdot w}{p^2}$. Commonly, a patch size of $16\times 16$ or $32\times 32$ is used. Smaller patch sizes increase the sequence length and vice versa.

The created patches are fed into the encoder after being linearly projected into a vector space of dimension $k$ through a learned embedding matrix F. These embeddings are concatenated with a learnable classification token $w_{\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}}$, whose purpose is to perform classification. The transformer visualizes the embedded patches as a set of unordered patches. To preserve spatial information, patch positions are encoded and added to the representation of the patches. The sequence of embedded patches with positional encoding is given by:

(20) $$x_0=[w_{\mathrm{c}\mathrm{l}\mathrm{a}\mathrm{s}\mathrm{s}};a_{1}F;a_{2}F;\dots ;a_{m}F]+F_{\mathrm{p}\mathrm{o}\mathrm{s}},F\in {\mathbb{R}}^{(p^{2}d)\times k},F_{\mathrm{p}\mathrm{o}\mathrm{s}}\in {\mathbb{R}}^{(m+1)\times k}.$$Here a 1-D positional encoding scheme is utilized to preserve patch positions (Dosovitskiy et al., 2020). The sequence $x_0$ is fed into the transformer encoder. The encoder comprises identical layers I, where each layer has two sub-components: a multi-head self-attention (MSA) block and a fully connected feed-forward dense block (MLP). The MLP block consists of two dense layers with a GeLU activation function. Both sub-components use skip connections followed by a normalization layer (LN). The operations are defined as:

(21) $${x^{\prime }}_f=\mathrm{M}\mathrm{S}\mathrm{A}(\mathrm{L}\mathrm{N}(x_{f-1}))+x_{f-1},f=1,\dots ,I$$

(22) $$x_f=\mathrm{M}\mathrm{L}\mathrm{P}(\mathrm{L}\mathrm{N}({x^{\prime }}_f))+{x^{\prime }}_f,f=1,\dots ,I.$$

In the encoder, the first element from the final layer $x_I^o$ is passed to the classifier for label prediction:

(23) $$b=\mathrm{L}\mathrm{N}(x_I^o).$$

The MSA block is a crucial component, determining the importance of individual patches relative to others in the sequence. MSA computes query (C), key (E), and value (Y) matrices using learned weights $M_{CEY}$. These are used to calculate attention scores:

(24) $$[C,E,Y]=x{M}_{CEY},M_{CEY}\in {\mathbb{R}}^{k\times 3D_E}$$

(25) $$A=\mathrm{s}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(\frac{C{E}^T}{\sqrt{D_E}}\right),A\in {\mathbb{R}}^{m\times m}$$

(26) $$\mathrm{S}\mathrm{A}(\mathrm{x})=\mathrm{A}\cdot \mathrm{Y}.$$

The MSA block concatenates outputs from all attention heads and passes them through a feed-forward layer with learnable weights:

(27) $$\mathrm{M}\mathrm{S}\mathrm{A}(x)=\mathrm{C}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{a}\mathrm{t}({\mathrm{S}\mathrm{A}}_1(x);{\mathrm{S}\mathrm{A}}_2(x);\dots ;{\mathrm{S}\mathrm{A}}_t(x))W,W\in {\mathbb{R}}^{t{D}_E\times D}.$$In this work, a tiny16 vision transformer is utilized. The last three layers (indexing, fully connected, and softmax) are replaced with global average pooling, a new fully connected layer, and a new softmax layer for transfer learning. The modified tiny16 model has three heads and $N\times 192$ hidden dimensions. The architecture is visually presented in Fig. 3.

Hyperparameter tuning using grey wolf optimization

The grey wolf optimizer (GWO) (Bazi et al., 2021) was inspired by the social behaviour of grey wolves, the leadership hierarchy, and pursuit of property in the group. Within their natural habitat, grey wolves typically inhabit groups. The range of group sizes is from five to twelve. A strict social dominance hierarchy is maintained. The most prominent male or female wolves are placed at the apex of the hierarchy as alphas. These individuals are primarily tasked with making decisions regarding the wolf pack’s habitat, foraging, resting, and feeding of the wolf pack. Every other pack follows the dominant canines. Beta wolves are the subsequent level of the alpha pack; they execute the directives and exercise dominion over the lower-level wolves. Delta wolves assist alpha and beta wolves in pursuing and investigating prey. They patrol the boundaries of the territory, communicate potential threats to other wolves, and take care of the needs of wounded or vulnerable individuals. Omegas are the lowest category of wolves, which exist outside the hierarchy of other canines. Their social hierarchy predominantly determines wolves’ foraging effectiveness. The mathematical modelling of grey wolf social behaviour involves identifying the optimal solution for the prey location and representing the wolf’s position in the search space as the solution. Alpha wolves are the most optimal solution, given their proximity to the prey. According to their social hierarchies, the beta and delta wolves represent the next best solution. In space, omega wolves adjust their position during the search process based on the whereabouts of alpha, beta, and delta wolves. Suppose that the position in the space of alpha, beta, delta, and omega is denoted by ${\omega }_{\alpha }$, ${\omega }_{\beta }$, ${\omega }_{\delta }$, and ${\omega }_{\gamma }$. Prey surrounding, hunting, attacking, and seeking are essential steps in the GWO. The surrounding is a process by which wolves encircle the prey during hunting time, which is a mathematically formulated equation:

(28) $${\mathrm{\Phi }}_D=\left|{\mathbf{C}}_{\mathbf{1}}\cdot {\omega }_{\rho }(t)-\omega (t)\right|$$

(29) $$\omega (t+1)={\omega }_{\rho }(t)-{\mathbf{C}}_{\mathbf{2}}\cdot {\mathrm{\Phi }}_D$$

(30) $${\mathbf{C}}_{\mathbf{1}}=2\cdot \tau \cdot {\mathbf{R}}_{\mathbf{1}}-\tau$$

(31) $${\mathbf{C}}_{\mathbf{2}}=2\cdot {\mathbf{R}}_{\mathbf{2}}$$where ${\mathbf{C}}_{\mathbf{1}}$ and ${\mathbf{C}}_{\mathbf{2}}$ are the two coefficient vectors, $\omega (t)$ and ${\omega }_{\rho }(t)$ represent the position vector of wolves and prey at the current iteration. ${\mathbf{R}}_{\mathbf{1}}$ and ${\mathbf{R}}_{\mathbf{2}}$ are two random matrices with the range of $[0,1]$, and $\tau$ is a matrix whose values decrease over the iteration from $2$ to $0$. Alpha wolves perform the role of guides during the prey stalking process. Delta and beta wolves are also implicated in this process. It is postulated that these three canines possess knowledge regarding the probable location where sustenance could be encountered. This information facilitates the determination of the three most efficient search agents, which are subsequently utilized to update the coordinates of other wolves, as represented by equations:

(32) $${\mathrm{\Phi }}_{\alpha }=\left|{\mathbf{C}}_{\mathbf{1}}\cdot {\omega }_{\alpha }-\omega \right|$$

(33) $${\mathrm{\Phi }}_{\beta }=\left|{\mathbf{C}}_{\mathbf{1}}\cdot {\omega }_{\beta }-\omega \right|$$

(34) $${\mathrm{\Phi }}_{\delta }=\left|{\mathbf{C}}_{\mathbf{1}}\cdot {\omega }_{\delta }-\omega \right|$$

(35) $${\omega }_1={\omega }_{\alpha }-{\mathbf{T}}_{\mathbf{1}}\cdot {\mathrm{\Phi }}_{\alpha }$$

(36) $${\omega }_2={\omega }_{\beta }-{\mathbf{T}}_{\mathbf{2}}\cdot {\mathrm{\Phi }}_{\beta }$$

(37) $${\omega }_3={\omega }_{\delta }-{\mathbf{T}}_{\mathbf{3}}\cdot {\mathrm{\Phi }}_{\delta }$$

(38) $$\omega (t+1)=\frac{{\omega }_1+{\omega }_2+{\omega }_3}{3}.$$

The attacking phase is equivalent to exploitation and is implemented by the factor $\tau$. When the prey ceases to move, the predatory wolves launch an assault on the defenseless prey. The value of $\mathbf{T}$ is a randomly selected value within the range $[2r,2r]$, where $r$ is within the range $[-1,1]$. The process of seeking or exploring the most optimal solution is related to the rising behavior of wolves. Wolves converge after locating prey after diverging in pursuing it. If $|T|>1$, wolves diverge in search of superior prey; otherwise, if $|T|<1$, they converge towards the prey. A random C is used to prevent local optima and promote exploration. The method produces random values at the beginning and end stages, promoting impartial investigation. In this research, the proposed models’ hyperparameters are considered optimization problems. The objective is to determine the optimal hyperparameters of models using the GWO algorithm. The architecture of the proposed model is mathematically described as:

(39) $${\varphi }_{\mathrm{a}\mathrm{c}\mathrm{c}}=\mathrm{N}\mathrm{e}\mathrm{t}\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{k}\left({\psi }_{\mathrm{H}\mathrm{p}},{\mathbf{\text{ω}}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{s}},{\mathrm{\Psi }}_{\mathrm{T}\mathrm{D}}\right)$$

(40) $${\rho }_{\mathrm{A}\mathrm{c}\mathrm{c}}^{max}=\mathrm{N}\mathrm{e}\mathrm{t}\mathrm{w}\mathrm{o}\mathrm{r}\mathrm{k}\left({\psi }_{\mathrm{H}\mathrm{p}},{\mathbf{\text{ω}}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{s}},{\mathrm{\Psi }}_{\mathrm{T}\mathrm{D}}\right),\kappa <{\kappa }^{max}$$where ${\psi }_{\mathrm{H}\mathrm{p}}$ denotes the list of hyperparameters, ${\mathbf{\text{ω}}}_{\mathrm{w}\mathrm{e}\mathrm{i}\mathrm{g}\mathrm{h}\mathrm{t}\mathrm{s}}$ represents the weights of the proposed network, and ${\mathrm{\Psi }}_{\mathrm{T}\mathrm{D}}$ denotes the training dataset. The objective function is to maximize the accuracy of the network for hyperparameters, as mathematically formulated in Eq. (40). In this work, we employed the grey wolf optimization to tune the proposed models during the training process. the ranges of hyperparameters are described in Table 1.

In this work, we also trained all selected models on static hyperparameters, and the static hyperparameters are learning rate, mini-batch size, epochs, and optimizer having values of 0.001, 16, 200, and SGDM. The proposed PIDSAN4 and tiny ViT models have significantly improved accuracy when the hyperparameters are tuned. The proposed PIDSAN2 improved with 1.9% accuracy, and the tiny ViT improved by 3.00% accuracy, as shown in Fig. 4. The GWO was selected for the hyperparameters tuning due to its balance among the exploration and exploitation, computational efficiency, which aligned well with the lightweight nature of our models. While, other alogrithms such as particle swarm optimization (PSO), firefly optimization, tree growth optimization (TGO) has also outperformed but GWO has consistently showed competitive performance in hyperparameters tuning in recent studies. the process of GWO for tuning are shown in Fig. 5.

Training process

The selected dataset is divided into multiple ratios: 50:40:10, 60:30:10, and 70:20:10. The 50%, 60%, and 70% of the dataset are utilized for the training process in various experiments, while 40%, 30%, and 20% of data are employed for testing purposes. A total of 10% of the data is used for validation. After analyzing the dataset, it is observed that the selected dataset has an imbalance problem, To address this issue and attain better generalization, we implemented a class weighting method during the training process. Weights are computed inversely proportional to the class occurrences, ensuring that the model gives more consideration to the underrepresented class. The weights are calculated as:

(41) $$C_{\omega }=\frac{T_s}{C_N\times C_s}$$where $C_N$ is the number of classes, $T_s$ is the total count of samples, $C_s$ represents the class samples, and $C_{\omega }$ is the class weight. The calculated weights are then passed into the loss function to reduce the bias towards the majority class during the learning process.

After designing the proposed models, the proposed models are trained using the training data, as shown in Fig. 6. The hyperparameters for training are selected based on the grey wolf optimization. After training, the trained models are employed for the testing phase.

Ablation study

After optimizing the hyperparameters, all the selected models are trained using various optimizers with the same configurations, as shown in Table 4. According to this table, it is observed that the proposed models achieved higher accuracy with the optimizer of SGDM and the learning rate of $1.22\times {10}^{-3}$. The training time of all the selected models is also noted, and it is noted that InceptionV3 has the highest training time, which is 13.53 h. The proposed model completed its training within 7.98 h. The second-lowest training time is 8.57 h, obtained by the MobileNetV2 model. Three experiments were performed using three ratios of data: 50:40:10, 60:30:10, and 70:20:10. The 50%, 60%, and 70% of the dataset are utilized for the training process in various experiments, while 40%, 30%, and 20% of data are employed for testing purposes, and 10% of data is used for validation. Figure 8 highlights the relationship between data splits and model performance, raising issues of underfitting, overfitting, and generalization considerations. Models such as Tiny16 ViT and the proposed PIDSAN4 show consistent performance improvements as training data increases, showing better generalization capabilities. Older architectures such as ResNet50 and VGG19 show lower precision and scalability with more training data, suggesting that they may be inadequate due to their limited ability to learn complex patterns from data sets. For the split of 50:40:10, lower training data may result in inadequate adaptation for most models, as shown in the modest accuracy values, except PIDSAN4 and Tiny16 ViT, which still achieve high performance. As the distribution increases to 70:20:10, most models improve accuracy. Still, the distance between PIDSAN4/Tiny16 ViT and other models increases, indicating that older models may be too suitable or unable to generalize due to their weak adaptability to larger data sets. The high performance of PIDSAN4 and Tiny16 ViT in all partitions shows that these models effectively balance generalization based on modern architectural advances.

A comprehensive comparison of proposed models and the state-of-the-art models has been conducted in this section. Table 5 illustrates the comparison based on the number of layers, number of parameters, and size of the models. From this table, Inception V3 has 316 total layers with 23.9 million parameters and 23.9 MB in size. The ResNet50 and MobileNetv2 have 177 and 154 layers with 25.6 and 3.5M parameters, respectively. Both models are 96 and 13 MB in size. VGG19 has 47 layers with 143.6M parameters and 535 MB in size. The proposed PIDSAN4 model has 107 layers with 7.5M parameters and 16.4 MB in size. The other proposed model, Tiny ViT, has 143 layers with 5.7M parameters and only 22.6 MB in size. As a result, concerning parameters, the proposed model is much more lightweight than InceptionV3, ResNet50, and VGG19. The receiver operating characteristic (ROC) curve shows all six of the models. InceptionV3, ResNet50, MobileNetV2, VGG19, Modified ViT, and PIDSAN4. The ROC curve plots the Sensitivity against the False Positive Rate for the model. The three models that performed the best in the evaluation and were most effective were InceptionV3, Modified ViT, and PIDSAN4 with an area under the curve (AUC) of 0.91, indicating that they are excelling at class discrimination. VGG19 was close behind with an AUC of 0.90 indicating it also performed strongly. MobileNetV2 and ResNet50 performed comparatively weaker than their peers with an AUC of 0.87 and 0.86 respectively suggesting that they are slightly less effective than others for the purpose of demonstrating class discrimination ability for this evaluation setting. The ROC is presented in Fig. 9.

Table 6 shows a comprehensive comparison of various ViTs and hybrid models using five relevant classification metrics of precision, recall, F1-score, specificity, and accuracy. The proposed PIDSAN4 model which is based on hybrid architecture that combines CNN with self-attention, showed more balance and superior performance across all metrics. It had the highest F1-score (0.894), which indicates the best precision and recall tradeoff. It is also superior to the other models in specificity (0.925) and accuracy (0.913). This shows its ability to correctly classify both the positive and negative samples, which is especially salient for maintaining low false positive rates in a high consequence classification task. The modified TinyViT made no changes to a pure ViT architecture, showed the highest precision (0.889) and recall (0.897) which show its strength in correctly identifying true positive samples. However, while it demonstrated significant sensitivity in detection, it shows a somewhat weaker F1-score (0.881) and accuracy (0.901) compared to that of PIDSAN4 which indicates a slight imbalance in false positive and false negative handling. While a slight imbalance should not affect its reliability, it may have relevance in some uses where consistent performance across classes is pertinent. The ConvNeXt-t and CrossViT-s models are CNN-Transformer hybrid approaches demonstrated F1-scores nearly identical at 0.872. While they both showed reasonable performance comparably, both also underperform compared to PIDSAN4 in specificity and accuracy. DaViT-S, which showcased dual attention mechanisms, showed the lowest specificity (0.877) and accuracy (0.882) of any of the models, therefore limiting its ability to correctly identify negatives or reject potential negative instances. Larger proportion of false negatives may induce higher levels of false alarms in potential real-world applications.

Table 7 describes a useful comparison of a number of metaheuristic optimization methods for hyperparameter tuning and their effect on the performance of the proposed PIDSAN4 model. The results show that a properly tuned hyperparameter tuning approach utilizing metaheuristics can provide some performance benefits regarding classification performance. Compared to the without tuned performance of the model accuracy is 0.881, F1-score is 0.891, the correctly tuned options optimize training dynamics that allow the classifiers to refine predictive accuracy and generalization. The GWO method is by far the best evaluated optimizer with respect to the metrics that are used. It have both the highest F1-score (0.894) and highest accuracy (0.913). The GWO must have performed accordingly because it is able to search the search space, and find optimal combinations of hyperparameters to use while training the networks in a way that improved their learning. The chimp optimizer provided strong accuracy which is 0.879, F1-score is 0.868 performance, followed closely by firefly optimizer accuracy (0.881). While the TGO ACO are effective in terms of performance improvements, they did not provide as much performance benefit as GWO and firefly, which indicates GWO and Firefly have better search features compared to the ACO and TGO. TGO performed worse than the baseline.

Figure 10 demonstrates the comparative performance of the PIDSAN4 model and the Modified ViT model while they both operated under varying intensities of image noise with the purpose of examining the performance of each model across three levels of Gaussian noise (1%, 3% and 5%). The performance results demonstrated that as the noise intensity increased the overall performance for each model would gradually decrease; however, the PIDSAN4 model maintained a better performance under all levels of noise. At noise level 1%, the PIDSAN4 model demonstrated the highest performance with an accuracy of 90.8% and an F1-score of 88.9%. Comparatively, the modified ViT recorded an accuracy of 89.4% and F1-score of 87.1% for the model performance. When the noise intensity level is raised to a level of 3%, the PIDSAN4 model again maintained a better performance with a model accuracy of 89.2% and an F1-score of 87.4% which compared to the modified ViT’s accuracy of 87.9% and F1-score of 85.3% respectively. By noise level 5% the PIDSAN4 model still have relatively better performance with an accuracy of 87.1% and F1-score of 86.6% while the modified ViT have an accuracy of 85.4% and F1-score of 83.3%.

Discussion

The results clearly show that the proposed PIDSAN4 and modified Tiny ViT models are more efficient on leukemia cancer datasets than state-of-the-art architectures such as InceptionV3, ResNet50, MobileNetV2, and VGG19. In particular, the PIDSAN4 model achieved the highest precision of 91.3% and a balanced sensitivity (89.2%), specificity (92.5%), and precision (83.8%). Its superior measurement demonstrates its generalization capability, as evidenced by the confusion matrix, in which it accurately classified 92.84% of leukemia blast samples and 91.29% of normal cells. These results demonstrate the network’s ability to effectively learn complex patterns, minimize classification errors, and maintain balance between both classes.

The modified Tiny ViT achieves slightly reduced accuracy 0.901 compared to InceptionV3 0.909, however, its inclusion in our investigation was as part of an intentional tradeoff between performance, computation, and interpretability in model building. The Tiny ViT is much smaller which has 5.7M parameters compared to InceptionV3 which is 23.9M and requires less time to train 12.41 vs. 13.53 h. This compactness means it is more appropriate for real-time use in less resource-rich environments, such as portable diagnostic devices and clinics. A transformer model enhances explainability since attention has been shown to help understand decision-making in some cases and we used LIME visualization to understand the decisions made by the model as shown in Fig. 8. Our framework contributes to model development, including Tiny ViT for model evaluation, and the view is not that Tiny ViT replaces a standard CNN, such as the InceptionV3 model, but can contribute to a dual-model strategy whereby models based on different architecture types can enable even more features and provide a strong basis for exploration in clinical applications.

Using modern architectural advances in AI models, such as including lightweight layers and efficient feature extraction mechanisms, the PIDSAN4 network has overcome heavy architectures such as VGG19, which suffers from overcomplicated fitting due to its large parameter sizes and limited scale. PIDSAN4 and Tiny ViT models offer superior accuracy and computational efficiency, making them suitable for integration into medical diagnostic workflows. Their lightweight architecture can be deployed in resource-limited environments, such as portable diagnostics and hospitals with limited computational resources. Balanced classification performance minimizes critical errors such as false negatives (false classification of leukemia cells as normal), reducing the risk of missed diagnosis. This high reliability and adaptability can improve early leukemia detection, support clinical decisions, and personalized treatment planning.

Figure 11 depicts LIME visualizations for five randomly chosen samples processed by the proposed Tiny Vision Transformer (Tiny16), and the PIDSAN4 model. The process of LIME consists of perturbing the input image and measuring how the prediction changes. It then fits a simple interpretable model locally around each sample to model the contribution of relevant image regions to a final decision. In these visualizations, superpixels such as small colored areas are those areas of the image that LIME estimated contributed most to a prediction decision made by the model. The warmer colors such as red, yellow, orange indicate the image regions that highly support a predicted class and the cooler colors like blue, green are areas of less and even negative influence. From the top row, the Tiny ViT model appeared to distribute attention more widely across the image area and focus attention on several, dispersed areas. This could be related to the Tiny ViT ability to apply long-range dependencies, and to attend to a global context and certainly less local attention than the PIDSAN4 model was able to apply. The PIDSAN4 model showed more local and focused attention across the same areas in the image. The LIME maps of the PIDSAN4 model showed that the regions that consistently highlighted the core nuclear regions and showed the red and yellow regions of attention. This consistent application of focus by the PIDSAN4 model reflects the model’s reliance on these specific morphological features of leukemic cells such as enlarged nuclei, increased chromatin density.