Hybrid transformer-CNN and LSTM model for lung disease segmentation and classification

Improved Transformer-based CNN model

In this improved Transformer-based CNN model (Fig. 2), segmentation on the pre-processed image is done by performing a feature retrieval technique in its blocks.

Generally, this model comprises three blocks: Encoder, Decoder, and Skip connection (Sun, Pang & Li, 2023). Here, the encoder block in the improved Transformer-based CNN model consists of two parallel branches: (i) Transformer branch and (ii) CNN branch (Yang et al., 2023a; Peng, Zhang & Guo, 2023). Each branch consists of a specific set of blocks, i.e., the Transformer branch has improved Swin Transformer blocks, and the CNN branch has Efficient Net blocks while these blocks perform the same process, ‘feature retrieval. The features retrieved by these blocks are then forwarded to the Swin Transformer and CNN Fusion Module (STCF), which performs convolutional operations to fuse those features. Finally, the resultant fused feature maps are obtained from each STCF module, which is then given to the decode block and skip connection to perform segmentation. Therefore, this model initially shows the pre-processed image as input into the Transformer and CNN branches. The improved Swin Transformer (IST) block performs feature retrieval in the Transformer branch on pre-processed images. Likewise, the CNN branch performs feature retrieval on the pre-processed images by Efficient Net block.

Texture features

Generally, the features of texture define the various characteristics or attributes extracted from an image, which specifically indicate the spatial patterns and structures present within the segmented image, $S^k$.The significant role of extracting texture features from segmented images denotes its informative data and defines and differentiates the diseased and non-diseased regions within it. The two types of techniques employed in the extraction of texture features are,

• 1)

  Extraction of Texture features using modified LGIP

• 2)

  Extraction of Texture features using multi-texton.

1. Modified LGIP-based texture feature extraction

Generally, LGIP is a familiar local texture feature-extracting technique that explicitly captures information about the local gradients within an image. Likewise, Local Binary Pattern (LBP) is another local texture feature extraction method that captures an image’s local structure by comparing its pixel intensity values in a neighbourhood around each centre pixel. These two local texture feature-extracting techniques are mainly used individually in existing research. These two techniques are combined and formed as a modified LGIP. The procedural process involved in the modified LGIP technique using improved LBP is shown in the sections below.

(i) Improved LBP in modified LGIP technique

The conventional LBP local texture feature extracting technique will allocate each pixel in an image with eight binary bits based on its intensity value with eight neighbours. Then, the resultant output from conventional LBP is transformed into binary format (Wady, 2022). This transformed binary representation from conventional LBP is mathematically formulated in Eq. (11), where $i_p$ is the intensity of pixel, $p$ in a neighbourhood around the central pixel, and the pixel intensity of the centre pixel is indicated as $i_c$. Here, the segmented image, $S^k$ is subjected to the extraction of improved LBP using the formulation Eq. (12). Initially, the segmented image, $S^k$ is transformed into a grey-scale image, $g{S}^k$. In the grey-scale image, the grey value of the eight nearby pixels in a neighborhood around the central pixel is indicated, and its value is evaluated in an improved manner formulated in Eq. (13).

(11) $$LBP=\sum _{i=0}^{n-1}{2}^{i}th\left(g_i-g_c\right)where,th\left(S^k\right)=\{\begin{array}{ll}1,& S^k\ge 0\\ 0,& S^k<0\end{array}$$

(12) $$ILBP=\sum _{i=0}^7{2}^{i}th\left(g_i-g_c\right)where,th\left(S^k\right)=\{\begin{array}{l}1,S^k\ge 0\\ 0,S^k<0\end{array}$$

(13) $$g_i=\left(p_i^2-{\displaystyle \frac{1}{\left|p_i\right|}adj\left(p_i\right)}\right)\left({\displaystyle \frac{L_{p_i}+R_{p_i}}{T_{p_i}}}\right)$$where, $p_i$ denotes one of the eight nearby pixels in a neighborhood around the central pixel and $L_{p_i},R_{p_i}and{T}_{p_i}$ indicates the left, right, and top pixels of $p_i$. Thus, from this improved LBP technique, the spatial patterns in segmented image, $S^k$ are captured effectively by following the above procedure.

(ii) Process executed after the onset of improved LBP Technique in modified LGIP

After improved LBP procedure, the procedure of general LGIP follows. Generally, LGIP partitions an image through a fractional partitioning technique for generating binary vectors corresponding to the image’s horizontal and vertical directions. This modified LGIP technique results from texture features effectively by providing information about the local gradient variations based on the distribution and arrangement of gradients within the defined local regions or neighborhoods in segmented images, $S^k$.

• In this modified LGIP, eight nearby pixels considered in improved LBP are initially utilized to compute the response of gradients in each pixel’s eight potential directions by applying the Sobel gradient operator.

• In this instance, gradient responses in eight orientations at each pixel are calculated using Sobel masks (M0 to M7). Then, using the signs of the eight replies, an eight-bit code is allocated to the pixel. The modified local gradient pattern (LGIP) technique uses an extension of the local binary pattern (LBP) method to capture more detailed texture information by considering the gradient responses in multiple orientations. The Sobel operator is typically used to calculate gradient responses in the horizontal and vertical directions, but for the modified LGIP, it seems that the Sobel operator has been extended to calculate gradients in eight different orientations. The parameters M0 to M7 likely represent the masks or kernels for the Sobel operator in these eight different orientations. In the standard Sobel operator, there are two 3 × 3 kernels: one for detecting horizontal changes and one for vertical changes. For the modified LGIP, these kernels have been adapted or extended to capture gradient information in eight specific directions around a pixel. For the modified LGIP, the kernels M0 to M7 would be similar in concept but oriented in different directions to capture gradients at angles other than just horizontal and vertical. These could include diagonal and intermediate orientations.

• Each pixel in an image is assigned an eight-bit value based on the polarity of its gradient value. The encoding process involves setting a single bit to 1(p) and the rest to 0, resulting in each bit representing the output of a specific mask.

• The eight bits can be evaluated using intensity comparisons between the central pixel and its neighbors to speed up computations.

• Improved LBP uses the Sobel gradient operator to stabilize illumination variation and noise. It is faster than LDP’s Kirsch edge detector and achieves gradient operator from the central pixel to neighbours in stable conditions (Peng, Zhang & Guo, 2023). Thus, the modified LGIP captures local texture features effectively by following these procedures.

2. Multi-texton Analysis-based Texture Feature Extraction

Multi-texton analysis is done by extracting and analyzing diverse textons in an image. Textons are the primitive patterns or fundamental elements in an image that constitute the texture of it. Initially, this multi-texton analysis technique works on extracting diverse sets of textons from the segmented image, $S^k$ for capturing several textons representing different texture patterns. Then, these extracted textons are encoded into an appropriate feature representation. Then, these features are mapped to a feature space to preserve their distinctive properties (Wady, 2022). Thus, it results in informative local texture features effectively from the segmented image, $S^k$. After the extraction of the local texture features using both the modified LGIP and Multi-texton techniques, they are combined and symbolized as $Te{x}_F$. The use of LGIP enhances texture feature extraction by capturing finer gradient variations, while multi-texton analysis provides a richer representation of texture patterns. This dual approach offers a more comprehensive understanding of the image features, which is crucial for accurate disease classification. The modified local gradient increasing pattern (LGIP) and Multi-texton analysis offer significant advantages in feature extraction by enhancing texture sensitivity and feature discrimination. LGIP improves texture detail capture by focusing on local gradient variations, which aids in distinguishing subtle patterns and structures in medical images while maintaining robustness against noise. Multi-texton analysis complements this by providing a comprehensive representation of diverse texture patterns, enriching the feature set and enhancing the model’s ability to recognize and classify complex textures. Together, these methods improve segmentation and classification accuracy, making them highly effective for analyzing intricate and varied imaging data.

Shape features

Shape features are quantitative measures or descriptors that characterize an image’s geometric properties and objects or regions. These features provide valuable information about objects’ contours, boundaries, and structural attributes, allowing tasks like differentiation and classifying objects based on their shapes. The common mechanisms used for shape feature retrieval include identification such as aspect ratio, lines, circularity, and boundaries and detecting areas of variation or stability by region growth and edge detection approaches (Minarno et al., 2016). Thereby, from the segmented image $S^k$, the obtained shape features are symbolized as $Sh{a}_F$.

Color features

Color features are quantitative measures that describe the characteristics of colors present in an image, mainly used in applications like image retrieval, content-based image analysis, object recognition, etc. The color feature provides important details on the arrangement, makeup, and characteristics of colors in an image. Based on color similarity, color features are extracted from photos by calculating a color histogram for each image, which identifies the percentage of pixels in an image that contains a certain value. The color feature will segment color proportion by region and spatial relationship among several color regions (Minarno et al., 2016). The input segmented image is applied to extract such practical features in this feature extraction step, and the resultant color feature is symbolized as $Co{l}_F$.

Deep features

Deep features from an image are extracted using DL models, training them using high-level image patterns. These features are derived from layers deep within the DL techniques’ network architecture from which more complex features are captured. The captured complex features give more meaningful information about the input data than the raw input. In this proposed work, deep features from segmented images, $S^k$ is extracting by employing two efficient DL architectures, such as ResNet and VGG16.

1. ResNet-based Deep Feature Extraction

ResNet (Harini & Lalitha Bhaskari, 2011) is an invention of Microsoft Corporation that became popular in image classification. With residuals, this architecture is improved because an input image is converted to an output without the need for any neural network operations, preserving the original image. A deep residual network that traverses several levels was used to create Resnet. Initially, this ResNet architecture is trained on a lung medical image dataset to prepare it as a pre-trained ResNet architecture with weights for efficiently capturing deep and informative features from the segmented image, $S^k$.

2. VGG16-based Deep Feature Extraction

Generally, VGG16 architecture (Sriporn et al., 2020) consists of 16 layers, where 13 are convolutional, and three are fully connected. These layers are involved in extracting more complex features from segmented images, $S^k$ by enabling VGG16 to capture the visual content at various levels of abstraction. Deep features extracted by VGG16 architecture from segmented images, $S^k$ are obtained from the output of each convolutional layer as the input segmented image, $S^k$ p passes through the architecture; each convolutional layer extracts high-level representations of cap S to the k. Further, the deeper layers in this architecture capture more complex patterns and structures in the input-segmented image, such as textures, shapes, and object components. Thus, this architecture extracts more informative and deep features efficiently.

After the extraction of deep features using both the ResNet and VGG16 architectures, they are combined and symbolized as $D{e}_F$.

Finally, the features extracted from the segmented image, $S^k$ Using specific techniques are concatenated, and it is symbolized as $F^k$: $F^k=\left[Te{x}_{F}Sh{a}_{F}Co{l}_{F}D{e}_F\right]$.

LinkNet in L-MLSTM model

Based on pixel-wise classification, the LinkNet model effectively computes and outputs lung disease classification results. The encoder and decoder routes have several encoder and decoder blocks, respectively.

The informative features, $F^k$ extracted from the segmented image, are given as input to each encoder block comprising convolutional layers, batch normalization, and ReLU activation functions. Samples of those input features are blocked by the encoder to obtain the local and global context information. The decoder blocks in the decoder path use feature-up sampling to provide the original image resolution, in contrast to encoder blocks. Therefore, to enable relevant data propagation at varied scales and orientations, each decoder block uses skip connections to mix features from the encoder block. This allows for the progressive recovery of the spatial data lost during the encoder path's down-sampling procedure. Here, the involvement of skip connection is essential to efficiently transfer low-level and high-level features by signifying the precise localization of objects, boundaries, and contours in the segmented image $S^k$ Finally, the architecture of this model is concluded with a final convolutional layer that passes generated classification results on lung diseases ( $Link{N}_C$) through a softmax activation function.

MLSTM in L-MLSTM model

This MLSTM is developed as the conventional LSTM model, but the difference is that this implemented MLSTM used a new activation function named ‘ScReLU’ instead of a sigmoid activation function. This replacement is introduced for enhancing the non-linearities introduced into the model to train and approximate intricate, non-linear relations within the segmented image; along with this ScReLU activation function, the MLSTM model is implemented based on conventional LSTM architecture, which means that it is implemented using cell state $c_t$, input gate $i_t$, output gate $o_t$, candidate vector $c_t^{\mathrm{\prime }}$, hidden state $h_t$, and forget gate $f_t$ using the following equations. The MLSTM model functions initially by deciding which new information should be stored in the cell state where the input gate makes this decision, $i_t$ in it. Then, the input gate generates an input vector by considering the previous hidden state and current input, merged with the candidate vector, $c_t^{\mathrm{\prime }}$ for updating the cell state; the cell state acts as a memory of the MLSTM model by allowing functions such as storing and accessing information. The forget gate can determine what data from the previous cell state should be kept or discarded. Further, it initializes the inputs as current input and previous cell state and a forget vector as output; it is then formulated under element-wise multiplication with the last state cell for removing unnecessary data. Finally, the classification results ( $MLST{M}_C$) on lung disease classification are regulated by the output gate by controlling which components of the cell state are used in the computation of the hidden state ( $h_t$) output. Iteration continues by feeding the recurrent part of the input vector after updating this part using the current cell state (an input vector comprises two components: Input and recurrent parts). The updation of each gate and cell state is mathematically formulated from Eqs. (14) to (19) (Poornima & Pushpalatha, 2019).

(14) $$i_t=ScReLU\left(w_i\left[F^k\left(t\right),h_{t-1}\right]+b{i}_i\right)$$

(15) $$f_t=ScReLU\left(w_f\left[F^k\left(t\right),h_{t-1}\right]+b{i}_f\right)$$

(16) $$o_t=ScReLU\left(w_o\left[F^k\left(t\right),h_{t-1}\right]+b{i}_o\right)$$

(17) $$c_t^{\prime }=ScReLU\left(w_c\left[F^k\left(t\right),h_{t-1}\right]+b{i}_c\right)$$

(18) $$c_t=f_t\ast c_{t-1}+i_t\ast c_t^{\prime }$$

(19) $$h_t=o_t\ast ScReLU\left(c_t\right)$$where the $F^k(t)$ is the input vector, $h_{t-1}$ represents the previous state hidden vector, weights and bias vectors of all gates and cell state are represented as $W$ and $b$. The employed activation function Sigmo-Comb Rectified Linear Unit (ScReLU), is obtained by combining three activation functions: $Sigmoid\left(f\left(F^k\right)={\displaystyle \frac{1}{\left(1+{\exp }^{-F^k}\right)}}\right)$, comb-H-sine $\left(f\left(F^k\right)=\sinh \left(\beta F^k\right)+{\displaystyle \frac{1}{\sinh }\left(\beta F^k\right)}\right)$, and Leaky ReLU $\left(f\left(F^k\right)=\{\begin{array}{cc}{F}^k;& if{F}^k\ge 0\hfill \\ \alpha F^k;& otherwise\hfill \end{array}\right)$ (Vijayaprabakaran & Sathiyamurthy, 2022). To express the formulation of the ScReLU activation function, Sigmoid (Xiangyang et al., 2023) and comb-H-sine (Guo et al., 2021) activation functions are added, and the result from the addition operation is shown in Eq. (20). Combining a bounded function (like Sigmoid) with an unbounded function (like Comb-H-Sine) provides a powerful activation function that can normalize outputs while retaining the ability to handle extreme values and capture complex patterns with the flexibility of unbounded functions. It improve gradient flow and learning efficiency across a wide range of input values and enhance the model’s regularization and generalization capabilities. This ensures that the activation function can provide both stability and flexibility, addressing potential limitations of using each function individually.

(20) $$f\left(F^k\right)={\left(1+{\exp }^{-F^k}\right)}^{-1}+\sinh \left(\beta F^k\right)+{\displaystyle \frac{1}{\sinh }\left(\beta F^k\right)}$$

(21) $$LeakyReLU\Rightarrow f\left(F^k\right)=\{\begin{array}{cc}{F}^k;& if{F}^k\ge 0\hfill \\ \alpha F^k;& otherwise\hfill \end{array}$$

Now, Eqs. (20) and (21) are merged to form the ScReLU activation function, which is formulated in Eq. (22), where $\mathrm{\alpha }$ is equivalent to 0.01. The ScReLU activation function combines the strengths of multiple activation functions; Sigmoid, comb-H-sine, and Leaky ReLU to provide a more versatile and effective non-linearity for neural networks. This composite function enhances the model's ability to learn complex patterns by improving gradient flow and mitigating issues like vanishing gradients through its smooth transitions and robust handling of negative inputs. ScReLU offers greater stability and adaptability across various input ranges, making it particularly useful for tasks requiring nuanced feature extraction and improved convergence. Its ability to blend different activation behaviors results in richer feature representations and faster, more stable training, thereby enhancing overall model performance. ScReLU’s combination of Sigmoid, Comb-H-Sine, and Leaky ReLU functions offers a flexible, adaptive approach that can handle a broad range of input values and non-linearities effectively. This makes it particularly suitable for complex tasks like lung disease segmentation and classification, where diverse and varying features need to be captured and processed. Its ability to manage gradient flow and avoid saturation effects further enhances its utility in deep learning models for medical image analysis.

(22) $$ScReLU\Rightarrow f\left(F^k\right)=\{\begin{array}{ll}{F}^k;& if{F}^k\ge 0\\ {\left(1+{\exp }^{-F^k}\right)}^{-1}+\sinh \left(\beta F^k\right)+{\displaystyle \frac{1}{\sinh }\left(\beta F^k\right);}& if0>F^k\ge -1;\left(-\mathrm{\infty },\mathrm{\infty }\right)\\ \alpha F^k;& otherwise\end{array}$$where, $\beta$ is a parameter for the comb-H-sine function, $\alpha$ is the slope for Leaky ReLU.

For $F_k\ge 0$: The ScReLU function returns the input value $F_k$ similar to the ReLU function.

For $-1\le F_k<0$: The function combines a sigmoid component with the comb-H-sine function, which captures a range of non-linear behaviors for slightly negative values. The $(-\mathrm{\infty },\mathrm{\infty })$ ranges with both positive and negative values, thus maintain meaningful gradients and handle extreme input values effectively, ensuring robust learning and performance across a wide range of input values.

For $F_k\le -1$: The function uses Leaky ReLU to ensure the gradients are not zero for highly negative values, preventing issues with gradient flow during training.

ScReLU stands out as a robust activation function due to its ability to introduce complex non-linearities through the combination of multiple activation functions. It improve gradient flow and mitigate issues like vanishing gradients and inactive neurons. It handle extreme values effectively, ensuring that the activation function remains effective across a wide input range. It provide adaptive behavior, allowing the model to learn efficiently from diverse input values. These properties make ScReLU a compelling choice for enhancing model performance, particularly in complex and varied tasks.

Figure 4 displays the activation function plot of the ScReLU function over traditional activation functions. The ScReLU activation function offers better efficiency over traditional activation functions like Sigmoid, comb-H-sine, and Leaky ReLU. ScReLU remains at zero for negative inputs and linearly increases with a positive slope for positive inputs, while other activation functions such as; the sigmoid function start near zero, slowly increase, and then plateaus at an output of 1 as input increases. The comb-H-sine function oscillates above and below zero without a clear trend as input increases. Leaky ReLU has a small, positive slope for negative inputs and a larger positive slope for positive inputs. Thus it is evident that the ScReLU activation function shows better performance. ScReLU avoids the vanishing gradient problem and is computationally more efficient, making it a better choice for deeper networks. ScReLU provides more stable and predictable outputs compared to the oscillatory nature of comb-H-sine. The scaling factor can be adjusted to improve the learning process, making it more flexible than standard ReLU. The use of ScReLU in Modified LSTM networks further enhances the model’s performance by providing stable and efficient learning for lung disease segmentation and classification.

The hybrid model that combines LinkNet architecture with modified long short-term memory (LSTM) networks offers substantial advantages for lung disease classification by leveraging the strengths of both components. LinkNet’s encoder-decoder structure provides precise spatial feature extraction and segmentation through its efficient skip connections, ensuring detailed and accurate identification of lung abnormalities. Meanwhile, the modified LSTM network, enhanced with the ScReLU activation function, captures complex temporal dependencies and contextual information, improving the model's ability to understand sequential patterns and variations over time. This integration of spatial detail and sequential learning results in superior classification performance, allowing the model to effectively differentiate between various lung diseases and enhance diagnostic accuracy. Finally, the classification results on lung diseases using LinkNet and MLSTM models are averaged for efficient and accurate results on lung disease classification. The final results are symbolized as $C^k$. Table 2 describes the parameters of the classifiers.

Dataset 1

One of the datasets utilized for L-MLSTM model training and testing is COVID-19 normal pneumonia-CT images. The dataset has CT images of normal, pneumonia, and COVID-19 patients’ lungs. In the Covid2CT category, there are 2,034 images; in the normal lungs category, it provides 2,119 images; the pneumonia category folder contains 3,390 images. Each image is 512 × 512 pixels.

Dataset 2

The second dataset utilized for model testing and training is pictures from chest CT scans. The material covers three forms of chest cancer: adenocarcinoma, large-cell carcinoma, and squamous-cell carcinoma. The total number of data presented is 1,000. The pictures are in JPG or PNG format. The main folder contains step folders comprising a 70% training set, a 20% testing set, and a 10% validation set. CT scan photos from the dataset show several forms of chest cancer. Table 4 describes the training, testing, and validation data for both datasets.

Preprocessing analysis

For datasets 1 and 2, the study of picture preprocessing using the mean filter and existing filters such as the Clahe, Gaussian, and mean filters is shown in Figs. 5 and 6. Looking at the results, the preprocessed image using the median filter has a more appealing appearance. The value of each pixel is replaced by the median of its surrounding pixels when using median filtering. The median reduces noise effectively without adding erroneous pixel values because it is less susceptible to extreme values, or outliers. For improved results, median filtering is a useful preprocessing method for lung pictures.

Table 5 presents the enhanced median filter’s performance for datasets 1 and 2 in terms of SSIM and PSNR in comparison to the current filters, which include Gaussian filtering, Clahe filtering, and conventional median filtering. Two computational tools that were commonly used in the evaluation of image quality were PSNR and SSIM. Based on the relationship between each pixel and the other pixels in an 11-by-11 neighborhood, the SSIM function determines the structural similarity index of each pixel. Table 5 makes clear that the improved median filter produced higher PSNR and SSIM rates—roughly 30.47382 and 0.903462 in Dataset 1 and 34.22451 and 0.914578 in Dataset 2, in that order. In contrast, gaussian, low pass, and conventional median filters obtained subpar results. As a result, compared to previous filters, the enhanced median filter ensures a more effective segmentation procedure.

Segmentation analysis

For datasets 1 and 2, the picture results of the pre-processed input image, TCNN, Segnet image, U-Net, fuzzy C-mean clustering, K-mean clustering, and ITCNN technique are represented in Figs. 7 and 8. An analysis is done on the segmented images by the ITCNN approach and conventional models. The ITCNN model for segmentation uses an Improved transformer-based CNN model, which enhances segmentation phase performance. The ITCNN model's segmentation accuracy is examined and contrasted with methods from the past. Some of the segmentation evaluation metrics are the dice score and the Jaccard similarity. The segmentation assessment of the recommended approach in comparison to traditional methods is displayed in Tables 6 and 7.

Dice score: A measure of how similar two samples are, the Dice score (sometimes called the Dice coefficient) is computed by dividing the size of their intersection twice by the total of the two samples’ sizes.

(23) $$Dicescore=\left(2\left|A\cap B\right|\right)/\left(\left|A\right|+\left|B\right|\right)$$

Jaccard similarity: A metric compares sample sets’ variety and similarity. It can be expressed as the product of the size of the sets’ union and the size of their intersection.

(24) $$J\left(A,B\right)=\left(\mid A\cap B\mid \right)/\left(\mid A\cup B\mid \right)$$

Analysis of confusion matrix

The efficacy of the L-MLSTM model in comparison to conventional techniques is assessed using a confusion matrix by examining true positive, true negative, false positive, and false negative results. Figures 9 and 10 display the L-MLSTM Confusion Matrix and traditional methods for datasets 1 and 2. Higher TPR means fewer missed cases of lung disease, which is critical for early diagnosis and treatment. Higher TNR reduces the incidence of false positives, ensuring that healthy individuals are not wrongly identified as diseased. The L-MLSTM model achieved a true positive rate (TPR) of 265 and a true negative rate (TNR) of 579, which are the highest among all conventional models. This indicates that the L-MLSTM model is exceptionally effective at correctly identifying both positive (diseased) and negative (healthy) cases. In comparison, conventional methods have lower TPR and TNR values, with the highest TPR being 228 and TNR being 542. The higher TPR signifies that the L-MLSTM model excels in detecting the presence of lung disease, reducing the risk of missed diagnoses. The higher TNR implies it is also better at correctly identifying non-diseased cases, thus reducing false alarms.

Similarly, the L-MLSTM model outperforms conventional approaches with a TPR of 264 and a TNR of 863. These results suggest that the model consistently performs better in both detecting and avoiding false positives, making it highly reliable for lung disease detection across different datasets. The model’s superior TPR and TNR contribute to overall reliability in clinical settings, where accurate diagnosis is crucial for patient management. Subsequently, lower FNR reduces the risk of overlooking true cases of lung disease, leading to better patient outcomes. Lower FPR helps in reducing unnecessary follow-up tests and psychological stress for patients falsely identified as having a disease. Fewer false positives and negatives improve the overall efficiency of the diagnostic process, saving time and resources in medical settings. The L-MLSTM model has a false negative rate (FNR) of 50 and a false positive rate (FPR) of 51, both of which are lower compared to conventional techniques. Lower FNR means fewer false negatives, thus reducing the risk of failing to detect actual cases of lung disease. A lower FPR means fewer false positives, which minimizes unnecessary concern and further testing for healthy individuals. The FNR is even lower at 36, which further enhances the model’s ability to correctly identify diseased cases without missing any. The reduced FNR in dataset 2, along with high TPR and TNR, suggests that the model is robust and consistently performs well across different datasets.

The results indicate that the L-MLSTM model not only performs better than conventional methods but also shows robustness across different datasets. The combination of LinkNet and modified LSTM allows the model to capture both spatial and temporal features more effectively, leading to better segmentation and classification performance. The use of advanced feature extraction techniques, such as modified LGIP and Multi-texton analysis, contributes to improved accuracy by providing more detailed and relevant information for classification. Thus, the L-MLSTM model demonstrates significant advantages over conventional techniques, with higher TPR and TNR, and lower FNR and FPR. These results highlight the model's effectiveness in accurately diagnosing lung diseases while minimizing false results.

Interpretation of both positive and negative metrics

The L-MLSTM model is evaluated over several previously developed methods such as HDE-NN (Gugulothu & Balaji, 2024), DBN (Manickavasagam & Sugumaran, 2023), LSTM, LINKNET, SVM, Bi-GRU, RNN, CNN, and VGG19 + CNN, for lung disease detection for datasets 1 and 2. The comparison is made at 60, 70, 80, and 90 learning percentages. As demonstrated in Fig. 11A, for dataset 1, the L-MLSTM approach's superior accuracy level of lung disease detection at a learning percentage of 70 is 89%, which was greater than the conventional models. Furthermore, at learning percentages of 80 and 90, the L-MLSTM model achieved sensitivity of 87% and 92%, respectively. The specificity, precision, and recall for the suggested work were 96%, 93%, and 95% respectively, with a learning percentage of 90. Figure 12 displays the performance assessment of the L-MLSTM model evaluated over conventional approaches for dataset 2. At an LP of 90, the L-MLSTM model’s accuracy was 95%, nearly 8% greater than previous models. The sensitivity and Precision of the L-MLSTM model are 88% for both, which is higher than the conventional technique. LinkNet and modified LSTM are used to classify lung disease. High accuracy signifies that the L-MLSTM model correctly identifies both diseased and healthy cases with greater precision. This is crucial for reliable diagnosis and effective treatment planning. Higher sensitivity reduces the risk of missed diagnoses, while increased specificity ensures fewer false positives. This dual improvement enhances the model’s reliability in distinguishing between affected and non-affected cases. High precision means that when the model predicts a disease, it is more likely to be correct, reducing unnecessary follow-ups and patient anxiety. As a result, the L-MLSTM can precisely detect lung disease. The L-MLSTM’s superior accuracy and sensitivity at higher learning percentages reflect its ability to learn and generalize better from training data. Improved specificity and precision indicate the model’s effectiveness in minimizing both false positives and negatives, leading to more reliable diagnoses.

A few negative metrics must be taken into account when evaluating a model’s performance. When comparing the model’s reduced error to other traditional techniques such as HDE-NN (Gugulothu & Balaji, 2024), DBN (Manickavasagam & Sugumaran, 2023), LSTM, LINKNET, SVM, Bi-GRU, RNN, CNN, and VGG19 + CNN for datasets 1 and 2, FPR and FNR values are taken into account. Figure 13 shows that the L-MLSTM model has the lowest FPR and FNR value rate for dataset 1. At the 80th learning percentage, the value of the FPR is approximately 0.03 as shown in Fig. 13A; the value of the L-MLSTM model was 5% smaller than that of the other conventional approaches. The FPR value of the L-MLSTM model, with a learning percentage of 90, is 0.07; however, the values of the other previous strategies were greater. Reduction of false positive lung illness diagnosis is associated with a lower FPR score. The FNR of the L-MLSTM approach is the lowest when compared to conventional procedures, with values of 0.12 at a learning percentage of 80 and 0.07 at a learning percentage of 90. The L-MLSTM approach’s negative metrics analysis for dataset 2 is assessed in Fig. 14. At the 90th LP, the L-MLSTM model has 0.02 FPR and 0.08 FNR; it gains comparatively low error value. Because the error measures in the suggested model have relatively low values, it performs better overall. Lower FPR means fewer healthy individuals are incorrectly identified as diseased, reducing unnecessary treatments and anxiety. Lower FNR ensures that fewer actual cases of disease are missed, leading to better patient outcomes. Lower FPR and FNR enhance the model’s reliability in clinical settings, improving trust in the diagnostic process and potentially leading to better patient care. The L-MLSTM model's reduced FPR and FNR are attributed to its hybrid architecture, which integrates advanced segmentation and classification techniques. This integration improves the model’s ability to differentiate between pathological and non-pathological conditions more effectively.

Figures 15 and 16 show the performance analysis of the proposed hybrid L-MLSTM model over conventional classifiers like HDE-NN (Gugulothu & Balaji, 2024), DBN (Manickavasagam & Sugumaran, 2023), LSTM, LINKNET, SVM, Bi-GRU, RNN, CNN, and VGG19 + CNN with regards to other measures such as F2-score, NPV, and F-measure for Dataset 1 and 2. The suggested model acquired better rates in NPV and F-measures for datasets 1 and 2. High F2-score, NPV, and F-measure values reflect the model’s ability to balance sensitivity and precision, providing a comprehensive assessment of its performance. These metrics ensure that the model performs well across various aspects of classification, including predicting negative cases correctly (NPV) and balancing precision and recall (F-measure). The suggested approach can detect and classify lung diseases precisely. The F2-score, NPV, and F-measure values of the L-MLSTM model highlight its capability to offer a well-rounded performance across different evaluation criteria. The hybrid nature of the model, combining LinkNet and Modified LSTM, likely contributes to its balanced and high-performance metrics. Thus the L-MLSTM model’s ability to achieve superior results in accuracy, sensitivity, precision, recall, and specificity, alongside lower FPR and FNR, and enhanced F2-score, NPV, and F-measure, demonstrates its effectiveness and reliability in lung disease detection. These results suggest that the L-MLSTM model offers a significant improvement over conventional technique. Its advanced hybrid architecture, which combines robust feature extraction and classification methods, contributes to its exceptional performance.

Ablation study for dataset 1 and dataset 2

The L-MLSTM model outperforms the model developed with conventional methods for dataset 2. By eliminating unwanted noise and outliers, the L-MLSTM technique improves the quality of the images during the pre-processing step of the L-MLSTM model. Then, to accurately classify lung illness, a hybrid classifier consisting of LinkNet and VGG16 is utilized. Tables 8 and 9 present the results of the L-MLSTM approach’s ablation research for datasets 1 and 2. The L-MLSTM approach performs superior to the model by conventional segmentation, conventional LGIP, and conventional feature extraction, as Table 8 makes abundantly evident. The L-MLSTM model’s accuracies is 91%, while the standard segmentation model's accuracy is 87%. The conventional LGIP model also has 4% less sensitivity than the L-MLSTM model. Comparing existing techniques and the L-MLSTM model regarding hostile measures, the L-MLSTM model produces values about 4% and 2% lower than the conventional models; the proposed scheme has just a 0.12 FNR error value. When evaluating the L-MLSTM approach with the traditional methods of dataset 1, the model performs better in detecting lung disease.

Table 9 lists the results of the ablation study for the L-MLSTM technique for dataset 2 using traditional feature extraction, conventional stage of segmentation, and traditional LGIP. Improved model performance is indicated by improved outcomes for the L-MLSTM system’s sensitivity, specificity, accuracy, NPV, F-measure, and MCC for dataset 2. The 93% accuracy of the L-MLSTM model indicates that the model using traditional segmentation, traditional LGBPHS, and conventional score level fusion model for dataset 2 has the lowest accuracy. The higher accuracy and sensitivity of the L-MLSTM model indicate its superior ability to correctly identify lung disease cases. This translates to fewer missed diagnoses and more accurate detection, which is crucial for effective treatment and patient outcomes. By combining LinkNet for segmentation and VGG16 for classification, the L-MLSTM model leverages advanced techniques to integrate detailed image features, enhancing overall diagnostic performance. The superior performance of the L-MLSTM model can be attributed to its sophisticated architecture, which effectively enhances image quality during pre-processing and integrates powerful classifiers. The use of LinkNet in segmentation ensures precise delineation of lung regions, while VGG16’s robust feature extraction and classification capabilities contribute to higher accuracy and sensitivity. The combined strengths of these components result in a model that outperforms traditional methods. The ablation study results confirm that the L-MLSTM model’s hybrid approach, incorporating advanced segmentation and classification techniques, provides substantial improvements over traditional methods. This validation of the model’s performance demonstrates its potential as a superior tool for accurate and reliable lung disease diagnosis.