Efficient deep learning model for classifying lung cancer images using normalized stain agnostic feature method and FastAI-2

Dataset description

The LC25000 dataset, proposed by Borkowski et al. (2019), is an open-source histopathological image dataset aimed at facilitating machine learning research for cancer diagnosis. It contains 25,000 colour images, divided equally into five different categories, such as lung adenocarcinoma, lung squamous cell carcinoma, benign lung tissue, colon adenocarcinoma, and benign colon tissue. Each image was originally taken from histopathological slides at a $1,024\times 768$ pixel resolution and then resized to $768\times 768$ pixels to keep an equal aspect ratio appropriate for deep learning tasks.

Here, we address the issue of lung cancer histopathology classification alone, i.e., the classification of images into three categories: lung adenocarcinoma, lung squamous cell carcinoma, and benign lung tissue. The data were already preprocessed during the time of data acquisition and augmented to eliminate additional preprocessing requirements of augmentation. The initial augmentation step was carried out with the Augmentor software tool (Borkowski et al., 2019), implementing random left and right rotation (max 25 degrees, probability = 1.0) and horizontal and vertical flip (probability = 0.5) to provide a rich representation of histopathological variations. The pre-augmented dataset has been tested to be HIPAA-compliant, maintaining patient confidentiality, and forms the standard utility for deep learning model development in cancer classification. With the use of this dataset, scientists can investigate strong machine learning methods for separating malignant from benign lung tissue that can lead to improvements in AI-based pathology diagnostics (Borkowski et al., 2019).

Colour-normalizing of the histopathological image from the LC25000 dataset

Pre-processing is carried out to enhance images by suppressing noise and enhancing crucial characteristics. As a result, crucial information is extracted from the images, and the images become compatible with deep-learning and machine-learning networks. Noise can be seen in the images of the LC25000 dataset because the biopsy was combined with a variety of medical materials, and there is not enough contrast between the afflicted tissue and the surrounding tissue. This article explains a technique for a broader type of colour correction known as colour borrowing, in which the colour properties of one image are taken from another image. The following series of steps are followed for this process. In the First step, the augmented image is converted into L*a*b* colour space, followed by L*a*b* colour space, which is projected on decorrelated colour space using the borrower image. Then finally, the projected image is back to RGB colour space from the L*a*b* colour space.

Step 1: Conversion of augmented RGB image into L*a*b* colour-space

In the RGB colour space, if the blue channel is dominant, it means that the blue intensity is significantly higher relative to the red and green channels. This affects the overall perception of colour, requiring adjustments across all three channels to ensure a consistent colour transformation. Since RGB channels are interdependent, modifying one component without adjusting the others can lead to unnatural colour shifts, making simultaneous adjustments necessary for maintaining colour consistency. Any technique that modifies colours is made more difficult due to this. A colour space that is orthogonal and free of connections between its axes is precisely what we are looking. L*a*b colour space can be the answer to this. During the conversion from RGB to L*a*b colour model, the Euclidean distance is calculated since the difference in perceived value between two colours in the L*a*b* colour space is consistent across all observers.

Three different aspects make up the L*a*b* colour space: the luminance component L*, and the chromaticity components a* and b*. These components represent the position of colour along the red-green and blue-yellow axes, respectively (Solis et al., 2012). The conversion formula initially establishes the tri-stimulus coefficients in the following manner:

(1) $$\begin{array}{rl}X& =0.43R+0.35G+0.19B\\ Y& =0.23R+0.71G+0.07B\\ Z& =0.02R+0.13G+0.94B.\end{array}$$

The calculation of the CIELab colour model is as follows:

(2) $$\begin{array}{rl}{L}^{\ast }& =116\left\{h\left({\displaystyle \frac{Y}{Y_s}}\right)\right\}-16\\ a^{\ast }& =500\left\{h\left({\displaystyle \frac{X}{X_s}}\right)\right\}-h\left({\displaystyle \frac{Y}{Y_s}}\right)\\ b^{\ast }& =200\left\{h\left({\displaystyle \frac{Y}{Y_s}}\right)\right\}-h\left({\displaystyle \frac{Z}{Z_s}}\right)\end{array}$$where the usual stimulus coefficients $X_s,Y_s,and{Z}_s$ represent the reference white values in the CIE standard illuminant, which are used to normalize the colour transformation process for ensuring perceptual consistency. After calculating L*a*b channels, the de-correlation of colour space is started based on the borrower image (discussed in Fig. 3).

Step 2: De-correlation of colour space using borrower image

Consider an image $I\left(i,j,k\right)$, where $i,j$ are the height and width of the image, respectively, and $k$ represents the number of channels of the image. Then Eq. (3) decorrelates the pixel’s intensity.

(3) $$In{p}_{img}\left[i,j,k\right]=\left[\left(In{p}_{img}-In{p}_{img}\left(mean\left[k\right]\right)\right)\ast \left\{{\displaystyle \frac{Bo{r}_{img}\left(std\left[k\right]\right)}{In{p}_{img}\left(std\left[k\right]\right)}}\right\}\right]+Bo{r}_{img}\left(mean\left[k\right]\right).$$

$In{p}_{img}$ is the colour-augmented input image, $In{p}_{img}\left(mean\left[k\right]\right)$ represents the mean of the inputted image of each channel, $Bo{r}_{img}\left(std\left[k\right]\right)$ keeps a record of the standard deviation of borrower image and $In{p}_{img}\left(std\left[k\right]\right)$ handle standard deviation of inputted image. $Bo{r}_{img}\left(mean\left[k\right]\right)$ will keep track of the mean of borrower images for each channel. After these colour processing adjustments, the next step is again back to RGB colour space from L*a*b* colour space (Reinhard et al., 2001; Shen et al., 2022).

Step 3: Image back to RGB from L*a*b* colour space

An inverse procedure converts L*a*b* channels into RGB colour space. Equations. (1) and (2) are redefined as follows:

(4) $$\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]=\left[\begin{array}{ccc}1& 1& 1\\ 1& 1& -1\\ 1& -2& 0\end{array}\right].\left[\begin{array}{ccc}0.58& 0& 0\\ 0& 0.41& 0\\ 0& 0& 0.71\end{array}\right].\left[\begin{array}{c}l\\ a\\ b\end{array}\right].$$

Equation (4), $LAB$ channels are converted into $XYZ$ stimulus using matrix multiplication. After that, we can transform the data from the $XYZ$ stimulus into the $RGB$ colour space by raising the pixel values to the power of ten to return to linear space. Equation (5) helps to understand the procedure.

(5) $$\left[\begin{array}{c}R\\ G\\ B\end{array}\right]=\left[\begin{array}{ccc}4.47& -3.59& 0.12\\ -1.2& 2.4& -0.17\\ 0.05& -0.25& 1.21\end{array}\right].\left[\begin{array}{c}X\\ Y\\ Z\end{array}\right]$$

After removing all the anomalies, these colour-normalized images are fed into deep learning frames to work for further classification of lung histology.

Classification framework

FastAI-2 is a deep learning framework developed on top of PyTorch to simplify model development while offering room for sophisticated customization. It incorporates several machine learning libraries and tools to enable researchers to obtain efficient training and precise results with little code. Unlike other deep learning frameworks that involve much configuration, FastAI-2 offers high-level APIs to enable quick experimentation while still allowing access to low-level PyTorch functionality to fine-tune model performance.

The structure of the framework follows three core design principles: rapid productivity, simple configuration, and adaptable architecture. It is realized through modular and layered APIs so that both newcomers and deep learning experts can make use of it. It further provides automatic hyperparameter search, learning rate scaling, and data augmentation with its built-in functions, promoting more efficient training. By linking high-level convenience and low-level personalization, FastAI-2 is a capable tool for applying deep learning across different areas (Praveen et al., 2022).

Two of the most important design goals that FastAI-2 strives to achieve are accessibility and productivity while maintaining flexibility. The residual neural network, often known as ResNet-34, is a 34-layer. In the foundation of modified ResNet34, we need to define key operations, including convolutional layers, batch normalization, ReLU activation, residual connections, pooling, and fully connected layers. Below is a structured approach to formulating the mathematical backbone of your modified ResNet-34.

Convolutional layer: Each convolutional layer applies a set of learnable filters $W$ to the input tensor $X$, producing a feature map $Y$. The convolution operation is given by:

(6) $$Y_{i,j,k}=\sum _m\sum _n\sum _c{W}_{m,n,c,k}.X_{i+m,j+n,c}+b_k$$where, in Eq. (6), $X_{i+m,j+n,c}$ is the input pixel at position $\left(i+m,j+n\right)$ in channel $c$, $W_{m,n,c,k}$ is the weight of the filter for kernel size $\left(m,n\right)$, $b_k$is the bias term, $Y_{i,j,k}$ is the output feature map at position $\left(i,j\right)$ for channel $k$. In modified ResNet-34, these convolutional layers are followed by batch normalization and ReLU activation.

Batch normalization: Batch normalization normalizes activations before passing them to the next layer. Given an activation $a$, the normalized output is $\widehat{a}=\raisebox{1ex}{$a-\mu $}\!\left/ \!\raisebox{-1ex}{$\sigma $}\right.$ where $\mu$ is the mean of activations and $\sigma$ is the standard deviation. Batch normalization introduces learnable parameters $\gamma$ (scale) and $\beta$ (shift), giving the final transformation $Batchnormalization=\gamma \hat{a}+\beta$.

ReLU activation: ReLU is applied element-wise $f\left(a\right)=max$ (0, a). This function preserved non-linearity while preserving positive activation. The modified ResNet34 residual learning, where the output of a block is computed as, $Y=f\left(X\right)+X$, where $f\left(X\right)$ represents a sequence of convolutional layers, $X$ is the input that is directly added to the output through the skip connection. This helps mitigate the vanishing gradient problem, allowing deeper networks to learn effectively.

Adaptive average pooling: Instead of a fixed pooling size, Adaptive Average Pooling reduces the feature map dynamically to a fixed-size tensor. Given an input tensor $X$ of shape $\left(H,W\right)$, the pooled output $Y$ is computed as:

(7) $$Y_{i,j}={\displaystyle \frac{1}{H^{\mathrm{\prime }}{W}^{\mathrm{\prime }}}\sum _{p=1}^{H^{\mathrm{\prime }}}\sum _{q=1}^{W^{\mathrm{\prime }}}{X}_{i+p,j+q}.}$$

This ensures a consistent feature map size for the fully connected layer, regardless of input image dimensions. In Eq. (7), $Y_{i,j}$ is the output of the adaptive average pooling operation at position $\left(i,j\right)$ in the pooled feature map. $X_{i+p,j+q}$ is the input feature map value at position $\left(i+p,j+q\right)$, which is part of the receptive field being averaged. $H^{\mathrm{\prime }}is$ height of the pooling region (window size in the vertical direction). $W^{\mathrm{\prime }}$ is the width of the pooling region (window size in the horizontal direction). Adaptive Average Pooling computes the average value of a local region of size $H^{\mathrm{\prime }}\times W^{\mathrm{\prime }}$ in the input feature map and assigns the result to a single pixel in the output feature map. The adaptive nature ensures that the feature map is resized dynamically based on the required output dimensions.

Fully connected layer and SoftMax activation: The final classification layer maps the feature vector FFF to output probabilities $z=WF+b$, where, $W$ is the weight matrix, $F$ is the flattened feature vector, $b$ is the bias term, and $z$ is the logits vector before applying Softmax. In this article $P\left(y_i\right)={\displaystyle \frac{e^{z_i}}{{\sum }_j{e}^{z_i}}}$ is used as the Softmax function, where $P\left(y_i\right)$ is the probability of class $i,$ ensuring that all outputs sum to 1. This foundation describes how modified ResNet-34 operates, incorporating convolutional layers, batch normalization, residual learning, adaptive pooling, and Softmax classification. These modifications ensure robust feature extraction, efficient gradient flow, and optimized classification for lung cancer histopathology. A modified ResNet-34 architecture consisting of $r$ residual blocks would provide $2^r$ several potential pathways for processing the data. This is because each residual block in ResNet would create two different paths. As a direct result, decreasing the total number of layers in the design will not significantly impact the efficiency with which it functions. Working with a lower number of layers would result in faster computations and an improvement in the capability of training networks. Figure 4 illustrates the layered approach the ResNet-34 model takes to solve the problem.

The preprocessing stage involves resizing images to $768\times 768$ pixels to maintain a uniform aspect ratio, followed by colour normalization using Lab colour space transformation with decorrelation to ensure stain consistency across histopathological slides. These preprocessed images are then fed into the modified ResNet-34 model with FastAI-2 (Fig. 5) for classification into three categories: Normal, Adenocarcinoma, and Squamous Cell Carcinoma. The performance of the proposed system is evaluated and compared against existing approaches to demonstrate its effectiveness. The article illustrates the processing pipeline of the modified ResNet-34 model used in this study. The input consists of colour-normalized histopathological images, which undergo multiple stages of feature extraction, pooling, and classification within the FastAI-2 framework utilizing modified ResNet-34.

This visualization effectively demonstrates how the proposed modifications, such as colour normalization, optimized convolutional blocks, and adaptive pooling, contribute to enhancing the model’s robustness, efficiency, and classification accuracy. By integrating these components, the modified ResNet-34 ensures a computationally efficient and high-performance deep learning framework for lung cancer histopathology classification.

Tunning the hyperparameters

Training a deep neural network (DNN) is a problem of global optimization. Among the most important hyperparameters to fine-tune in the process of DNN training is the learning rate. An ill-chosen learning rate significantly impacts model performance: a small learning rate might result in training that takes longer and converges slowly, and a large learning rate might cause the loss function to oscillate or even diverge, rendering the model incapable of finding the optimal solution.

In FastAI-2, the learn.lr_find method is used to determine the optimal learning rate. This technique involves gradually increasing the learning rate after each mini-batch, while recording the corresponding loss function values. By plotting the losses against the learning rates, the behaviour of the loss function can be visualized, providing insights into the ideal learning rate for model training. Figure 6 illustrates the learning rate optimization process for three different architectures: modified ResNet-34, VGG-11, and SqueezeNet1_1. This approach enables the selection of a learning rate within the linear zone of rapid loss reduction, ensuring faster convergence and improved model performance. The use of FastAI-2’s learning rate finder enhances the training efficiency for multi-class classification, as demonstrated across the tested networks.

The graph that is displayed above (Fig. 6) can be broken down into three distinct zones: the shallow zone, which describes an environment in which a change in learning rate has a negligible impact on loss, the linear zone, which describes an environment in which we observe a rapid decrease in the loss function, and the divergent zone, which describes an environment in which a learning rate that is too high causes loss to bounce and eventually diverge from the local minima. In a perfect world, we would conduct our job in the linear zone, characterized by the most significant decline in the loss function. We do not want to operate at the minimum because doing so would prevent us from being able to update the weights. The minimum is located at the point where the gradient of the loss function is not changing. A good rule of thumb would be to choose a location at least one magnitude higher than the absolute minimum. When we use learn.lr_find, we have the option of utilizing Valley to work just perfectly in our situation. The remaining parameters, such as the number of trainable parameters, are 0.55 million, epochs are 5, batch size is 32, the activation function is ReLU, the number of convolution layers is 31, the random state is 2, and two strides are taken in this experiment. 0.55 million trainable parameters were chosen to trade off model complexity and generalization, allowing efficient feature learning without overfitting. The epochs were fixed at five based on empirical experience using FastAI-2’s one-cycle learning rate policy, where learning rates adapt dynamically to improve convergence (Smith, 2018). The model was able to achieve peak accuracy at five epochs after which no appreciable improvement was seen. A batch size of 32 was selected in order to achieve training stability and computational efficiency, and ReLU was employed because of its ease of use and capability to eliminate the vanishing gradient problem. The model employs 31 convolution layers for extraction of deep features to improve the accuracy of classification. Fixing the random state at 2 provides reproducibility of the results, whereas strides of 2 provide equilibrium between feature resolution extraction and efficiency in computation. These parameters were empirically adjusted to produce high accuracy and lower execution time, providing a strong and effective classification system.

Training and validation losses

After tuning the hyper-parameters, the training and validation procedure is started, and for each epoch, training loss, validation loss, accuracy, and time taken in each epoch are recorded and displayed in Table 3.

Figure 7 shows validation and training losses during the procedure. Table 3 shows that training loss starts from 0.215 and continuously dips up to 0.00517 as the number of epochs increases. This show model is trained perfectly with derived learning_rate.

Evaluation of the proposed framework

The proposed framework was rigorously tested on simulated data to evaluate its performance on previously unseen instances. To ensure an objective evaluation, the test data were pre-processed and categorized consistently with the training and validation sets. The multi-class performance metrics for the model’s predictions on the test set are presented in Table 4, and the confusion matrix is illustrated in Fig. 8.

The overall accuracy of the model is 99.78%, as calculated using the Weighted Average approach. The model achieved a macro precision of 0.993, macro recall of 0.996, and macro F1-score of 0.996, demonstrating consistent performance across all classes. Similarly, the micro precision, recall, and F1-score were each 0.996, reflecting the model’s strong predictive capability.

Class-specific accuracies are 99.48% for Adenocarcinoma, 99.86% for Squamous Cell Carcinoma, and 100% for Normal. These results highlight the model’s ability to accurately distinguish between the three classes, achieving high precision, recall, and F1-scores for each category. The use of macro, micro, and weighted averages ensures a comprehensive evaluation, making the performance metrics more robust and reliable for multi-class classification.

Table 5 is a comparative study of deep learning models that were employed using FastAI (PyTorch) and TensorFlow to classify lung cancer histopathology. These models were compared based on their most important performance metrics, such as precision, recall, F1-score, accuracy, ROC AUC score, and run time. Of all the models, FastAI’s ResNet-34 had the best accuracy of 99.78% with an F1-score of 100%, illustrating its high performance in classifying lung cancer histopathological images. In contrast, TensorFlow’s ResNet-34 achieved a slightly lower accuracy of 96.45%, indicating that FastAI’s implementation optimizes model training more effectively.

When comparing VGG-11 models, FastAI’s version achieved 95.99% accuracy, slightly outperforming TensorFlow’s VGG-11 at 95.83%, though the latter took significantly longer to train (1 h 41 min vs. 1 h 28 min). SqueezeNet1_1, known for its computational efficiency, exhibited the fastest training times in both frameworks, with FastAI completing training in 00:19:18, compared to TensorFlow’s 00:25:43. However, the trade-off for SqueezeNet1_1 was lower accuracy (95.59% in FastAI and 95.42% in TensorFlow), making it a viable option for resource-constrained environments but less suitable for high-precision classification tasks.

Overall, FastAI models consistently outperformed their TensorFlow counterparts in terms of accuracy and training efficiency, particularly in the case of modified ResNet-34, which achieved both higher classification performance and faster convergence in FastAI. While TensorFlow models required longer execution times, their accuracy remained close to that of FastAI, making them viable alternatives where framework flexibility or integration into existing TensorFlow-based pipelines is required. These findings highlight that FastAI provides a computational advantage with optimized learning techniques, making it a preferable choice for lung cancer histopathology classification.

Figure 9 shows five levelled images based on their predicted class by the proposed system, followed by an actual class of the image, classification loss, and probability of classification. Out of five images, two have a predicted probability of 50% with minimal loss; those are easily truncated by manual inspection. To provide evidence that the model that has been proposed is effective, we conduct a thorough analysis of the data that has been gathered and compare it to the findings that have been obtained using the most recent and cutting-edge methodologies. When the lesions of the lungs are classified concurrently, as shown in Table 6, the recommended model performs more effectively than the currently thought of as being state-of-the-art methods. In terms of accuracy, the proposed model has surpassed the previously used one and is a better fit overall with the literature.