Novel conditional tabular generative adversarial network based image augmentation for railway track fault detection

Existing work gaps

Previous studies on railway track fault detection primarily relied on classical data augmentation techniques. However, these methods often fail to capture complex variations in track faults, leading to lower detection performance. The limitations of traditional augmentation approaches highlight the need for more advanced techniques to enhance model robustness and improve fault detection accuracy.

Railway tracks images dataset

This research utilized a standard dataset (Adnan & Salman Ibne Eunus, 2024) based on railway track images. The track images in the dataset were resized to a shape of (224 × 224 × 3). The dataset contains both defective and non-defective railway track images. The sample track dataset is illustrated in Fig. 2. The dataset set contains a total of 384 railway tracks images. This dataset is utilized for the classification of defective and non-defective railway tracks using a vision-based neural approach.

Railway tracks images preprocessing

We have applied several image-processing steps to the track images dataset. Initially, we removed images that were corrupted. The dataset folder contains two subfolders Defective and non-defective. We set the same size of the input images to a height of 256 pixels and a width of 256 pixels. Then, we looped over the label folders in the dataset folder. We appended each image and its label to the data and label lists. Finally, the pixels we read are converted into NumPy arrays for the data and label lists. The preprocessed dataset is then utilized to conduct our subsequent research experiments.

Novel proposed CTGAN based image augmentation

The novel proposed CTGAN image augmentation approach is examined in this section. Figure 3 illustrates the workflow architecture of image augmentations. In this research on image augmentations, we have developed an innovative approach, the conditional tabular generative adversarial network (CTGAN) (Lee, Yoon & Kwon, 2020). In CTGAN, the generator neural network focuses on producing realistic synthetic data by learning the data distribution of the input dataset, while the discriminator network evaluates the authenticity of both real and generated data. For image generation, the generator creates images from random noise vectors conditioned on specific input features, and the discriminator classifies images as real or fake based on learned features. Through adversarial training, where the generator enhances its ability to create realistic track images, and the discriminator enhances its capability to distinguish fakes, CTGAN can generate high-quality synthetic images that mimic the characteristics of the original dataset.

The unique combination of our proposed research approach is illustrated in Fig. 4. This architecture provides a comprehensive overview of the fault detection methodology. Initially, the image dataset is augmented using the CTGAN model, and the output of the CTGAN model is subsequently integrated with the Random Forest (RF) model to enhance the classification of railway track fault detection. We generated 2,000 images for each category using the CTGAN approach, balancing the dataset as shown in Fig. 5.

The CTGAN model consists of two primary components:

Generator (G): The generator maps a latent space $Z\sim P_Z(z)$ to the image space, conditioned on specific input features: (1) $$G:Z\times C\to X^{\mathrm{\prime }}$$where:

• $Z\sim \mathcal{N}(0,I)$ is a multivariate Gaussian distribution in the latent space.

• C represents the conditioning variable (i.e., track fault categories).

• X′ is the synthetic railway track image generated by the model.

Discriminator (D): The discriminator distinguishes between real X and synthetic $X^{\mathrm{\prime }}$ images while preserving the feature distribution:

(2) $$D:X\times C\to [0,1]$$where $D(X,C)$ represents the probability that X is a real image given condition C.

Objective function: The adversarial loss function follows the standard GAN formulation:

(3) $$\underset{G}{min}\underset{D}{max}{\mathbb{E}}_{X\sim P_{data}(X)}[\log D(X|C)]+{\mathbb{E}}_{Z\sim P_Z(Z)}[\log (1-D(G(Z|C)))]$$where $P_{data}(X)$ is the real data distribution, and $P_Z(Z)$ is the prior noise distribution.

Once the railway track images are generated using CTGAN, they are integrated into the dataset to enhance railway track fault detection using the RF classifier.

Feature extraction: Each railway track image (either real or synthetic) is transformed into a feature vector F through a feature extraction function $\mathrm{\Phi }$:

(4) $$F=\mathrm{\Phi }(X)$$where $\mathrm{\Phi }:X\to {\mathbb{R}}^d$ extracts a $d$-dimensional feature vector from the image X.

The extracted feature vectors are passed to the RF classifier, which consists of an ensemble of N decision trees $T_i$:

(5) $$\hat{Y}=\frac{1}{N}\sum _{i=1}^N{T}_i(F)$$where $\hat{Y}$ represents the predicted class label.

Description of artificial intelligence models used

Artificial intelligence-based railway track image classification leverages advanced machine learning algorithms (Younas et al., 2024; Khalid et al., 2024; Rustam et al., 2024; Raza et al., 2023, 2024) to automate the detection and classification of railway track defects from images. By utilizing deep learning techniques, this technology enhances the accuracy and efficiency of maintenance operations, ensuring safer and more reliable railway systems through real-time monitoring and early fault detection.

• K-neighbors classifier (KNC): is a type of supervised machine learning algorithm, which is a category of non-parametric models (Al-Otaibi et al., 2024). KNC works on a straightforward concept: it determines the classification of a new data point by identifying the majority class among its $k$ closest neighbors. The key parameter, $k$, determines the number of neighbors to consider. The distance metric, often Euclidean, is used to compute the closeness among points. Given a set of training data $(x_i,y_i)$ where $x_i\in {\mathbb{R}}^n$ expressed the features of the ith sample and $y_i$ its associated class label, the classification of a new sample $x$ is determined by:

  (6) $$y=\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}(\{y_i:d(x,x_i)\le d(x,x_j)\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{l}\mathrm{e}\mathrm{a}\mathrm{s}\mathrm{t}k\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}\mathrm{s}\mathrm{o}\mathrm{f}j\})$$where $d(x,x_i)$ value is the distance between the sample point $x$ and a point $x_i$ in the training set.

• Logistic regression (LR): is a statistical method that uses the logistic function to predict the probability of occurrence of an event by fitting image data (Rahmatinejad et al., 2024). It is particularly useful in the situation of fault detection in railway tracks where the outcomes are typically binary as ‘Fault’ or ‘No Fault’. In LR, the probability that an outcome y value is 1 given predictors x is modeled as the logistic function of a linear combination of the predictors. The LR model is given by the following equation:

  (7) $$p(y=1|\mathbf{x})=\frac{1}{1+e^{-({\beta }_0+{\beta }_1{x}_1+\cdots +{\beta }_n{x}_n)}}$$where:

  • –

    $p(y=1|\mathbf{x})$ data is the probability of the track having a fault given predictors $\mathbf{x}$.

  • –

    ${\beta }_0,{\beta }_1,\dots ,{\beta }_n$ are the parameters of the model.

  • –

    $x_1,\dots ,x_n$ are the feature values of the track data.

  • –

    $e$ is the base data value of the natural logarithm.

• Support vector classification (SVC): is a supervised machine learning method (Shoghi et al., 2024) that can be applied to classify railway track segments as either ‘faulty’ or ‘non-faulty’ based on various input features derived from sensor data, such as vibration levels, acoustic signals, and other track condition indicators. The basic principle behind SVC is to find the ideal hyperplane that best separates the data value points into different classes. For railway track fault detection, the input features of each track segment (data points) are mapped into a high-dimensional data space. SVC then attempts to find the hyperplane that maximally separates the faulty and non-faulty classes. The aim of SVC is to solve the following optimization problem:

  (8) $$\underset{\mathbf{w},b}{min}\frac{1}{2}{\mathbf{w}}^{\mathrm{\top }}\mathbf{w}$$ (9) $$\mathrm{s}.\mathrm{t}.y_i({\mathbf{w}}^{\mathrm{\top }}{\mathbf{x}}_i+b)\ge 1,\mathrm{\forall }i,$$where $\mathbf{w}$ value is the normal vector to the hyperplane space, b is the bias value term, ${\mathbf{x}}_i$ are the feature vectors, $y_i$ are the labels associated with each feature vector (1 for non-faulty and $-1$ for faulty), and $i$ indexes over all the data points.

• Random forest classifier (RFC): combines multiple DTs during construction to improve the overall performance of the model (Amiri et al., 2024). It is particularly effective in dealing with high-dimensional datasets and complex data structures, making it suitable for applications like fault detection in railway tracks. RFC functions by creating numerous DTs during the training phase and producing the class that represents the most frequent class predicted by the individual trees. The classification decision in a RFC is typically given by taking the majority vote across all trees. Mathematically, this can be represented as:

  (10) $$y=\mathrm{m}\mathrm{o}\mathrm{d}\mathrm{e}\left(\{y_1,y_2,\dots ,y_n\}\right)$$where $y$ is the predicted class, and $y_1,y_2,\dots ,y_n$ are the classes predicted by each of the $n$ trees in the forest.

• Convolutional neural network (CNN): is a deep networks (Lee et al., 2024), most commonly applied to analyzing visual imagery data. They have proven very effective in areas such as image recognition and classification, paving the way for innovative applications in various fields, including railway track fault recognition. In the context of railway track fault detection, CNNs can analyze visual data from track images to identify defects such as cracks, breaks, and other irregularities. The output layer in a CNN typically involves a softmax function. The softmax function converts the output scores from the final layer into probability values. The mathematical expression for the softmax data function is:

  (11) $$P(y=k|\mathbf{x})=\frac{e^{z_k}}{{\sum }_{j=1}^K{e}^{z_j}}$$

  Here, $P(y=k|\mathbf{x})$ is the probability that input $\mathbf{x}$ belongs to class $k$, $z_k$ represents the input to the softmax data function from the last fully connected layer of the network, and K is the total values of classes.

Hyperparameter optimization

In our study, we optimized the parameters of various applied machine learning methods through a handcraft recursive training and testing process, utilizing a k-fold validation approach to ensure robust and unbiased model performance. Hyperparameter optimization is a critical step in our methodology, allowing us to fine-tune the models for better accuracy and generalization. Table 2 presents the optimal hyperparameters identified for each method. This rigorous hyperparameter tuning process enabled us to enhance the performance of railway track fault detection. We selected the hyperparameters based on their effectiveness in achieving high performance scores for railway track fault detection.

For CTGAN, we used the following training parameters: batch size = 500, generator learning rate = 2e-4, discriminator learning rate = 2e-4, latent dimension = 128, PacGAN number (pac) = 10, and stopping criteria = fixed epoch limit.

Assessment metrics

To evaluate the performance of the machine learning approaches, several metrics were employed, including accuracy, recall, F1 score, and precision score. The detailed mathematical explanation of each metric used is shown in Table 3. Here, the terms TP refer to true positive, FP refers to false positive, TN refers to true negative, and FN refers to false negative.

Computing infrastructure

The experimental setup for this study involves implementing machine learning techniques using the Python programming language. All experiments were conducted in an online environment, specifically Google Colab, which provided a GPU backend with NVIDIA T4 and compute capability of 7.5 to facilitate the computational requirements. The Google Colab environment was configured with a 90-GB value of disk space and a 13-GB value of RAM, ensuring sufficient resources for data processing and model training.

Results of classical CNN

In Fig. 6, the performance of a classical CNN model is evaluated for railway track fault recognition using time series data. The results include metrics such as accuracy, loss, recall, and precision scores, which were monitored throughout the training process. The time series graph revealed that during the initial five epochs, the loss scores for both training and validation were relatively high, indicating initial model instability. Similarly, accuracy, precision, and recall scores were low during these initial epochs. However, as the training progressed, the CNN model updated its weights effectively, leading to a gradual decrease in loss scores and a corresponding improvement in accuracy, precision, and recall. This trend demonstrates the model’s learning curve and its ability to enhance performance over time, ultimately achieving better results in detecting faults in railway tracks.

The unseen testing results of the classical CNN method, as depicted in Table 4, demonstrate robust performance metrics across different target classes. The overall accuracy achieved is 0.97. When broken down by target class, the CNN method shows a precision score of 0.98, a recall score of 0.97, and an F1 score of 0.97 for the Non-Defective class. For the Defective class, the precision score is 0.96, the recall score is 0.97, and the F1 score is 0.97. The average performance across both classes also yields a precision, recall, and F1 score of 0.97. These results indicate that the classical CNN method is highly effective in accurately distinguishing between defective and non-defective items. The consistent performance across all metrics suggests a balanced model that does not favour one class over the other. However, there is potential for further improvement to push the accuracy beyond the current 0.97.

Results of machine learning methods

Table 5 presents a comparative analysis of different machine learning methods used for detecting faults in railway tracks. The KNC attained an overall accuracy score of 96%, with precision, recall, and F1 scores each at 0.96 for both defective and non-defective track conditions, reflecting strong balanced performance across categories. The LR and SVC methods both significantly outperformed KNC, achieving an overall accuracy of 99% along with perfect precision, recall, and F1 scores of 0.99 for both defective and non-defective states, demonstrating high efficacy in fault detection. However, the standout performer is the RFC, which not only matched the highest accuracy of 99% but achieved a perfect precision and recall score of 1.00 for defective track conditions. This indicates that the proposed RFC can identify all actual defective cases without falsely categorizing any non-defective tracks as defective, which is critical in preventing both unnecessary maintenance costs and potential accidents due to missed defects. For non-defective tracks, RFC recorded precision and recall scores of 0.99, maintaining exceptional reliability in identifying true negative cases. These results highlight the effectiveness of RFC in railway track fault detection, suggesting it is the most robust model among those tested.

Figure 7 presents the confusion matrix validation analysis of the applied approaches for railway track fault detection. The analysis highlights the error rate of each method. The KNC had 31 incorrect predictions, LR had 8, RF had 4, and SVC had 9 incorrect predictions. This analysis validates that the proposed RF model achieved a minimal error rate, indicating its superior performance in railway track fault classification with a high correct classification rate. The significantly lower error rate of the RF model underscores its effectiveness and reliability in identifying faults accurately compared to the other methods evaluated.

K-fold cross validations analysis

For the validation of machine learning methods performance for railway track fault detection, a K-fold validation technique is employed. The analysis in Table 6 is conducted using a 10-fold validation scheme for each method to ensure the stability and generalizability of the results. The KNC achieved an accuracy of 96% with a standard deviation of ±0.0092, indicating high performance with relatively low variability in prediction outcomes across different subsets. Logistic regression (LR) and SVC both exhibited an even higher accuracy of 98%, with standard deviations of ±0.0063 and ±0.0073 respectively, suggesting robustness in model predictions. Notably, RFC demonstrated the highest accuracy among the evaluated models, achieving a score of 99% with a standard deviation of ±0.0081. This result underscores the effectiveness of RFC in handling complex patterns in data about railway track anomalies, thus providing a highly reliable tool for fault detection in critical transportation infrastructure.

Computations complexity analysis

Table 7 provides a summary of the computational complexity analysis for the methods applied. The runtime computations, measured in seconds, reveal significant differences in the performance of each method. The KNC demonstrated the fastest computation time at 0.02110 s, however with a low accuracy score. The LR method required 0.28747 s, showing a moderate computational demand. The RFC, while offering robust predictive power, exhibited the highest computational cost at 2.70125 s. These results highlight the trade-offs between computational efficiency and potential predictive accuracy of railway track fault detection.

State of the art comparison

The state-of-the-art results comparison of our proposed novel study is shown in Table 8. Our research demonstrates superior performance when compared to existing methodologies. Shafique et al. (2021) achieved a 97% accuracy using random forest (RF) and decision tree (DT) models. Eunus et al. (2024) reported a 93% accuracy with their ECARRNet approach. Mahale & Gaikwad (2024) obtained a 98% accuracy utilizing a combination of InceptionV3, ResNet50V2, and VGG16 models. In contrast, our proposed model, which integrates CTGAN with RF, achieved a remarkable 99% accuracy for railway track fault detection.

In addition, a recent study (Bird et al., 2022) used GAN data augmentation for fruit defect image classification, achieving a highest accuracy of 88%. This performance is lower compared to our proposed approach.

Ablation study

In this study, we conducted an ablation study to evaluate the performance of classical CNN methods and a proposed RFC approach in detecting defective track conditions. The classical CNN method achieved an overall accuracy of 0.97. When examining the performance by target class, the CNN method demonstrated a precision score of 0.98, a recall score of 0.97, and an F1 score of 0.97 for the Non-Defective class. For the Defective class, the precision score was 0.96, the recall score was 0.97, and the F1 score was 0.97. Despite these high scores, there is a noticeable performance gap using the classical approach, particularly in achieving perfect classification.

In contrast, the proposed approach utilizing RFC emerged as the standout performer, as illustrated in Fig. 8. It not only matched the highest accuracy of 99% but also achieved perfect precision and recall scores of 1.00 for defective track conditions. This indicates that the proposed RFC approach can identify all actual defective cases without falsely categorizing any non-defective tracks as defective, thus demonstrating its superior capability in precise defect detection and reducing false positives. The results underscore the efficacy of the RFC approach in improving detection accuracy and reliability compared to the classical CNN method.

Statistical significance analysis

Our research evaluates the effectiveness of the proposed CTGAN+RF approach using statistical analysis. Performance differences with other models are assessed via t-statistics and p-values. CTGAN+RF significantly outperforms CNN (t = −18.989, p = 6.0046) and KNC (t = −25.999, p = 7.2749), confirming its superiority.

Limitations

In our proposed model, the KNC method demonstrated the fastest computation time of 0.02110 s but with a notably low accuracy score. This suggests that while KNC is efficient in terms of speed, it may not be the most reliable option for railway track fault detection. The LR method required 0.28747 s, indicating a moderate computational demand while balancing efficiency with accuracy. On the other hand, the RFC offered robust predictive power but at the cost of the highest computational time of 2.70125 s.

These findings underscore the inherent trade-offs between computational efficiency and predictive accuracy in detecting railway track faults. Recognizing these limitations, future work will focus on optimizing the model’s layer stack to simplify the architecture, thereby reducing computational costs while striving to maintain or improve accuracy.

Implementation challenges

Implementing the proposed model in a real-time railway monitoring system is feasible, provided that sufficient computational resources and real-time data acquisition mechanisms are available. The model can be deployed as part of an edge computing system installed on railway inspection vehicles or integrated into cloud-based platforms for centralized monitoring and decision-making. The following challenges can be faced:

• The effectiveness of the model relies on high-quality, real-time data acquisition from sensors, cameras, and drones, which may be affected by environmental conditions, hardware limitations, and data transmission delays.

• Railway infrastructure conditions evolve due to wear, weather effects, and repairs.

• Running complex machine learning models on real-time railway monitoring systems requires computational efficiency, particularly if resources are limited in mobile setups.