CTCN: a novel credit card fraud detection method based on Conditional Tabular Generative Adversarial Networks and Temporal Convolutional Network

GAN and CTGAN

Generative adversarial network are generative models based on the theory of zero-sum games. GAN consist of a generator G and a discriminator D (Goodfellow et al., 2014). During training, the generator continually improves its ability to create fake data to deceive the discriminator, while the discriminator judges whether the input data is real or generated. The two components iteratively optimize each other until they reach a dynamic equilibrium. The generator finally generates simulated samples and completes data augmentation. The loss function of GAN is shown in Eq. (1): (1)${\displaystyle \underset{G}{min}\underset{D}{max}V\left(G,D\right)=E_{x\sim P_r}\left\{log\left[D\left(x\right)\right]\right\}+E_{z\sim P_z}\left\{log\left[1-D\left(G\left(z\right)\right)\right]\right\}}$ where x represents real sampling, P_r represents the real sampling distribution, z represents random noise, P_z represents random noise distribution, G(z) represents fake sample data generated by generator G, and D(⋅) represents the output value of the discriminator D.

Although GAN can generate synthetic samples that conform to the real data distribution, they are not suitable for generating tabular data. CTGAN is a generative model based on GAN that has been optimized for the generation task of tabular data (Xu et al., 2019). CTGAN takes into account the conditional information in tabular data, and uses special generator and critic structures as well as other techniques. The architecture of CTGAN is shown in Fig. 1.

CTGAN consists of two neural networks: a generator G and a Critic C (similar to the discriminator in classical GAN architecture). To overcome the non-Gaussian and multimodal distribution of continuous columns in tabular data, CTGAN employs mode-specific normalization. A conditional generator and training by sample are used to address the issue of imbalanced categories in discrete columns. Additionally, CTGAN incorporates some recent advances in GAN training, such as the loss function of WGAN-GP (Gulrajani et al., 2017) and the critic structure of PacGAN (Lin et al., 2018), to improve training stability and the quality of generated data. The loss function of CTGAN is shown in Eq. (2): (2)${\displaystyle L=E_{G\left(z\right)\sim P_g}\left[D\left(G\left(z\right)\right)\right]-E_{x\sim P_r}\left[D\left(x\right)\right]+\lambda E_{y\sim P_y}\left[{\left(\left|\right|{\nabla }_{y}D\left(y\right)\left|\right|-1\right)}^2\right]}$ where y represents the sample linearly interpolated to the real data x, λ represents the gradient penalty factor, P_r and P_g represent the distribution of real and generated data.

The credit card dataset is a type of tabular data that contains both data and classification information. CTGAN is specifically designed to generate tabular data and can effectively learn the distribution of credit card data, producing synthetic samples that conform to the real data distribution. This can be useful for data augmentation while maintaining data utility.

Temporal convolutional network

Temporal Convolutional Network is a type of convolutional neural network specifically designed for processing time-series data. Its structure includes causal convolutions, dilated convolutions, and residual connections (He & Zhao, 2019), as illustrated in Fig. 2.

In time series processing, causal convolution ensures that the model does not use future information when processing sequential data. Dilated convolution can expand the receptive field of convolution operations and increase the model’s perception range, while residual connections can speed up network convergence and improve model accuracy. These unique design features enable TCN to perform well in time series data processing.

To capture long-term historical information in time series processing, the depth of causal convolution can be increased or the convolution kernel can be enlarged (Li et al., 2020). However, increasing the size of the convolution kernel leads to an increase in the number of network weight parameters, and increasing the convolution depth can result in problems such as gradient vanishing, increased training complexity, and poor fitting effect. To overcome these issues, the concept of dilation was introduced into convolutional networks. Dilated convolution can increase the receptive field according to the dilation factor, enhancing the network’s learning and memory capabilities over longer periods of time (Yu & Koltun, 2015). A dilated causal convolution with a dilation factor d = 1,2,4 and a convolution kernel k = 3 is shown in Fig. 3.

Dilated convolution allows the filter to be applied to a region larger than the length of the filter itself by skipping part of the input (He & Zhao, 2019). For an input sequence X = (x₁, x₂, ⋅⋅⋅, x_T) of length T and a filter f:0, ⋅⋅⋅, k − 1, the dilated convolution operation F on element s in the sequence is shown in Eq. (3). (3)${\displaystyle F\left(s\right)=\left(X_{\ast d}f\right)\left(s\right)=\sum _{i=0}^{k-1}f\left(i\right)\cdot {X_s}_{-d\cdot i}}$ where d is the dilated factor, * indicates the convolution operation, k is the filter size, and s-d⋅ i is for the direction of the past.

Since the receptive field size of TCN depends on the network depth n, filter size k, and dilated factor d, it is crucial to ensure stability as TCN becomes deeper and larger. To address this issue, a generic residual module can be used instead of a regular convolutional layer, which includes two layers of dilated causal convolutions and non-linear mapping. Weight normalization and dropout are also added to each layer for network regularization (Bai, Kolter & Koltun, 2018).

Experimental dataset

We have used three different datasets from the real world to evaluate the proposed method. These datasets exhibit dissimilar transaction volumes and proportions of deceptive transactions, with the specificities expounded in Table 1.

The Europe dataset is a real credit card transaction dataset used in Dal Pozzolo et al. (2015), comprising 284,807 transaction records spanning two days of European credit card holders. The dataset includes 31 feature vectors, such as time, transaction amount, other attributes (V1 to V28) processed via PCA, and a “Class” attribute distinguishing fraudulent and normal transactions. The “Class” attribute takes a value of 0 for normal transactions and 1 for fraudulent transactions, with 492 fraud transactions present in the dataset.

The Taiwan dataset (Yeh & Lien, 2009) is a credit card user payment dataset obtained from the UCI machine learning repository. It includes 30,000 transaction records, of which 6,636 are default records. The dataset comprises 25 feature vectors, such as credit card user’s overdue payment, demographic factors, credit data, payment history, and bills. The “default.payment.next.month” attribute indicates whether the credit card is overdue and takes a value of 0 or 1.

The German dataset is the South German credit data released on the UCI database, consisting of 1,000 transaction records, of which 300 belong to customers with poor credit. The dataset contains 21 features representing customer financial status, such as financial record status, measures of prepayments, bank accounts or securities, business terms, installment payment rates of additional cash levels, property, age, and the number of existing credits.

Evaluation metrics

We used a confusion matrix to evaluate the performance of our model in detecting credit card fraud. The confusion matrix, shown in Table 2, is used to calculate several evaluation metrics.

In this matrix, TP indicates the number of actual normal transactions and predicted to be normal transactions, FP indicates the number of transactions that are actually fraudulent but are predicted to be normal, FN indicates the number of transactions that are actually normal but are predicted to be fraudulent, TN indicates actual fraudulent transactions and predicts the number of fraudulent transactions.

Using the parameters from the confusion matrix, we define the following evaluation metrics (Li et al., 2021b):

Accuracy is the probability of correctly classified transactions in all transactions and is defined as: (4)${\displaystyle Accuracy=\frac{TP+TN}{TP+TN+FP+FN}}$

Recall is the probability of being predicted as a normal transaction in the sample of normal transactions and is defined as: (5)${\displaystyle Recall=\frac{TP}{TP+FN}}$

Precision is the probability that a normal transaction in the sample of normal transactions is actually a normal transaction and is defined as: (6)${\displaystyle Precision=\frac{TP}{TP+FP}}$

F-Score is a metric that combines precision and recall and is defined as: (7)${\displaystyle F-\mathrm{Score}=\left(1+{\beta }^2\right)\cdot \frac{Precision\cdot Recall}{{\beta }^2\cdot Precision+Recall}}$

In Eq. (7), β is the coefficient that describes the relative importance of precision and recall. When β = 1, the importance of precision and the importance of recall are equivalent, and we obtain the F1-Score: (8)${\displaystyle F1-\mathrm{Score}=\frac{2\cdot Precision\cdot Recall}{Precision+Recall}}$

The AUC-ROC measures the generalization ability of the classification problem model. The ROC curve consists of two parameters: the TPR and the FPR, which are defined as: (9)${\displaystyle TPR=\frac{TP}{TP+FN}}$ (10)${\displaystyle FPR=\frac{FP}{TN+FP}}$

For different classification thresholds, we can obtain a series of TPR and FPR values to plot the ROC curve. The AUC is a measure of the area under the ROC curve from (0,0) to (1,1), which is one of the important metrics for performance evaluation of classification problems (Forough & Momtazi, 2022).

The Matthews Correlation Coefficient (MCC) is an important measure of classification performance for binary classification problems and is defined as: (11)${\displaystyle MCC=\frac{TP\cdot TN-FP\cdot FN}{\sqrt{\left(TP+FP\right)\left(TP+FN\right)\left(TN+FP\right)\left(TN+FN\right)}}}$

This metric can also be used to assess the classification performance of a binary classification problem when the samples are extremely unbalanced (Chicco, Tötsch & Jurman, 2021). Specificity is the probability that a sample of fraudulent transactions will be correctly predicted as fraudulent transactions: (12)${\displaystyle Specificity=\frac{TN}{\left(FP+TN\right)}}$ The combined normal transaction recall and fraudulent transaction recall metric, G-mean, is defined as follows: (13)${\displaystyle G-\mathrm{mean}=\sqrt{Recall\times Specificity}=\sqrt{\frac{TP}{TP+FN}\times \frac{TN}{TN+FP}}}$

This metric is useful for evaluating the performance of a classification model when the data is imbalanced (Aurelio et al., 2019).

Experimental results

Performance experiment of improved CTGAN

To evaluate the performance of the improved CTGAN in handling imbalanced data, we randomly generated a set of artificially imbalanced datasets using the Python 3.6 sklearn package, which consisted of 500 data points. Among them, the majority class had 399 samples, and the minority class had 101 samples. We then conducted experiments with undersampling RUS, oversampling SMOTE, hybrid sampling SMOTETomek, CTGAN, and improved CTGAN. Figure 5 shows the visualization results of these sampling methods on the dataset.

Figure 5A illustrates the distribution of the original data. We observe class overlap between minority and majority class samples, which leads to a blurred classification boundary. Figure 5B shows the data distribution after RUS processing, which randomly removes majority class samples to make the number of majority and minority class samples equal. However, this method may result in the loss of some useful information, and the class overlap still persists in the dataset. Figure 5C shows the data distribution after SMOTE processing, which generates new samples randomly along the lines connecting adjacent minority class samples. However, the generated new samples are similar to the original samples, and some of them fall into the majority class region, further exacerbating class overlap. Figure 5D shows the data distribution after SMOTETomek processing, which employs the Tomek link algorithm for data cleaning based on SMOTE, removing redundant data. We observe that SMOTETomek can eliminate redundant data, but the class overlap and blurred classification boundary persist. Figures 5E and 5F show the data distribution after CTGAN and improved CTGAN processing, respectively. Both methods generate new samples that are distributed similarly to the real samples. However, some of the new samples generated by CTGAN fall into the majority class region, exacerbating the overlap between minority and majority class samples. In contrast, improved CTGAN generates samples that conform to the real data distribution in the distribution area of the original samples, without generating noisy samples.

To further validate the performance of the improved CTGAN in handling imbalanced data, we conducted an experiment using TCN as the fraud detection algorithm to detect the dataset processed by CTGAN. We then compared the results with those of the other 10 credit card datasets. These 10 datasets included the original imbalanced dataset and nine balanced datasets processed by ROS, RUS, SMOTE, ADASYN, Near Miss(NN), NCL, SMOTEENN, SMOTETomek, and CTGAN sampling methods. We used F1-Score, MCC, and G-mean as evaluation metrics. F1-Score measures the accuracy and recall of the model, MCC measures the classification performance of the model, and G-mean measures the data imbalance. The experimental results are presented in Table 3.

Table 3:

F1-Score, MCC and G-mean based on different sampling methods.

Methods	Europe			Taiwan			German
	F1-Score	MCC	G-mean	F1-Score	MCC	G-mean	F1-Score	MCC	G-mean
Original	0.8026	0.8040	0.8677	0.4675	0.3886	0.5892	0.4533	0.3298	0.5691
RUS	0.0283	0.1082	0.9107	0.5250	0.3961	0.6738	0.5913	0.3942	0.7131
ROS	0.7034	0.7108	0.9090	0.4785	0.3030	0.6803	0.5784	0.3921	0.6970
NN	0.0037	0.0109	0.3456	0.3846	0.1408	0.5786	0.5690	0.3554	0.6937
NCL	0.8152	0.8153	0.8887	0.4955	0.3335	0.6819	0.5543	0.3239	0.6603
SMOTE	0.6302	0.6475	0.9066	0.5314	0.4030	0.6800	0.5748	0.3610	0.6958
ADASYN	0.5576	0.5881	0.9063	0.4700	0.2899	0.6738	0.5618	0.3420	0.6557
SMOTEENN	0.5520	0.5849	0.9100	0.4783	0.3031	0.6815	0.5530	0.3297	0.6575
SMOTETomek	0.7150	0.7205	0.9057	0.4955	0.3335	0.6819	0.5859	0.3798	0.7053
CTGAN	0.8037	0.8034	0.8922	0.5134	0.3881	0.6588	0.6042	0.4130	0.7243
Improved CTGAN	0.8187	0.8185	0.9108	0.5349	0.4060	0.6841	0.6108	0.4227	0.7300

DOI: 10.7717/peerjcs.1634/table-3

Notes:

The values in bold indicate the best results.

The experimental results presented in Table 3 demonstrate the superior performance of the improved CTGAN in handling imbalanced data. In particular, the datasets processed by the improved CTGAN achieved the best results in all three metrics of F1-Score, MCC, and G-mean, with significant improvements observed in the German dataset, where the metrics were improved by 15%, 9%, and 16%, respectively. This can be attributed to the fact that the improved CTGAN filters the original data by removing most of the overlapping majority class samples and generates minority class samples that conform to the true data distribution to construct a balanced dataset. It is worth noting that while CTGAN also performed well in terms of F1-Score and G-mean metrics, the newly generated samples by CTGAN tended to fall into the majority class region, exacerbating the overlap between majority and minority class samples and making the classification boundary between them blurred.

RUS, NN, and NCL construct a balanced dataset by removing majority class samples, and the three datasets processed by these three undersampling algorithms achieved good experimental results in the G-mean metric. However, undersampling algorithms can lead to the loss of some useful samples in the majority class, which can affect the detection results of the detection algorithm. For example, the F1-Score and MCC metrics of the Europe dataset after being processed by RUS were only 0.0283 and 0.1082, respectively. ROS, SMOTE, ADASYN, SMOTEENN, and SMOTETomek are five sampling algorithms that construct a balanced dataset by expanding the minority class samples. The experimental results of their processed data in F1-Score, MCC, and G-mean metrics were also better than those of the original dataset. However, these five sampling algorithms only start from the local neighborhood of the minority class samples and do not consider the overall distribution of the minority class samples. The generated data cannot effectively fit the true data distribution and can reduce the performance of the detection algorithm. For example, the F1-Score and MCC metrics of the Europe dataset after being processed by ADASYN were 0.5576 and 0.5881, respectively.

CTCN experiment

For the purpose of appraising the potency of our advanced methodology, we undertook a sequence of comparative experiments juxtaposing CTCN against other prevalent models employed in fraud detection, including but not limited to DT, LR, SVM, RF, CNN, ANN, GRU, LSTM, LSTM-Attention, RNN-LSTM, and CNN-GRU as detailed in references (Forough & Momtazi, 2021; Forough & Momtazi, 2022; Roseline et al., 2022; Karthika & Senthilselvi, 2023). We performed these experiments on three distinct datasets and used accuracy, recall, and F1-Score as the evaluation metrics. The results are presented in Table 4.

Table 4:

Accuracy, Recall and F1-Score based on different detection algorithms.

Methods	Europe			Taiwan			German
	Accuracy	Recall	F1-Score	Accuracy	Recall	F1-Score	Accuracy	Recall	F1-Score
DT	0.9993	0.7839	0.8141	0.8250	0.3480	0.4617	0.7454	0.3229	0.4246
LR	0.9991	0.5925	0.7058	0.8226	0.3185	0.4364	0.7909	0.5626	0.6101
SVM	0.9992	0.6111	0.7415	0.8233	0.3634	0.4701	0.7666	0.4895	0.5497
RF	0.9993	0.7160	0.7972	0.8235	0.3367	0.4514	0.7424	0.2291	0.3410
CNN	0.9993	0.7551	0.8000	0.8223	0.3578	0.4648	0.7363	0.6875	6027
ANN	–	0.7416	0.7648	0.8229	0.3854	0.4842	0.7696	0.4895	0.5529
LSTM	–	0.7408	0.7866	0.8238	0.3704	0.4756	0.7787	0.4687	0.5521
GRU	–	0.7208	0.7792	0.8250	0.3817	0.4848	0.7515	0.4895	0.5340
LSTM-Attention	0.9993	0.7901	0.8101	0.8232	0.3592	0.4671	0.7757	0.4687	0.5487
RNN-LSTM	0.9993	0.7777	0.8000	0.8224	0.3770	0.4780	0.7818	0.5208	0.5813
CNN-GRU	0.9993	0.7716	0.8143	0.8254	0.3629	0.4728	0.7909	0.5625	0.6101
CTCN	0.9993	0.8299	0.8187	0.7981	0.5381	0.5349	0.7181	0.7604	0.6108

DOI: 10.7717/peerjcs.1634/table-4

Notes:

The values in bold indicate the best results.

Based on the results presented in Table 4, our method achieved the highest accuracy on the Europe dataset, which is comparable to the performance of DT, RF, CNN, LSTM-Attention, RNN-LSTM, and CNN-GRU models. However, our method performed worse than the CNN-GRU model on the Taiwan dataset, and was inferior to the LR and CNN-GRU models on the German dataset. Given the imbalanced nature of credit card datasets, even if all transactions are classified as normal, the detection model may still achieve high accuracy. Therefore, accuracy alone is not the most informative indicator when evaluating imbalanced data classification. Instead, recall, which measures the proportion of actual normal transactions that are correctly identified as such, is a crucial metric for evaluating credit card fraud detection methods. Our method achieved the highest recall on all three datasets, indicating its superior coverage of minority classes. Additionally, the F1-Score, which combines both precision and recall, is a comprehensive evaluation index. Our method outperformed all comparison models in terms of F1-Score.

The AUC metric serves as an inherently intuitive means to assess the efficacy of the detection algorithms. We present the AUC comparative results between CTCN and eleven distinct detection algorithms, visually depicted in Figs. 6, 7 and 8.

As depicted in Figs. 6, 7 and 8, the AUC value of CTCN surpasses that of other machine learning and deep learning methods. These findings indicate that CTCN can significantly enhance the overall classification performance of the data.

To further assess the efficacy of CTCN, we have also generated ROC curves for various algorithms on the Taiwan dataset and German dataset, as shown in Fig. 9. For the Europe dataset, we employed the same evaluation metrics as those employed in Forough & Momtazi (2022) to make direct comparisons of experimental results between CTCN and the three algorithms, namely, ANN, GRU and LSTM in Forough & Momtazi (2022). Consequently, we do not provide its ROC curves here.

Figure 9A shows the ROC curves of different algorithms on Taiwan dataset. It can be seen that SVM has the smallest ROC curve coverage area, while CTCN has the largest ROC curve coverage area, and the remaining methods have similar ROC curve coverage areas. Figure 9B shows the ROC curves of different algorithms on the German dataset. It can be seen that the ROC curve coverage area of CTCN is superior to that of the other detection algorithms. The ROC curve coverage area of CNN-GRU is marginally inferior to that of CTCN, while the ROC curve coverage area of DT is the smallest. The larger the ROC curve coverage area, the better the classification performance of the algorithm. CTCN achieved the largest ROC curve coverage area for both the Taiwan dataset and German dataset. Thus, the classification performance of CTCN is superior. In conclusion, our proposed approach outperforms popular techniques such as DT, LR, SVM, RF, CNN, ANN, GRU, LSTM, LSTM-Attention, RNN-LSTM, and CNN-GRU in credit card fraud detection tasks. It should be noted that CTCN runs slightly slower than other fraudulent transaction detection methods due to its equalization of the dataset by improving CTGAN, but the overall detection accuracy of CTCN is better than other fraudulent transaction methods.