Gated attention based generative adversarial networks for imbalanced credit card fraud detection

Data preprocessing

Data preprocessing mainly involves a data normalization, as defined as: (1)${\displaystyle x^{\prime }=\frac{x-min\left(x\right)}{max\left(x\right)-min\left(x\right)},}$

where x represents the original data, x′ represents the normalized data, min(x) and max(x) are the minimum and maximum values of x, respectively.

GA-GAN for data generation

When using GA-GAN for data generation, we adopt training data to train our GA-GAN for producing synthetic data. To this end, four primary components: training the generator, training the discriminator, iterative training and gated attention unit, are included, as depicted in Fig. 2. These four components are described below.

(1) Training the generator

Motivated by the advantage of variational autoencoder (VAE) (Pinheiro Cinelli et al., 2021), we design a VAE-based generator G with a similar structure to VAE. Specifically, the training data X is initially fed into a encoding network which maps X into a latent representation space. The encoding network consists of two FC layers, in which the first layer is equipped with a ReLU activation function to produce sparse activations, and the second layer provides the mean value µand the variance value σ², respectively.

Then, based on the µand σ², a random noise $Z=\left\{z^1,z^2,\dots ,z^m\right\}$ with a feature dimension of m is reparameterized to generate latent representations. Finally, a decoding network, similar to the encoding network, is leveraged to produce synthetic credit card transactions data that can deceive the discriminator. To ensure that the learned distribution accurately approximates the prior distribution of input data, the generator aims to minimize the following loss function: (2)${\displaystyle L_G=min\left(-E_{X\sim P_{data}\left(X\right)}\left[log{p}_{\theta }\left(X|G\left(Z\right)\right)\right]+\beta \cdot D_{KL}\left(q_{\varphi }\left(G\left(Z\right)\right)|X\parallel p\left(G\left(Z\right)\right)\right)\right),}$ where p_θ(X|G(Z)) represents the likelihood of the data X for the obtained output G(Z) of the generator G.D_KL(q_ϕ(G(Z)|X) ∥ p(G(Z))) is the Kullback–Leibler (KL) divergence between the approximate posterior q_ϕ(G(Z)|X) and the prior distribution p(G(Z)). The parameter β controls a trade-off between the reconstruction fidelity and the regularization term.

(2) Training the discriminator

The real data X and synthetic data G(Z) are mixed together and fed into the model’s discriminator. In the discriminator, input data are initially fed into three convolutional layers for feature extraction, each of which has a convolutional kernel of 1 × 3. Then, a gated attention unit is designed for feature enhancement, which focuses on capturing key clues while suppressing redundant details in feature representations. Subsequently, flattening and average-pooling (AvgPool) operations are performed, followed by three FC layers. The loss function of the discriminator D is defined as: (3)${\displaystyle L_D=min\left(-E_{X\sim P_{data}\left(X\right)}\left[logD\left(X\right)\right]-E_{Z\sim P_Z\left(Z\right)}\left[log\left(1-D\left(G\left(Z\right)\right)\right)\right]\right).}$

(3) Iterative training

The aforementioned two steps are repeated iteratively, continuously updating the parameters of both the generator G and the discriminator D, until the model converges. The convergence is reached when the generator G can produce synthetic fraudulent transactions that sufficiently deceive the discriminator D, while D is capable of accurately identifying the real fraudulent transactions and the synthetic ones. In this sense, the purpose of the iterative training process is to solve the resulting minimax optimization problem: (4)${\displaystyle \underset{G}{max}\underset{D}{min}L\left(D,G\right)=E_{X\sim P_{data}\left(X\right)}\left[logD\left(X\right)\right]+E_{Z\sim P_Z\left(Z\right)}\left[log\left(1-D\left(G\left(Z\right)\right)\right)\right].}$ (4) Gated attention unit

Inspired by the advantage of gated attention mechanisms (Xue, Li & Zhang, 2020; Niu, Zhong & Yu, 2021; Hua et al., 2022), we design a gated attention unit (GAU) based on gated attention mechanisms, as illustrated in Fig. 2. The detailed steps of GAU is listed below.

Initially, a linear layer is utilized to transform an input data I to a new tensor Z_h with a feature dimension of 2H, where H is the number of hidden layer neurons. This process is expressed as: (5)${\displaystyle Z_h=\text{Linear}\left(I\right).}$

Then, through a chunk operation, Z_h is split into two tensors: V and G_w, each of which has the same feature dimension H. Among them, V denotes the value matrix, and G_w is the gated weights. Additionally, the input data I is fed into a linear layer to produce the query (Q) and key (K) matrices with a dimension of H, as defined as: (6)${\displaystyle \left(Q,K\right)=\gamma \cdot \text{Linear}\left(I\right)+\beta ,}$ where γ and β are are learnable scaling factors and biases, respectively.

Next, we compute the similarity between each query Q and key K, and then obtain the attention scores A through a ReLU activation function and a squared operation, as expressed as: (7)${\displaystyle A=ReL{U}^2\left(\frac{Q\cdot K^T}{\sqrt{H}}\right).}$

Subsequently, a dropout layer is employed to prevent overfitting. In this case, the attention matrix A aims to capture the relationships between the elements of the sequence in a refined manner. Then, multiplying each value vector by its corresponding attention weight is performed to adjust the contribution of the value matrix V, as expressed as: (8)${\displaystyle \text{Att}\text{\_}\text{scores}=\text{Dropout}\left(A\right)\cdot V,}$ where the weighted output Att_scores reflects the relationships between different elements in input data. Then, Att_scores is multiplied in an element-wise way by the corresponding gating vector G_w for producing the final weighted values, as defined as: (9)${\displaystyle O_w=\text{Att}\text{\_}\text{scores}\odot G_w.}$

Finally, a linear layer is leveraged to transform the feature dimension of O_w into the appropriate dimensionality for the next layer.

Effect of GAU

To demonstrate the effectiveness of the used GAU in the discriminator, Fig. 3 shows a comparison of the obtained results on the European and Card Fraud datasets, when using GAU or not. When removing the GAU, our method is implemented by a FC layer.

As shown in Fig. 3, the obtained results with GAU clearly outperform the results achieved without GAU. In particular, on the European dataset, the obtained F1-score without GAU is 0.754. In contrast, the obtained F1-score with GAU is 0.854, indicating a increase of 0.1 over the case without GAU. On the Card Fraud dataset, the achieved F1-score with the GAU is 0.989, yielding an improvement of 0.37 over the case without GAU. This indicates that the GAU enhances the model’s ability to distinguish between fraudulent and non-fraudulent transactions.

Effect of the structure of generator

This work designs a VAE-based generator with the similar structure to VAE (Pinheiro Cinelli et al., 2021). To validate the effectiveness of VAE-generator in our GA-GAN, Fig. 4 illustrates a performance comparison of detection results on two datasets. It can be seen from Fig. 4 that VAE-based generator obtains a significant improvement in terms of F1-score over the original generator (Goodfellow et al., 2014) without VAE, in which the generator is implemented by three FC layers. In particular, on the European dataset, VAE-based generator gives a F1-score of 0.854, whereas the generator without VAE yields a F1-score of 0.791. On Card Fraud dataset, VAE-based generator improves a F1-score from 0.970 to 0.989 compared with the generator without VAE. This demonstrates the advantage of VAE-based generator in our GA-GAN for data generation.

Effect of synthetic data

To intuitively demonstrate the impact of synthetic data generated by GA-GAN on credit card fraud detection tasks, we compare the confusion matrices obtained from training with and without synthetic data across two datasets, as shown in Fig. 5. The results indicate that on the original European dataset (Fig. 5A), our model achieves a F1-score of 0.770, suggesting a certain level of false negatives (missed fraudulent transactions), where actual fraud cases are not effectively identified. In this case, 46 fraud cases are not correctly classified. However, after training with synthetic data generated by GA-GAN (Fig. 5B), the obtained F1-score is improved significantly to 0.875. In this case, 24 fraud cases are not correctly classified. This demonstrates that GA-GAN not only enhances the identification of fraudulent transactions but also maintains high accuracy for normal transactions. Similarly, in the Credit Fraud dataset, the original F1-score is 0.983 (Fig. 5C), which is promoted to 0.989 (Fig. 5D) when using the augmented dataset with GA-GAN, approaching near-perfect classification. In this case, the number of fraud cases, which are not correctly recognized, is reduced from 627 to 382. This further validates the effectiveness of GA-GAN in reducing both false positives and false negatives. Moreover, these findings highlight the potential of GA-GAN as a powerful tool in addressing class-imbalanced credit card fraud detection problems. By generating high-quality synthetic data, GA-GAN effectively mitigates the challenges posed by data imbalance, significantly enhancing the overall performance of our model.