Enhanced detection of accounting fraud using a CNN-LSTM-Attention model optimized by Sparrow search

Data collection and preprocessing

The dataset used in this article is sourced from the CSMAR (Guotai’an) database, spanning the years 2014-2021. After matching the data, labels were assigned: a label of 0 indicates that a company did not commit financial fraud, while a label of 1 signifies that the company engaged in financial fraud during the specified year. The data was further refined by filtering relevant indicators and removing missing values and outliers to minimize the impact of data anomalies.

The data indicators in this article undergo dimensionality reduction using principal component analysis (PCA) combined with the Pearson correlation coefficient for collaborative processing. The primary goal of dimensionality reduction is to eliminate redundant and irrelevant information present in the high-dimensional original data, which can otherwise lead to reduced accuracy during the identification process.

PCA achieves dimensionality reduction by transforming the high-dimensional data into a lower-dimensional space through specific mapping. The core idea behind PCA is to preserve as much of the essential features from the original data as possible while maximizing the variance in the reduced dimensions, thereby minimizing redundant information.

The PCA process involves several steps. First, the sample data is centered by calculating the mean, followed by obtaining the covariance matrix. Then, the eigenvalues and eigenvectors of the covariance matrix are computed. The eigenvalues are sorted in descending order, and the corresponding top eigenvectors are retained. Finally, the original data is transformed into a new set of principal components, effectively reducing its dimensionality.

For dimensional vectors w in the target subspace n, the original data is transformed using the following mapping function to maximize variance: (1)${\displaystyle {max}_w\frac{1}{m-1}\sum _{i=1}^m{\left(w^T\left(x_i-\overline{x}\right)\right)}^2,}$ where m represents the number of samples, x_i is data sample ith, and $\overline{x}$ is the average vector of all data samples. PCA reduces the original dimension X to a K dimension, which can be expressed as Y = W^TX.

PCA dimensionality reduction optimizes the data structure to maximize the retention of essential information within the data’s internal characteristics. However, PCA tends to overlook the differences between samples, leading to variations in data projection after dimensionality reduction. To address this issue, this article incorporates the Pearson correlation coefficient for feature selection during data preprocessing.

The accounting fraud dataset for listed companies includes numerous indicators, such as company size, asset–liability ratio, current ratio, and quick ratio. However, the correlation between these indicators can negatively impact the accuracy of model training, resulting in poor discrimination and a lack of generalization. To mitigate this, the Pearson correlation coefficient method is used to calculate the correlation between multiple variables, as determined by the following formula: (2)${\displaystyle \rho =\frac{\mathrm{Cov}\left(X,Y\right)}{\sqrt{\mathrm{Var}\left(X\right)\mathrm{Var}\left(Y\right)}},}$ where ρ is the correlation between variables X and Y.

Rationale for model selection

In corporate financial accounting fraud detection, traditional methods like neural networks, logistic regression, and support vector machines are widely used. However, these models often fall short in delivering high accuracy due to the complex and evolving nature of fraud patterns. To overcome these challenges, this paper introduces an advanced approach that integrates CNN and LSTM networks. CNNs are effective at capturing spatial patterns and performing feature extraction, while LSTMs excel at handling temporal dependencies and sequence data, making this hybrid approach ideal for uncovering intricate fraud patterns in financial data. This hybrid approach has also demonstrated promising results in various other applications. For example, in time series anomaly detection, CNNs extract key features while LSTMs handle sequential dependencies, leading to effective irregularity detection. In healthcare, this combination has been used to analyze both spatial and temporal trends for predictive diagnosis (Nguyen et al., 2021), and in transportation systems, it has proven valuable for identifying fraudulent trips based on trajectory data (Ding et al., 2021; He et al., 2023). CNN-LSTM models have also shown significant potential in biological sequence data analysis (Le et al., 2019). For instance, they have been applied to predict protein structures and interactions, as well as to analyze DNA and RNA sequences by capturing complex spatial motifs and temporal dependencies inherent in biological data (Nguyen et al., 2019). These diverse applications highlight the robustness and adaptability of the CNN-LSTM hybrid model, making it highly suited to tackling the intricate and dynamic patterns found in financial fraud detection and beyond.

Moreover, we incorporate an attention mechanism to allow the model to prioritize the most relevant features, improving its ability to detect subtle anomalies that signal fraud. To maximize performance, we employ the SSA for parameter optimization, ensuring that the CNN-LSTM model operates at optimal efficiency. This method is specifically designed to overcome the limitations of traditional models and achieve higher detection accuracy for corporate accounting fraud.

Our proposed fraud detection framework follows a four-step process: data preprocessing, feature extraction from key indicators, internal change pattern identification, and hyperparameter optimization. We present the SSA-CNN-LSTM-Attention recognition model, which extracts and analyzes fraud-related characteristics and internal patterns in accounting data. By leveraging the strengths of neural networks and intelligent optimization algorithms, our approach aims to significantly improve detection accuracy. Figure 1 illustrates the architecture of our model, with further details provided in the subsequent sections.

Convolutional neural network

CNNs are used at processing multi-dimensional data and effectively extracting features from time series. Given that different indicators carry varying levels of importance, we developed a CNN-LSTM-Attention model to achieve a high-precision accounting fraud identification and classification system.

In our approach, the CNN is used for local feature extraction. It processes the input data through convolutional layers, refines the features using pooling layers, and then integrates the output via fully connected layers. The structure consists of convolutional layers, pooling layers, and fully connected layers. When data is fed into the model, the local blocks are convolved using a sliding window based on a predefined convolution size, which allows the model to capture the feature weights of the local regions. Pooling is then applied to eliminate features with smaller weights, removing redundant information and enhancing recognition accuracy.

However, during the convolutional feature extraction process, the model may overlook the correlation between features. Since the accounting fraud data of listed companies exhibits time series properties, understanding the dependencies between data points is crucial. To address this, we incorporated an LSTM network to extract and analyze these time series features.

The convolutional layer extracts features from the input data. During training, the network adjusts the kernel weights of the convolutional layer. This approach offers better storage efficiency and faster learning rates compared to other neural network models. Stacking multiple convolutional layers allows for the extraction of more complex features from the input data. The convolution kernel processes data by sliding from top to bottom and left to right, performing point-wise multiplication.

Following the convolutional layer is the pooling layer, which downsamples the data to re-extract features and simplify the network’s structure. This process helps to reduce computational complexity and retain essential information.

After the convolutional and pooling layers, the fully connected layer integrates the data. It connects to the neurons of the upper layer, converting the incoming data into a flat array. This array serves as the input for the fully connected layer, preserving a significant amount of useful information during the transformation.

In our method, the CNN architecture includes two convolutional layers, 2 max pooling layers, and a fully connected layer. Convolution is performed using 1-D operations, with the ReLU activation function applied in the convolutional layers and the Sigmoid function used in the fully connected layer. The CNN extracts H_c, which represents the feature vector: (3)${\displaystyle \begin{array}{c}{C}_1=\mathrm{ReLU}\left(H_t\otimes W_1+b_1\right),\hfill \\ P_1=max\left(C_1\right)+b_2,\hfill \\ C_2=\mathrm{ReLU}\left(P_1\otimes W_2+b_3\right),\hfill \\ P_2=max\left(C_2\right)+b_4,\hfill \\ H_c=\mathrm{Sigmoid}\left(P_2\otimes W_3+b_5\right),\hfill \end{array}}$ where C₁ and C₂ denote the outputs of the two convolutional layers, while P₁ and P₂ denote the outputs of the two pooling layers. W₁, W₂, and W₃ are the weight matrices, and b₁, b₂, b₃, b₄, and b₅ represent the biases. The symbol ⊗ indicates the convolution operation, and max refers to the maximum function.

LSTM neural network

The LSTM network, introduced by Hochreiter & Schmidhuber (1997), addresses the issues of vanishing and exploding gradients in traditional neural networks by incorporating memory cells and gating mechanisms. This innovation enables the network to retain long-term information and effectively handle dependencies and historical data in time series.

The LSTM network model is composed of a series of time series modules, each featuring an input gate, a forget gate, an output gate, and a memory unit. The input gate controls the information flow into the memory unit, the forget gate manages the retention of historical data, and the output gate regulates the information output. The forget gate updates the memory by evaluating the previous hidden state and the current input, determining how much of the past information should influence the current state. The input gate decides the amount of new information to store, using a sigmoid function to output values between 0 and 1; a value of 0 means no update is made, while 1 indicates that the information should be updated. The output gate determines how much of the information from the memory unit is passed to the next layer, also using a sigmoid function to output either 0 (no information passed) or 1 (information passed). This gating mechanism allows LSTM networks to selectively manage the flow of information, enabling the network to preserve and process relevant data while discarding irrelevant information.

The LSTM network can be expressed by the following formulas: (4)${\displaystyle \begin{array}{c}X=\left[\begin{array}{c}\hfill X_t\hfill \\ \hfill h_{t-1}\hfill \end{array}\right],\hfill \\ f_t=\delta \left(W_f\cdot X+b_f\right),\hfill \\ i_t=\delta \left(W_i\cdot X+b_i\right),\hfill \\ O_t=\delta \left(W_o\cdot X+b_o\right),\hfill \\ c_t=f_t\circ c_{t-1}+i_t\circ tanh\left(W_c\cdot X+b_c\right),\hfill \\ h_t=O_t\circ tanh\left(c_t\right),\hfill \end{array}}$ where X_t denotes the input at time t; h_t represents the hidden state at time t; W_f, W_i, W_o, and W_c are the weight matrices for the forget gate, input gate, output gate, and memory cell, respectively; b_f, b_i, b_o, and b_c are the corresponding biases for these gates and memory cell; ∘ denotes the dot product calculation; and δ represents the activation functions used in the network.

We employed the Adam algorithm to update the network weights. This algorithm iteratively adjusts the weights based on training samples, utilizing the second-order moment of the gradients to adaptively compute the learning rate for the parameters.

Attention mechanism

The attention mechanism is a model that mimics the way the human brain focuses on specific parts of information. In this mechanism, while input and output data are connected with varying weights in the hidden layer, the encoding output is incorporated into the attention range. The attention mechanism assigns different weights to input data features, amplifying certain information. This means that the concentrated information is received more sensitively and processed more quickly. In scenarios where LSTM takes too long to process data, its ability to retain information diminishes. By integrating the attention mechanism, which emulates how the human brain processes large volumes of information, the network’s information processing capabilities are enhanced. The attention mechanism alleviates the Encoder-Decoder model’s limitation on handling fixed-length vector data by calculating similarities. During this calculation, the input that is more similar to the target output receives a higher attention weight. For different outputs (y), the attention mechanism computes the similarity between each input (x) and the output (y), leading to different attention weights for each input. These weights are then used in a weighted summation, increasing the model’s flexibility.

The output vector from the LSTM layer is fed into the attention layer, where the probabilities corresponding to the LSTM output vector are calculated. The weight parameter matrix is then continuously updated as follows. (5)${\displaystyle \begin{array}{c}{e}_t=\mu tanh\left(w{h}_t+b\right),\hfill \\ a_t=\frac{exp\left(e_t\right)}{\sum _{j=1}^t{e}_j},\hfill \\ s_t=\sum _{t=1}^i{a}_t{h}_t,\hfill \end{array}}$ where e_t represents the attention probability distribution μ at time t; w denotes the weight coefficients; b represents the bias coefficients; a_t is the attention weight; and s_t is the output of the Attention layer at time t.

Model optimization using sparrow search

SSA is a recent advancement in swarm intelligence. It categorizes the sparrow population into two groups: discoverers and joiners. A fitness function is employed to calculate each sparrow’s fitness value, which in turn facilitates the exchange of roles and positions within the group. Unlike genetic algorithms, particle swarm optimization, or ant colony algorithms, SSA effectively mitigates the risk of converging prematurely to a suboptimal solution.

Let X represent the randomly initialized sparrow population, where (6)${\displaystyle X=\left(\begin{array}{cccc}\hfill x_1^1\hfill & \hfill x_1^2\hfill & \hfill \dots \hfill & \hfill x_1^d\hfill \\ \hfill x_2^1\hfill & \hfill x_2^2\hfill & \hfill \dots \hfill & \hfill x_2^d\hfill \\ \hfill ⋮\hfill & \hfill ⋮\hfill & \hfill \dots \hfill & \hfill ⋮\hfill \\ \hfill x_n^1\hfill & \hfill x_n^2\hfill & \hfill \cdots \hfill & \hfill x_n^d\hfill \end{array}\right),}$ d is the dimensionality of the population, and n is the number of sparrows. The fitness of each sparrow, F, is then defined as follows: (7)${\displaystyle F_x=\left(\begin{array}{c}\hfill f\left(\left[\begin{array}{cccc}\hfill x_1^1\hfill & \hfill x_1^2\hfill & \hfill \cdots \hfill & \hfill x_1^d\hfill \end{array}\right]\right)\hfill \\ \hfill f\left(\left[\begin{array}{cccc}\hfill x_2^1\hfill & \hfill x_2^2\hfill & \hfill \cdots \hfill & \hfill x_2^d\hfill \end{array}\right]\right)\hfill \\ \hfill ⋮\hfill \\ \hfill f\left(\left[\begin{array}{cccc}\hfill x_n^1\hfill & \hfill x_n^2\hfill & \hfill \cdots \hfill & \hfill x_n^d\hfill \end{array}\right]\right)\hfill \end{array}\right).}$

We have (8)${\displaystyle X_{i,j}^{t+1}=\left\{\begin{array}{c}\hfill X_{i,j}\cdot \left(\mathrm{exp\; -}\frac{\mathrm{1}}{\alpha \cdot ite{r}_{\max }}\right),\mathrm{if}{R}_2<ST\hfill \\ \hfill X_{i,j}+Q\cdot L,\mathrm{otherwise}\hfill \end{array}\right.}$ where t represents the current iteration number, iter_max is the maximum number of iterations, and X_i,j is the position of the ith sparrow in the jth dimension. Here, α, R₂ ∈ (0, 1] and ST ∈ (0.5, 1] denote the early warning value and safety value, respectively. Q represents random numbers following a Gaussian distribution, and L is a 1 × d matrix with all elements equal to 1.

When a discoverer within the sparrow population detects a predator, it will immediately raise an alarm. If the alarm value exceeds the safety threshold, the discoverer will update its location. Otherwise, it will continue searching within the space: (9)${\displaystyle X_{i,j}^{t+1}=\left\{\begin{array}{c}\hfill Q\cdot \mathrm{exp(}\frac{X_{worst}-X_{i.j}^t}{i^2}\mathrm{),if}i>\frac{n}{2}\hfill \\ \hfill X_p^{t+1}+\left|X_{i,j}-X_p^{t+1}\right|\cdot A^{+}\cdot L,\mathrm{otherwise}\hfill \end{array}\right.}$ where X_p represents the optimal position currently held by the discoverer, X_worst denotes the current global worst position, and A is a matrix where each element is randomly assigned a value of either 1 or −1, with A⁺ = A^T(AA^T)⁻¹.

During foraging, the joiner observes the finder. Once the finder locates food, the joiners immediately compete for it. If a joiner wins, it takes over the finder’s food; otherwise, it continues searching the space for food. The joiner updates its position according to the formula. If a joiner’s fitness value is low and it fails to obtain food, it will exchange positions to increase its chances of finding more food: (10)${\displaystyle X_{i,j}^{t+1}=\left\{\begin{array}{c}\hfill X_{\mathrm{best}}^{\mathrm{t}}\mathrm{+}\beta \cdot \left|X_{i,j}^t-X_{b\mathrm{est}}^{\mathrm{t}}\right|\mathrm{,if}{f}_i<f_g\hfill \\ \hfill X_{i,j}^t+K\cdot \left|\frac{X_{i,j}^t-X_{\mathrm{worst}}^{\mathrm{t}}}{\left(f_i-f_w\right)+\varepsilon }\right|,\mathrm{otherwise}\hfill \end{array}\right.}$ where $X_{\mathrm{best}}^{\mathrm{t}}$ is the global optimal position at the current iteration, β is a step control parameter that follows a N(0, 1) distribution, and K ∈ [ − 1, 1]. f_i represents the fitness value of the ith sparrow, while f_g and f_w denote the global optimal and worst fitness values, respectively. The term ɛ is an adjustment factor introduced to prevent the denominator from becoming zero and causing an undefined or infinite result.

The objective function is (11)${\displaystyle MAE=\frac{1}{n}\sum _{i=1}^n\left|y_i-{\hat{y}}_i\right|,}$ where n is the number of samples, y_i is the true value, and ${\hat{y}}_i$ is the predicted value.

The randomly initialized discoverers and joiners compete for food and continuously update their positions, searching for the sparrow with the highest global fitness value as the global optimum. This process continues until the maximum number of iterations is reached. As a result, the SSA is utilized to optimize the hyperparameters of the CNN-LSTM-Attention model. The process of SSA optimizing the hyperparameters of the CNN-LSTM-Attention model is illustrated in Fig. 2.

Assessment metrics

In the financial accounting of listed companies, accounting fraud has long been a significant concern. To effectively identify potential accounting fraud, discriminant detection technology has become an essential tool. When assessing the performance of a discriminant detection algorithm, several indicators are typically used to quantify its effectiveness, including detection rate (DR), false acceptance rate (FAR), accuracry (ACC), and F1-score. DR measures the percentage of abnormal data correctly identified as outliers. In the context of financial accounting, it reflects the successful identification of potential accounting fraud. The formula for calculating DR is: ${\displaystyle \text{DR}=\frac{\text{TP}}{\text{TP}+\text{FN}},}$ where TP (true positives) and FN (false negatives) represent the number of correctly identified fraudulent cases and the number of missed fraudulent cases, respectively. A high DR indicates that the model is effective in detecting potential fraud.

FAR represents the percentage of normal data values incorrectly classified as outliers. In the context of financial accounting detection, FAR indicates the proportion of legitimate situations mistakenly flagged as potential fraud. The formula for calculating FAR is: ${\displaystyle \text{FAR}=\frac{\text{FP}}{\text{FP}+\text{TN}},}$ where FP (false positives) and TN (true negatives) correspond to the number of incorrectly identified fraudulent cases and the number of correctly identified non-fraudulent cases, respectively. A lower FAR indicates that the model performs better in minimizing false alarms.

ACC refers to the proportion of correct detections, representing the model’s ability to accurately identify both positive and negative samples. The formula for calculating ACC is: ${\displaystyle \text{ACC}=\frac{\text{TP}+\text{TN}}{\text{TP}+\text{TN}+\text{FP}+\text{FN}}.}$

The F1-score is a metric that combines both precision and recall to provide a comprehensive measure of a model’s performance. The formula for calculating the F1-score is: ${\displaystyle \text{F1-score}=\frac{2\times \text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}}=\frac{\text{TP}}{\text{TP}+\frac{1}{2}\left(\text{FP}+\text{FN}\right)},}$ where precision represents the proportion of positive predictions that are correctly identified as positive, and recall represents the proportion of actual positive cases that are correctly identified. A higher F1-score indicates a better balance between precision and recall.