Economic insights into credit risk: a deep learning model for predicting loan repayment capacity

Introduction

Predicting loan repayment capacity is a critical issue in financial management, particularly in corporate financial management. A borrower’s ability to repay a loan is influenced by a multitude of factors, including socioeconomic, educational, behavioral elements, and others. These factors vary significantly depending on the context and type of enterprise or business activity. Numerous studies in the academic literature have explored these aspects, contributing to the development of models that aim to more accurately predict repayment capacity.

Specifically, several socioeconomic factors have been found to impact loan repayment capacity. For example, age, education level, marital status, and household characteristics were identified as significant factors influencing the repayment behavior of microfinance clients in Ethiopia. Older borrowers generally demonstrated better repayment capacity due to accumulated experience and more stable sources of income (Abdulahi, Charkos & Haji, 2022). Moreover, household size was found to have a negative impact on repayment ability, suggesting that larger families may have more strained financial resources (Ojiako, 2012). A similar pattern was also observed in studies on cooperative farmers, where factors such as net farm income and the number of years of cooperative membership positively correlated with repayment capacity, while age and level of education showed negative influences (Ilavbarhe, Alufohai & Keyagha, 2017).

In the business environment, income diversification is considered a key factor in enhancing loan repayment capacity. For instance, multiple income streams from both agricultural and non-agricultural activities can serve as a financial buffer against shocks, thereby improving borrowers’ ability to adhere to repayment schedules. For example, the study by Okonta, Achoja & Okuma (2023) on poultry, fish, and arable crop farmers in Nigeria demonstrated a clear hierarchy in repayment capacity based on income diversity and the effectiveness of management practices. Microfinance institutions also emphasize the importance of integrating credit assessment mechanisms to evaluate a borrower’s financial health, particularly their ability to generate stable cash flows from multiple income sources (Okero & Waweru, 2023). Credit risk assessment is another crucial factor in predicting loan repayment, involving an evaluation of the borrower’s credit history and financial condition, typically reflected in credit scores obtained from credit reference bureaus (Okero & Waweru, 2023). A systematic evaluation of these elements enables better insight into default likelihood, allowing financial institutions to adjust their lending strategies accordingly.

Computer-based models employing advanced statistical methods, such as logistic regression and machine learning algorithms, have also been applied to improve the accuracy of loan repayment prediction (Abdulahi, Charkos & Haji, 2022). Large-scale modelling has also proven effective for policy-relevant forecasting in other domains, reinforcing the importance of principled model design and validation for risk prediction (Nguyen et al., 2021). Cultural and institutional contexts also significantly shape repayment behavior. The influence of social capital and group lending models has been recognized as instrumental in facilitating repayment success. In Indonesia, community lending models involving collaborative participation among borrowers showed higher repayment rates, largely due to social pressure from group members to fulfill loan obligations (Indriani et al., 2023; Abdirashid & Jagongo, 2019). Additionally, geographical and economic conditions, including credit accessibility, also play a vital role in assessing repayment capacity within microfinance frameworks (Dang et al., 2022; Enimu, Eyo & Ajah, 2017).

In a comparative analysis of different contexts, subtle differences in repayment performance between cooperative members and microfinance beneficiaries highlight the importance of contextual adaptation in financial decision-making (Gudde Jote, 2018). Studies show that when institutions adjust loan terms based on local practices and borrower histories, repayment rates can improve significantly, emphasizing the need for tailored approaches in lending strategies. Moreover, mechanisms for monitoring loan repayments also play a key role in predicting and enhancing repayment capacity. Flexible repayment schedules, as opposed to rigid ones, can facilitate better compliance with repayment obligations. The study by Field & Pande (2008) indicates that while repayment frequency itself may not significantly affect default rates, flexibility in scheduling can promote more sustainable financial behavior. Furthermore, promoting financial literacy among borrowers has been shown to improve their ability to manage loans effectively, leading to better repayment outcomes (Kariuki & Namusonge, 2024).

With increasing demand for credit risk assessment, we propose a deep learning-based approach that integrates diverse information such as socioeconomic characteristics, credit history, behavioral factors, and income structure to provide more accurate predictions of loan repayment capacity. Our design is informed by advances in multimodal representation learning, e.g., unified frameworks that couple sequence encodings with predicted structural views via contrastive objectives to boost generalization (Nguyen et al., 2024), and we analogously integrate heterogeneous borrower signals (demographics, credit history, behavioural traces) within a single architecture. The proposed model not only contributes academically by offering deeper insights into borrower behavior, but also has strong practical value in supporting financial institutions to build more precise and context-adaptive credit risk assessment systems. This represents an important step toward developing sustainable lending strategies, minimizing credit risk, and promoting long-term financial stability.

Beyond its academic contributions, this study also carries significant practical implications. For commercial banks and credit institutions, a more accurate prediction of loan repayment capacity enables the development of risk-sensitive lending policies, reducing default rates and improving portfolio quality. By integrating diverse borrower information into a unified predictive framework, financial institutions can refine credit scoring systems, tailor loan products to specific client segments, and design more flexible repayment plans that align with borrower conditions. Furthermore, in the broader context of financial risk management, such predictive models help institutions comply with regulatory requirements, strengthen early-warning mechanisms, and enhance resilience against systemic risks, thereby fostering a more sustainable and inclusive financial ecosystem.

Related work

The rapidly growing field of machine learning and deep learning has witnessed numerous important applications in predicting loan repayment capacity. Recent studies emphasize the effectiveness of various algorithms in enhancing prediction accuracy, thereby improving decision-making processes in loan approvals. In 2022, a prominent study by Abdulahi, Charkos & Haji (2022) demonstrated that the implementation of machine learning techniques, particularly the Random Forest algorithm, significantly improved the accuracy of predicting the repayment capability of small and medium-sized enterprises in Malaysian financial institutions. This aligns with the findings of Addo, Guegan & Hassani (2018), who asserted that machine learning and deep learning models have considerably enhanced the credit risk analysis framework through advanced data-driven predictive capabilities. The integration of these models has enabled the effective utilization of large datasets, allowing for more accurate assessments of default probabilities compared to traditional methods. Furthermore, the study by Wu (2022) highlights the increasing reliance of banks on machine learning to improve both the speed and accuracy of default predictions, signaling a pivotal shift from manual evaluations to automated systems. In 2023, Jumaa, Saqib & Attar (2023) emphasized the potential of deep learning methods in refining credit risk assessment frameworks, asserting that consumer loan default prediction models developed using these techniques deliver higher accuracy and better identification of borrower risks. For example, research by Liu et al. (2023) demonstrated that Bayesian deep learning can significantly enhance predictive outcomes by identifying subtle variations in default indicators—factors that are critical for lenders. The optimal application of different machine learning models is also discussed by Zhou (2023), who compared the performance of models such as Logistic Regression, Decision Trees, and XGBoost, thereby illustrating their capabilities in contextualizing borrowers’ asset values and financial conditions.

Additionally, alternative approaches have emerged in recent literature. For instance, Netzer, Lemaire & Herzenstein (2019) focused on textual analysis of loan applications, showing that qualitative insights extracted from applicants’ textual data can substantially enhance the assessment of repayment behavior. This innovative intersection between natural language processing and financial forecasting reflects a trend of complementing traditional numerical analysis with qualitative indicators, providing a more holistic view of borrower risk. Similarly, Kvamme et al. (2018) integrated diverse methodological frameworks, employing convolutional neural networks (CNNs) to predict mortgage defaults with impressive results, showcasing the flexibility of deep learning models in handling complex datasets. The study by Xu, Lu & Xie (2021) on P2P lending emphasized the benefits of using machine learning methods in assessing repayment risks, effectively leveraging robust datasets to fine-tune prediction criteria. As the field continues to advance, blending machine learning with deep learning appears to be an emerging trend. For instance, Li, Tian & Zhuo (2023) explored blended models to improve the accuracy and stability of credit default predictions. Interest in ensemble methods is also on the rise, with (Hamori et al., 2018) comparing ensemble learning methods with various neural network architectures. Their results indicate that although deep learning provides advanced predictive capabilities, ensemble methods like Random Forest still offer competitive accuracy and reliability. Such integrated approaches demonstrate how traditional algorithms can be enhanced by modern technologies, providing financial institutions with more powerful toolsets for predicting repayment capacity.

In practical applications, the research by Bature et al. (2023) illustrates the development of a credit default prediction system using machine learning, clearly showing how theoretical models can be operationalized into useful applications for financial institutions. This transition from model to real-world implementation is crucial, reflecting the growing trend of applying advanced analytics in day-to-day lending processes. However, challenges remain. The need for more robust feature extraction techniques in deep learning, as discussed by Aniceto, Barboza & Kimura (2020), highlights the inherent complexity of credit data. This complexity calls for ongoing research and innovation in feature selection and extraction as fundamental components in the development of reliable predictive models. As the landscape of credit risk assessment continues to evolve, deepening the integration of machine learning and deep learning technologies holds great promise for achieving higher predictive accuracy in loan repayment forecasting. Studies by Turiel & Aste (2019) recognize the potential of AI-based models to recalibrate lending practices, underscoring the importance of technological adaptation in modern financial services.

Material and methodology

Data description and pre-processing

We utilized a dataset from LendingClub (https://www.kaggle.com/datasets/jeandedieunyandwi/lending-club-dataset), which includes both approved and rejected loan applications. This dataset serves as a valuable resource for analyzing credit risk and predicting loan repayment behavior. It contains borrower demographic information, loan characteristics, and creditworthiness indicators such as FICO scores. Key features include loan amount, interest rate, loan term, borrower income, debt-to-income ratio, FICO score, loan purpose, and repayment status. This dataset supports various tasks such as credit risk modeling, loan approval prediction, and economic behavior analysis.

Given the complexity of the original data, we applied several preprocessing techniques before using it for model training and evaluation:

• Feature integration: The original data sources consisted of multiple files containing complementary information. We merged these sources into a single dataset by matching records using a unique borrower identifier.

• Handling missing and invalid values: The dataset contained a significant number of missing values (NaN) and invalid entries (e.g., abnormally large values). Missing values were imputed using the mean of the respective feature. For invalid entries, we applied a filtering strategy: if more than 30% of the records in a feature were invalid, the feature was discarded; otherwise, only the affected rows were removed.

After analyzing the processed dataset, we observed an imbalance between negative and positive labels. The number of data points with negative labels was significantly higher than those with positive labels. There are many potential issues and risks associated with using an imbalanced dataset. If we train a classifier on such data, the model may classify all minority class instances as belonging to the majority class, potentially resulting in high test accuracy. However, such a model could be meaningless, as it would lack the ability to recognize samples from the minority class.

After preprocessing the data, we randomly split the dataset into three subsets: training, validation, and test, with respective ratios of 0.6, 0.1, and 0.3. The purpose of this split is to train the model effectively while ensuring reliable evaluation and tuning. The training set is used to fit the model parameters, the validation set is used to optimize hyperparameters and prevent overfitting, and the test set is used to assess the final performance of the model on unseen data. The statistical distribution of the dataset across these subsets is illustrated in Fig. 1.

Proposed architecture

Our proposed method is a Transformer-based architecture tailored for tabular data, consisting of three main stages: Feature Encoding and Gated Transformer Processing and Prediction Module, as illustrated in Fig. 2. The design effectively integrates both categorical and numerical features into a unified embedding space and leverages gated self-attention mechanisms to capture complex feature interactions.

Encoding data

To encode the data before training the model, we construct a processing block called the Feature Encoder. Figure 3 illustrates the data encoding process. Categorical features $x_{cat}$ are tokenized and mapped to embedding vectors via an embedding table:

(1) $${\tilde{e}}_c=\mathrm{E}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{d}\mathrm{d}\mathrm{i}\mathrm{n}\mathrm{g}(\mathrm{T}\mathrm{o}\mathrm{k}\mathrm{e}\mathrm{n}\mathrm{i}\mathrm{z}\mathrm{e}(x_{cat}))\in {\mathbb{R}}^{d_{\mathrm{e}\mathrm{m}\mathrm{b}}}.$$

Numerical features $x_{num}$, typically scalars or low-dimensional vectors, are first normalized and then projected into the embedding space through a shared linear layer:

(2) $${\tilde{e}}_n=W_n\cdot \mathrm{L}\mathrm{a}\mathrm{y}\mathrm{e}\mathrm{r}\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}(x_{num})+b_n,W_n\in {\mathbb{R}}^{d_{\mathrm{e}\mathrm{m}\mathrm{b}}\times d_{\mathrm{i}\mathrm{n}}}.$$

All features are then combined via concatenation, along with a special embedding token $[cls]$:

(3) $$E={\tilde{e}}_c\oplus {\tilde{e}}_n\oplus e_{[\mathrm{c}\mathrm{l}\mathrm{s}]}\in {\mathbb{R}}^{(n+m+1)\times d_{\mathrm{e}\mathrm{m}\mathrm{b}}},$$where $n$ is the number of categorical features, and $m$ is the number of numerical features.

Gated transformers

The Gated Transformer is a variant of the Transformer (Vaswani et al., 2017) architecture originally introduced in the field of natural language processing. It is modified and fine-tuned to better suit the characteristics of tabular data features. The Gated Transformer consists of two main components: multi-head self-attention layers and gated feedforward layers. The input representation Z is first processed by a multi-head self-attention mechanism to capture interactions among features:

(4) $$Z_{\mathrm{a}\mathrm{t}\mathrm{t}}=\mathrm{M}\mathrm{u}\mathrm{l}\mathrm{t}\mathrm{i}\mathrm{H}\mathrm{e}\mathrm{a}\mathrm{d}\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{n}(Z)=[{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_1,{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_2,\dots ,{\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_h]W^O,$$where

(5) $${\mathrm{h}\mathrm{e}\mathrm{a}\mathrm{d}}_i=\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}(Z\times W_i^Q,Z\times W_i^K,Z\times W_i^V),$$Here, Z = E corresponds to the input embedding of tabular data described in Section Encoding Data, $W^O\in {\mathbb{R}}^{d\times d}$ is the output projection matrix, and $\{W_i^Q,W_i^K,W_i^V\}\subset {\mathbb{R}}^{d\times \frac{d}{h}}$ are the learnable projection matrices for the $i$-th attention head. The attention output $Z_{\mathrm{a}\mathrm{t}\mathrm{t}}$ is then modulated by a token-wise gating mechanism. Specifically, a gating vector $g$ is computed as:

(6) $$g=\sigma (Z_{\mathrm{a}\mathrm{t}\mathrm{t}}^{(l)}{w}^G),$$where $\sigma (\cdot )$ denotes the sigmoid function, $w^G\in {\mathbb{R}}^{d\times 1}$ is a learnable parameter, and $g^{(l)}\in [0,1{]}^n$ controls the activation strength of each token embedding. The gated output is then passed through two linear projections and combined as follows:

(7) $$Z_{out}=\mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}(g\odot Z_{\mathrm{a}\mathrm{t}\mathrm{t}})\oplus \mathrm{L}\mathrm{i}\mathrm{n}\mathrm{e}\mathrm{a}\mathrm{r}(Z_{\mathrm{a}\mathrm{t}\mathrm{t}}),$$where $\odot$ denotes element-wise multiplication, and $\oplus$ indicates residual addition. The resulting output $Z\in {\mathbb{R}}^{n\times d}$ serves as the input for the MLP layer.

Results and discussion

Conclusion

In this article, we introduced a novel Gated Transformer-based deep learning model tailored for tabular data in the domain of credit risk assessment. By effectively integrating categorical and numerical features into a unified embedding space and leveraging multi-head self-attention with token-wise gating, our model captures complex feature interactions and improves interpretability, addressing key challenges in economic predictive modeling. Experimental results demonstrate that our approach outperforms advanced tabular data models across multiple metrics, including accuracy, AUPRC, MCC, and F1, highlighting its robustness and predictive power. This work provides a flexible and powerful framework for extracting meaningful economic insights from complex credit datasets, facilitating more informed and responsible financial decision-making. In addition, future research should systematically evaluate the model’s transferability across diverse datasets and financial contexts to address the current limitation of platform dependency. Another important avenue lies in developing robust bias detection and mitigation strategies to prevent the reinforcement of historical socioeconomic disparities. Finally, simplifying the model architecture while preserving predictive performance may enhance its acceptance in highly regulated financial environments.

Supplemental Information