Deep context-attentive transformer transfer learning for financial forecasting

Problem statements and literature review

Machine learning and deep learning have made significant strides in stock index price prediction, yet several challenges remain. This research addresses the following key issues:

• (1)

  Stock market price movements are inherently nonlinear and nonstationary, meaning their statistical properties, such as mean and variance, fluctuate over time. Effective forecasting models must balance short-term dynamics with a broader understanding of long-term trends. In addition, stock prices exhibit cyclical and seasonal patterns, adding complexity to predictions. This study focuses on financial time-series data to explore forecasting methods that address these temporal challenges.

• (2)

  Global financial markets are increasingly interconnected due to advancements in information dissemination, corporate crossholdings, and international market influences. While these factors shape broader trends, this study focuses on the individual behavior of stock index prices, setting aside complex interdependencies between markets.

Conventional stock price forecasting relies on mathematical models such as ARIMA and generalized autoregressive conditional heteroskedasticity (GARCH), which support volatility analysis and trend identification (Dadhich et al., 2021). GARCH with mixed data sampling has been applied to enhance volatility forecasting in the Chinese stock market (Yu & Huang, 2021). While effective in structured environments, linear models struggle with nonlinearity and non-stationarity, limiting adaptability in dynamic markets. Machine and transfer learning address these limitations by capturing short-term fluctuations and long-term trends. Recent studies highlight nonlinear forecasting methods, with SVM and RF improving prediction accuracy (Henrique, Sobreiro & Kimura, 2018). Least squares SVM (LSSVM) demonstrates strong forecasting performance but has high computational costs (Gong et al., 2019). RF models have been used for predicting clean energy ETFs, outperforming logistic regression methods (Sadorsky, 2021). To enhance financial forecasting, the proposed 2CAT model leverages deep learning architectures, adapting to complex market behaviors.

Deep learning (DL) enhances stock market forecasting by modeling complex nonlinear relationships through hierarchical structures (Gupta & Katarya, 2021). RNNs are effective for time-series prediction but suffer from vanishing or exploding gradients. LSTM networks address this issue using memory cells and gating mechanisms, improving retention of relevant patterns (Mehtab, Sen & Dutta, 2021). Gated recurrent units (GRUs) further enhance training efficiency and enable feature extraction from industry-specific datasets (Chung et al., 2014; Chen, Xue & Xing, 2023). However, both LSTMs and GRUs face memory constraints when handling long historical sequences.

Convolutional neural networks (CNNs) provide strong feature extraction capabilities and improve stock trend prediction. Hybrid CNN-LSTM and CNN-BiLSTM-Attention models refine forecasting accuracy by integrating spatial and sequential patterns (Lu et al., 2020; Zhang, Ye & Lai, 2023). CNNs have also been used with attention mechanisms and GRUs in smart grid applications (Li, 2023). Despite their benefits, hybrid models increase computational complexity and pose risks of overfitting.

The transformer model captures long-range dependencies and enables parallel processing, making it effective for financial time-series forecasting (Vaswani et al., 2017). Originally developed for natural language processing, it has been adapted for stock prediction by integrating social media data with market trends (Zhang et al., 2022). Studies show transformers outperform traditional methods for major indices like CSI 300 and S&P 500, though they require large datasets and may overfit smaller financial data (Wang et al., 2022).

TL reduces training time and enhances financial forecasting, particularly in data-scarce environments such as emerging markets or unstable economic conditions. By leveraging insights from mature markets, TL improves predictive accuracy forecasting models (He, Pang & Si, 2019). Deep learning applications have integrated TL with industrial chain data using multilayer perceptron (Wu, Wang & Wu, 2022). Selective TL with adversarial training has demonstrated efficiency across datasets (Li, Dai & Zheng, 2022), while TL-based regression techniques refine accuracy and optimize transaction frequency (Merello et al., 2019).

Previous studies have successfully applied time-series forecasting models to financial datasets, demonstrating their ability to identify predictive patterns in stock trends. While research has explored short-term dynamics, long-term trends, and cyclical patterns, their simultaneous integration remains underexamined. Existing approaches often focus on specific markets or industries, overlooking broader market interactions that could enhance predictions. These limitations present an opportunity to develop a novel solution that aligns with the research objectives and contributions of this study, which will be outlined in the following section.

Research objectives and contributions

This section presents the research objectives and contributions to clarify the focus of this study. To address the problem statements outlined in the previous section, the research objectives are as follows:

• Enhance the ability to capture long-term, short-term, and cyclical patterns in time-series data.

• Utilize interconnections across domains to improve prediction accuracy through TL.

• Develop a TL model that ensures robust generalization for unseen datasets.

To achieve these objectives, the key contributions of this study include:

1. Proposing a TL model incorporating 2CAT for multiday prediction.

2. Designing a CNN-correlation-based attention mechanism to extract short-term patterns, long-term dependencies, and cyclical similarities in time-series data.

3. Applying TL within the 2CAT framework to improve prediction accuracy for unobserved datasets.

4. Validating the effectiveness of 2CAT through experiments on six public stock indices across different timeframes.

Conceptual overview of the proposed 2CAT technique

Before delving into the mathematical details, it is important to establish a high-level understanding of our approach and its significance for time-series forecasting. This overview aims to make the technical content that follows more accessible to researchers across various disciplines. Time-series data presents unique challenges compared to other types of sequential data. While standard transformer models excel at capturing dependencies in sequential data like text, they were not specifically designed to handle time-series characteristics such as trends, seasonality, and cyclical patterns that occur at various time scales. Our approach transforms the standard transformer architecture by incorporating signal processing techniques familiar to researchers in time-series analysis. These modifications create a specialized model that better captures the temporal dynamics inherent in time-series data:

1. Breaking down the signal (series decomposing block): We decompose the input time-series into trend and seasonal components—a fundamental technique in time-series analysis. This decomposition allows the model to process long-term trajectories and recurring patterns separately, similar to how analysts might manually separate these components when analyzing financial or climate data.

2. Local pattern recognition (1D CNN Block): We replace the standard linear projections with one-dimensional convolutional neural networks (1D CNNs). This modification helps identify localized patterns—such as sharp rises, falls, or fluctuations—that often contain critical information in time-series data like stock prices, sensor readings, or physiological signals.

3. Finding temporal relationships (correlation block): Instead of the standard attention mechanism, we implement correlation-based approaches that are particularly effective for time-series:

   • Autocorrelation: This detects how a time series relates to itself at different time lags, helping identify cyclical patterns such as daily, weekly, or seasonal effects.

   • Cross-correlation: This captures relationships between input and output sequences across various time shifts, essential for forecasting where current values may influence future outcomes with varying delays.

4. Optimal time lag detection (time-shift and concatenation): This mechanism identifies the most informative time delays for prediction—similar to how economists might determine that a leading indicator precedes economic changes by a specific number of months.

Signal processing techniques modified in standard transformer model

To implement the conceptual approach described above, we modified the original transformer framework by introducing two key architectural enhancements within the proposed 2CAT model. First, we added a series decomposing block that separates time-series data into its fundamental components. Second, we replaced the traditional attention mechanism with a correlation block and a 1D CNN block, enabling a more effective capture of temporal relationships in time-series data. The correlation block serves different functions depending on its location: in the encoder, it performs autocorrelation to identify patterns within a single sequence; in the decoder, it performs cross-correlation to identify relationships between different sequences. Figure 1 illustrates this modified CNN-based correlation transformer architecture, with the encoder and decoder modules outlined by dashed lines.

One-dimensional convolutional neural network

Traditional transformers use linear projections to create query, key, and value vectors. Instead, our model employs one-dimensional convolutional neural networks (1D CNNs) for this purpose. These 1D CNNs are particularly effective at capturing repeating short-term patterns in time series data.

After embedding and positional encoding, we process the input sequence through three parallel 1D CNNs with different parameter sets. These networks produce three distinct vector representations:

• $Q_{\mathrm{\Delta }}$ (query vectors)

• $K_{\mathrm{\Delta }}$ (key vectors)

• $V_{\mathrm{\Delta }}$ (value vectors)

The $\mathrm{\Delta }$-symbol indicates that these parameters belong to either the encoder (Δ = e) or the decoder (Δ = d) components. Each CNN applies standard convolutional operations to extract temporal features at different scales. These projections are then passed to the correlation block for further processing.

This 1D CNN approach replaces the standard linear projection layers found in the original transformer architecture, allowing our model to better identify and leverage temporal patterns in the data.

Auto- and cross-correlation mechanisms in encoder and decoder

Our model improves upon the standard attention mechanism by incorporating signal processing techniques that capture temporal relationships more effectively. We implement these correlations in the frequency domain for computational efficiency.

To integrate autocorrelation and cross-correlation, the conventional attention mechanism was modified (Fig. 2). Both correlations are performed in the frequency domain to reduce computational complexity in handling $Q_{\mathrm{\Delta }}$ and $K_{\mathrm{\Delta }}$. The fast Fourier transform (FFT) enables this transformation.

In the encoder, $Q_e$ undergoes FFT to produce $Q_e^F$. Its complex conjugate $Q_e^{\ast F}$ replaces $K_e$, and their multiplication results in autocorrelation, expressed in Eq. (3). This is then converted back to the time domain via inverse FFT, leading to Eq. (4), where $r_{Q_e{Q}_e}$ represents the autocorrelation.

(3) $$R_{Q_e^F{Q}_e^{\ast F}}=Q_e^F{Q}_e^{\ast F}$$

(4) $$r_{Q_e{Q}_e}=\mathrm{I}\mathrm{F}\mathrm{F}\mathrm{T}\left(R_{Q_e^F{Q}_e^{\ast F}}\right).$$

In the decoder, autocorrelation follows the process used in the encoder. Knowledge transfer occurs by passing $K_e$ and $V_e$ vectors. Cross-correlation is obtained by multiplying $Q_d$ and $K_e$ in the frequency domain, as shown in Eq. (5). Applying IFFT converts the cross-correlation back to the time domain, yielding Eq. (6), where $c_{Q_d{K}_e}$ represents the cross-correlation.

(5) $$C_{Q_d^F{K}_e^{\ast F}}=Q_d^F{K}_e^{\ast F}$$

(6) $$c_{Q_d{K}_e}=\mathrm{I}\mathrm{F}\mathrm{F}\mathrm{T}\left(C_{Q_d^F{K}_e^{\ast F}}\right).$$

The function $\mathrm{T}\mathrm{S}{\mathrm{C}}_V(\cdot )$ manipulates $V_{\mathrm{\Delta }}$ in both the encoder and decoder through time shift and concatenation (Wu et al., 2021). $V_{\mathrm{\Delta }}$ shifts left by $m$, with the truncated section appended to the end, as shown in Fig. 3, where $N$ represents the data length. The green box marks the truncated part, which is reattached to the end, highlighted in magenta. Equation (7) defines the time length, linked to feature count, where $j$ is the integer result of b times the logarithm of N, with b as a hyperparameter.

(7) $$j=\lfloor b\times \log N\rfloor .$$

Each shift is assigned a time length, denoted as $m_k$, where $k$ ranges from one to $j$. The accumulated length follows a consistent pattern: $m_2$ = $m_1+m_1$, $m_3$ = $m_1+m_2$, and the final length $m_k$ = $m_1+m_{k-1}$. This accumulation ensures $m_k$ remains less than $m_{k+1}$.

Autocorrelation and cross-correlation are computed using different values of $m_k$, where $k$ ranges from 1 to $j$, producing a series of correlation values. These values are sorted in descending order as ${\tau }_p$, with ${\tau }_1$ being the highest and ${\tau }_j$ the lowest. The sorting process is defined by the $\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{T}\mathrm{o}\mathrm{p}(\cdot )$ function in Eq. (8).

(8) $${\tau }_1,{\tau }_2,\dots ,{\tau }_j=\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{T}\mathrm{o}\mathrm{p}\left(r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left(m_1\right),r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left(m_2\right),\dots ,r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left(m_j\right)\right).$$

To simplify the explanation, autocorrelation $r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}$, is expressed, while cross-correlation replaces it with $c_{Q_{\mathrm{\Delta }}{K}_{\mathrm{\Delta }}}$. After deriving indices ${\tau }_1$ to ${\tau }_j$ via Eq. (8), $r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}$ is approximated using a Softmax function, as shown in Eq. (9), where ${\hat{r}}_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}$ represents the approximation.

(9) $${\hat{r}}_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left({\tau }_1\right),{\hat{r}}_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left({\tau }_2\right),\dots ,{\hat{r}}_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left({\tau }_j\right)=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}\left(r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left({\tau }_1\right),r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left({\tau }_2\right),\dots ,r_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left({\tau }_j\right)\right)$$

After obtaining Eq. (9), feature aggregation is performed to derive autocorrelation and cross-correlation attention. Autocorrelation attention is calculated by summing the product of $\mathrm{T}\mathrm{S}{\mathrm{C}}_V(\cdot )$ and ${\hat{r}}_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}(\cdot )$ in Eq. (10). Similarly, cross-correlation attention is the summation of $\mathrm{T}\mathrm{S}{\mathrm{C}}_V(\cdot )$ and ${\hat{c}}_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}(\cdot )$, as shown in Eq. (11).

(10) $$\mathrm{A}\mathrm{u}\mathrm{t}\mathrm{o}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}-\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}=\sum _{k=1}^j\mathrm{T}\mathrm{S}{\mathrm{C}}_V\left({\tau }_k\right){\hat{r}}_{Q_{\mathrm{\Delta }}{Q}_{\mathrm{\Delta }}}\left({\tau }_k\right)$$

(11) $$\mathrm{C}\mathrm{r}\mathrm{o}\mathrm{s}\mathrm{s}-\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}-\mathrm{A}\mathrm{t}\mathrm{t}\mathrm{e}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}=\sum _{k=1}^j\mathrm{T}\mathrm{S}{\mathrm{C}}_V\left({\tau }_k\right){\hat{c}}_{Q_{\mathrm{\Delta }}{K}_{\mathrm{\Delta }}}\left({\tau }_k\right).$$

Work procedures of 2CAT

The 2CAT framework follows a structured sequence to process time-series data effectively. The procedure consists of the following key stages:

Training approach for transfer learning

TL enables us to leverage knowledge from one domain to enhance performance in another, particularly valuable when working with limited target data. In our research, we apply TL to reduce the computational resources required and accelerate training compared to starting from scratch. TL and fine-tuning are fundamental techniques within deep learning that facilitate the transfer of knowledge from one problem to another (Zhuang et al., 2021).

The architectural framework of our proposed approach is depicted in Fig. 4. We denote the source dataset as ${\mathrm{D}}_{\mathrm{s}}$ and the target dataset as ${\mathrm{D}}_{\mathrm{t}}$. Our methodology follows a structured three-stage process designed to effectively extract features from ${\mathrm{D}}_{\mathrm{s}}$ and apply them to ${\mathrm{D}}_{\mathrm{t}}$. Table 1 provides a summary of these three stages, outlining their purpose and impact on the model learning process.

Dataset

This study was conducted on six stock index datasets: the US stock market and five other Asian stock markets. Daily trading information was retrieved from Yahoo Finance (https://finance.yahoo.com/). The data collection period spanned from February 1, 2021, to January 31, 2024. Each collected market dataset was stored in a 2D format, containing both text and numerical data, separated into five columns: trading date, opening price, highest price, lowest price, and closing price. Each price value belongs to the domain of real numbers ( $\mathbb{R}$). Each collected 2D dataset is equivalent to a matrix of size $N\times 5$, where $N$ is the number of trading days in each market. Before processing, each market dataset was converted to a numeric dataset by eliminating the trading date column (text type). The resulting 2D numeric dataset was then transformed into a 1D dataset by serially concatenating the columns for opening price, highest price, lowest price, and closing price. The data are summarized in Table 2.

Following standard practice, all datasets were split into training, validation, and test sets in chronological sequence, adopting a 7:1:2 ratio. The computational experiments were conducted on a laptop equipped with a 1.70 GHz Intel Core i5-12500H CPU, an NVIDIA RTX 3050Ti graphics card, and 16 GB of RAM. The software frameworks included Keras 2.3.1 and PyTorch 1.13.1, running on a Python 3.9 interpreter. The proposed 2CAT model employs Transfer Learning (TL) to improve efficiency by leveraging pre-trained representations, reducing computational demands compared to full-scale training from scratch. To ensure scalability beyond consumer-grade hardware, the model was successfully deployed on Google Colab, executing without observed issues in a cloud environment. While this confirms functionality in cloud-based settings, further examination is necessary to assess its scalability in larger-scale financial applications, which may require enhanced computing resources, such as distributed systems or high-performance GPU clusters, to handle extensive datasets and real-time processing.

State-of-the-art approaches in comparison

The TL model using the proposed 2CAT model was compared with three state-of-the-art (SOTA) approaches listed in Table 3. These SOTA models were chosen for their contemporary relevance and novelty. The study aimed to demonstrate that the TL model adopting the proposed 2CAT model is competitive with other SOTA techniques in the field of time-series forecasting. However, SOTA large language models (LLMs) were not included because they require extensive datasets. Testing them with only a few thousand data points would not fairly represent their capabilities and could impair their performance.

Attention mechanism analysis

Attention weight visualizations from five forecasting models—CNN-BiLSTM-Attention, CNN-GRU-Attention, Deep-Transformer, Informer, and the proposed 2CAT—are presented in Fig. 5. These visualizations provide insight into how each model interprets temporal dependencies in financial index data. CNN-BiLSTM-Attention (Fig. 5A) exhibits vertically stratified attention with emphasis on recent time steps, reflecting a strong temporal hierarchy where certain intervals consistently receive higher weights. In contrast, CNN-GRU-Attention (Fig. 5B) displays horizontal banding, suggesting uniform weighting across entire time steps, potentially indicating a sequential processing approach with limited point-to-point specificity. Deep-Transformer (Fig. 5C) introduces a semi-uniform distribution, predominantly weighted toward recent inputs, which, while less selective, allows for moderate differentiation across time points. Informer (Fig. 5D) further emphasizes temporal relationships through a gradient pattern, concentrating attention on the most recent data while progressively diminishing focus on earlier values, aligning with its design for sparse, long-sequence processing. Distinct from the others, 2CAT (Fig. 5E) reveals sparse and discontinuous attention patterns, marked by specific focal points rather than continuous regions, suggesting a prioritization of selective temporal interactions to capture precise dependencies rather than generalized trends. Taken together, these visualizations highlight structural differences in how each model allocates attention, shaping their ability to model financial time-series relationships with varying degrees of specificity and emphasis.

Performance analysis of forecasting methods

The experimental results for forecasting methods applied to six stock market indices (Dow Jones Industrial Average (DJIA), Nikkei 225 (N225), Hang Seng Index (HSI), Shanghai Stock Exchange (SSE), Bombay Stock Exchange (BSE), and the Stock Exchange of Thailand (SET)) were evaluated using mean squared error (MSE), mean absolute error (MAE), and coefficient of determination (R²). The analysis compared four established methods—CNN-BiLSTM-Attention, CNN-GRU-Attention, Deep-Transformer, and Informer—alongside the proposed 2CAT method, with transfer learning effects examined using DJIA as the source domain.

As shown in Table 4, the proposed 2CAT method consistently achieves lower error metrics across all indices. On DJIA, it records an MSE of 0.0655, MAE of 0.2023, and R² of 0.9169, outperforming Deep-Transformer (MSE: 0.1360, MAE: 0.2850, R²: 0.8274). Similar advantages appear for HSI, where the proposed 2CAT method achieves MSE of 0.0146, MAE of 0.0945, and R² of 0.8212.

Performance gaps are most evident in SET, where the proposed 2CAT method (MSE: 0.1606, R²: 0.9094) surpasses alternatives, which display negative R² values (−0.1656 to −1.5157). Deep-Transformer and Informer exhibit high volatility, particularly for N225 and BSE, indicating adaptation challenges across different market conditions.

Impact of transfer learning

The results from transfer learning using DJIA as the source domain are presented in Table 5. The proposed 2CAT method maintains performance advantages, consistently outperforming benchmark models across target indices. On HSI, transfer learning improves MSE from 0.0146 to 0.0137, indicating effective knowledge transfer.

CNN-GRU-Attention remains stable, while CNN-BiLSTM-Attention exhibits variability, especially on SET, where R² declines from −1.5157 to −2.1231. Deep-Transformer and Informer continue to struggle with N225 and BSE indices under transfer learning conditions.

Overall, TL yields mixed results, offering slight improvements for some indices while causing marginal performance reductions for others. These findings suggest that while DJIA serves as a valuable source domain, target indices retain unique characteristics that require further model adaptation.

Statistical tests and confidence intervals

To assess the effectiveness of the proposed 2CAT method, statistical analysis is conducted to compare its performance against baseline methods across multiple datasets. This section outlines the research question and hypotheses that guide the evaluation, using summarized performance metrics presented in Tables 4 and 5. These consolidated results form the basis for testing whether the proposed 2CAT method demonstrates a statistically significant improvement over baseline approaches.

Ablation study

This section presents an ablation study to evaluate the contribution of key components in the proposed 2CAT model for financial market forecasting. The experiments systematically assess the impact of architectural improvements over the base transformer, the effect of rotary positional encoding, and performance across different prediction horizons. Each component is individually examined to quantify its contribution to the overall model performance.

Cross-market generalizability

The numerical findings are visually supported by the bar charts in Fig. 6. The proposed 2CAT method, represented by dark and light blue bars, consistently achieves lower MSE and MAE values across all indices. The R² chart highlights consistently higher values, indicating better explanatory power for stock price movements. While established models perform well on some indices, they lack robustness across markets. The proposed 2CAT method maintains stable performance, enhancing generalizability for stock market forecasting applications.

Model sensitivity

Performance metrics reveal varying model sensitivity to market characteristics. Deep-Transformer and Informer show significant degradation on certain indices, such as N225, indicating adaptation challenges. In contrast, the proposed 2CAT method demonstrates stability across diverse conditions, suggesting improved robustness. Modest gains in transfer learning highlight difficulties in knowledge transfer across markets with different financial dynamics, emphasizing the need for models that balance generalizability with market-specific adaptability.

Key insight on transfer learning

The findings confirm that the proposed 2CAT method delivers statistically significant improvements over existing forecasting approaches. Furthermore, the transfer learning implementation provides measurable advantages over the non-transfer variant, reinforcing its potential for cross-market applications. These results, supported by statistical tests (Table S1), contribute to discussions on the efficacy of transfer learning in financial forecasting and highlight its relevance in multi-market modeling.

Theoretical and practical implications

This study offers both theoretical and practical contributions to financial time-series forecasting. The proposed 2CAT model, built on a context-attentive transformer architecture, demonstrates that financial markets benefit from models capable of capturing complex temporal dependencies, including long-term trends, short-term fluctuations, and cyclical patterns. The integration of signal decomposition, sparse attention mechanisms, and convolutional layers enhances the ability of the model to reflect non-linear and non-stationary behaviors observed in real-world financial data. The analysis of attention weight distributions suggests that domain-specific attention patterns provide improved forecasting accuracy. This supports the argument that selective and context-aware attention structures are more suitable for financial applications than uniformly distributed attention approaches.

On the practical side, the experimental results show that 2CAT performs reliably across a range of international stock indices, including those with high volatility. Its strong performance on the SET index, where other models failed, highlights its adaptability in challenging market conditions. This robustness makes it suitable for risk management, investment planning, and strategy development tailored to specific market dynamics. Notably, transfer learning demonstrated statistically significant improvements in forecasting accuracy, reinforcing its potential as an effective approach for leveraging knowledge across structurally different markets. These results indicate that while careful adaptation remains necessary, transfer learning can enhance model efficiency and predictive performance in financial applications.

Given its consistent predictive performance across multiple forecast horizons, 2CAT is well-suited for implementation in financial decision support systems, especially in scenarios relying on structured historical price data.

Limitations and future directions

The study lacks computational efficiency assessment and inference latency benchmarks across hardware configurations, critical for determining suitability in real-time financial applications. Guidelines for model retraining frequency in evolving markets are also absent. The architecture excludes sentiment analysis and external event integration, limiting predictive capacity to price-based signals. Preliminary experiments with macroeconomic indicators showed inconsistent correlations (Table S5), with temporal misalignment between economic and financial time-series data preventing their integration.

The South African market dataset was selected as an ideal case study for emerging market dynamics, avoiding high acquisition costs and licensing restrictions associated with more frequently studied markets. Its relative isolation from global financial centers allows examination of localized economic patterns with reduced external interference.

For comparative analysis, CNN-BiLSTM-Attention, CNN-GRU-Attention, Deep-Transformer, and Informer models were included. However, advanced architectures like Time2Vec and Temporal Fusion Transformer were excluded due to implementation constraints, creating a gap in the comparative evaluation.

Future research will focus on computational efficiency through hardware benchmarking, developing adaptive retraining frameworks based on market volatility, and integrating structured financial data with sentiment analysis capabilities. The study will explore distributed computing implementations for real-time applications and model pruning techniques to optimize inference time. The implementation of excluded models will be considered once aligned financial and macroeconomic features are promptly implemented.