MSKT: multimodal data fusion for improved nursing management in hemorrhagic stroke

Background

Hemorrhagic stroke, characterized by intracerebral hemorrhage caused by non-traumatic rupture of blood vessels in the brain parenchyma, presents a significant clinical challenge in emergency departments and intensive care units (Balgude et al., 2024). Epidemiological data indicate that hemorrhagic stroke accounts for approximately 10–15% of all strokes, with an acute-phase mortality rate as high as 45–50%. Among survivors, around 80% suffer from severe neurological deficits. The primary pathophysiological mechanism involves rapid hematoma formation and expansion, leading to increased intracranial pressure, reduced cerebral blood flow, and exacerbated neurological dysfunction. Inflammatory responses play a significant role in this process, aggravating hematoma expansion and increasing the risk of neurological deterioration (Spronk et al., 2021; Huang et al., 2024).

A critical determinant of patient outcomes is the ability to predict and potentially prevent hematoma expansion, which is strongly associated with neurological deterioration and poor prognosis (Minhas et al., 2019; Li et al., 2022). However, despite its clinical importance, accurate prediction of hematoma expansion remains challenging due to the complex, dynamic, and non-stationary nature of its progression. Traditional prediction methods using static imaging markers or isolated clinical parameters often fail to capture the multidimensional and time-varying characteristics of this process (Fang et al., 2022; Onat et al., 2022).

Nursing interventions play a pivotal role in hemorrhagic stroke management, particularly in monitoring disease progression and implementing timely interventions to prevent complications (Theofanidis & Gibbon, 2016; Khattar et al., 2017). By formulating and implementing individualized nursing plans—such as managing intracranial pressure through postural adjustments and respiratory care—disease progression can be effectively controlled, promoting neurological recovery (Sun, Sun & Su, 2022; Jiang et al., 2023a; Fu et al., 2023). Effective nursing management requires real-time access to accurate predictive information that can guide personalized care plans.

With the advancement of data-driven healthcare, there is growing recognition that integrating multiple data sources—including vital signs, laboratory results, medication records, and nursing observations—could significantly enhance prediction accuracy and support more informed clinical decision-making (Alzakari et al., 2024; Teoh et al., 2024). This integration enables precise personalized efficacy evaluation and prognosis prediction, providing a scientific basis for the development of individualized nursing plans and further optimizing nursing outcomes.

The integration and analysis of such multimodal data, however, present substantial computational challenges. The data are typically high-dimensional, heterogeneous, temporally variable, and often contain significant noise and missing values. Traditional machine learning approaches struggle with handling these complexities, particularly the non-stationary time-series patterns that characterize hematoma evolution (Nikolić et al., 2024; Sarhan, Saif & Elshennawy, 2024). This creates a pressing need for advanced modeling techniques that can effectively process multimodal clinical data and provide reliable, real-time predictions to support nursing decisions in hemorrhagic stroke management.

Problem analysis

In clinical practice, hematoma expansion is influenced by multiple factors, including but not limited to blood pressure fluctuations, coagulation abnormalities, initial bleeding location, and volume. These factors can be quantified through multidimensional data from the MIMIC-IV database, such as ICU admission records, laboratory test results, and medication therapy records. To effectively capture the complex dynamic process of hematoma volume changes over time, we employ a non-stationary Gaussian process to establish a mathematical model that describes the evolutionary pattern of this time-volume relationship.

From a mathematical perspective, the time-volume relationship of hemorrhagic stroke can be viewed as a dynamically evolving system. Traditional static models struggle to accurately capture its rapidly changing characteristics, while non-stationary Gaussian process models can more precisely describe this dynamic process by defining time-varying mean functions and covariance functions. We utilize clinical indicators from the MIMIC-IV database (such as blood pressure, coagulation indicators PT, INR, and medication usage) as input features to fit the mean and covariance functions of the model. By analyzing the temporal patterns of these features, the model can dynamically adjust the predicted distribution of hematoma volume and output a conditional probability distribution at each time point, describing the possible values of hematoma volume and its uncertainty range.

Model construction

We propose a probabilistic framework to describe the temporal characteristics of hematoma volume changes. Considering the randomness and time-varying nature of hematoma expansion, we employ a non-stationary stochastic process for modeling. Let $V\left(t\right)$ represent the hematoma volume at time $t$, with its probability distribution expressed as:

(1) $$V\left(t\right)\sim N\left(M\left(t\right),\mathrm{\Sigma }\left(t\right)\right).$$where $M\left(t\right)$ represents the expected volume that changes over time, and $\mathrm{\Sigma }\left(t\right)$ represents the volume variability. This expression reflects the dynamic changes of hematoma volume over time and its uncertainty.

From a clinical perspective, we focus on whether the hematoma volume will exceed the clinically defined expansion threshold c within a time window $T$ from the initial time $t_0$. This can be transformed into the following probability problem:

(2) $$Target:P\left(\bigcup _{s\in \left[t_0,t_0+T\right]}\left\{V\left(s\right)\ge c\right\}\right).$$

According to the basic principles of probability theory, the above event and its complement constitute a complete event space:

(3) $$P\left(\mathbf{\bigcup }_{s\in \left[t_0,t_0+T\right]}\left\{V\left(s\right)\ge c\right\}\right)+P(\mathbf{\bigcap }_{s\in \left[t_0,t_0+T\right]}\{V\left(s\right)<c\})=1.$$

To solve this probability problem, we adopt a segmented time analysis method. The time interval $\left[t_0,t_0+T\right]$ is uniformly divided into N subintervals, each with a length of $\mathrm{\Delta }T=T/N$. When $N$ is large enough, it can be assumed that within each small time segment $\left[t_0+\left(i-1\right)\mathrm{\Delta }T,t_0+i\mathrm{\Delta }T\right]$, $M\left(t\right)$ and $\mathrm{\Sigma }\left(t\right)$ are approximately constant, making the process within that subinterval locally stationary.

Within each subinterval, we focus on the frequency of the event $\left\{V\left(t\right)\ge c\right\}$. Let $\alpha \left(t\right)$ be the average occurrence rate of this event per unit time, then the probability that this event does not occur in an interval of length $\mathrm{\Delta }T$ can be approximated by a Poisson process as:

(4) $$P\left(noeventoccursintheinterval\right)=e^{-\alpha \left(t\right)\mathrm{\Delta }T}.$$

Considering all subintervals, the probability that the event $\left\{V\left(t\right)\ge c\right\}$ never occurs throughout the entire time window $\left[t_0,t_0+T\right]$ is:

(5) $$P(\mathbf{\bigcap }_{s\in \left[t_0,t_0+T\right]}\{V\left(s\right)<c\})=\underset{N\to \mathrm{\infty }}{lim}\prod _{i=1}^N{e}^{-\alpha \left(t_0+i\mathrm{\Delta }T\right)\mathrm{\Delta }T}.$$

As $N$ approaches infinity, the above expression can be transformed into an integral form:

(6) $$P(\mathbf{\bigcap }_{s\in \left[t_0,t_0+T\right]}\{V\left(s\right)<c\})=e^{-{\int }_{t_0}^{t_0+T}\alpha \left(t\right)dt}.$$

Combining with Eq. (3), we obtain the target probability:

(7) $$P\left(\mathbf{\bigcup }_{s\in \left[t_0,t_0+T\right]}\left\{V\left(s\right)\ge c\right\}\right)=1-e^{-{\int }_{t_0}^{t_0+T}\alpha \left(t\right)dt}.$$

The core challenge lies in determining the relationship between $\alpha \left(t\right)$ and the model parameters $M\left(t\right)$ and $\mathrm{\Sigma }\left(t\right)$. At any moment t, the instantaneous probability of the event $\left\{V\left(t\right)\ge c\right\}$ occurring is:

(8) $$\beta \left(t\right)=P\left(V\left(t\right)\ge c\right)={\int }_c^{\mathrm{\infty }}{\displaystyle \frac{1}{\sqrt{2\pi \mathrm{\Sigma }\left(t\right)}}{e}^{-\frac{{(v-M\left(t\right))}^2}{2\mathrm{\Sigma }\left(t\right)}}dv.}$$

It can be proven that under stationary assumptions and sufficiently small time intervals, $\alpha \left(t\right)$ and $\beta \left(t\right)$ have the following relationship:

(9) $$\alpha \left(t\right)=\beta \left(t\right).$$

Substituting Eqs. (8) and (9) into Eq. (7), we get:

(10) $$P\left(\mathbf{\bigcup }_{s\in \left[t_0,t_0+T\right]}\left\{V\left(s\right)\ge c\right\}\right)=1-e^{-{\int }_{t_0}^{t_0+T}{\int }_c^{\mathrm{\infty }}{\displaystyle \frac{1}{\sqrt{2\pi \mathrm{\Sigma }\left(t\right)}}{e}^{-\frac{{(v-M\left(t\right))}^2}{2\mathrm{\Sigma }\left(t\right)}}dvdt}}.$$

Simplified as:

(11) $$P\left(\mathbf{\bigcup }_{s\in \left[t_0,t_0+T\right]}\left\{V\left(s\right)\ge c\right\}\right)=1-e^{-{\int }_{t_0}^{t_0+T}\beta \left(t\right)dt}.$$

Considering individual differences in patient conditions in clinical practice, we introduce a personalized modeling approach. Let $X_j$ represent the feature vector of patient $j$, including demographic features, baseline clinical parameters, and early imaging features. We construct two deep neural networks ${\phi }_1$ and ${\phi }_1$ to learn personalized mappings of $M\left(t\right)$ and $\mathrm{\Sigma }\left(t\right)$:

(12) $$M_j\left(t\right)={\varphi }_1\left(X_j,t;W_1\right)$$

(13) $${\mathrm{\Sigma }}_j\left(t\right)={\varphi }_2\left(X_j,t;W_2\right)$$where $W_1$ and $W_2$ are the parameter sets of the two neural networks.

To train these two networks, we construct a likelihood function based on observational data. Suppose at time point $t_k$, the hematoma volume observed for patient j is $v_j^k$, then the corresponding observation likelihood can be expressed as:

(14) $$L_j^k={\displaystyle \frac{1}{\sqrt{2\pi {\mathrm{\Sigma }}_j}}{e}^{-\frac{(v_j^k-M_j{\left(t_k\right)}^2}{2{\mathrm{\Sigma }}_j\left(t_k\right)}}.}$$

The overall log-likelihood function is:

(15) $$\mathcal{L}={\sum }_j{\sum }_{k}log\left(L_j^k+\delta \right).$$

Model training employs backpropagation and stochastic gradient descent optimization algorithms, with parameter update rules:

(16) $$W_1\leftarrow W_1-\eta {\mathrm{\nabla }}_{W_1}L$$

(17) $$W_2\leftarrow W_2-\eta {\mathrm{\nabla }}_{W_2}L$$where $\eta$ is the learning rate.

In clinical applications, hematoma expansion is typically defined as: compared to the baseline scan, an absolute increase in hematoma volume ≥12.5 ml or a relative increase ≥33%. Therefore, for patient $j$, if the initial hematoma volume is $v_j^0$, the critical threshold for hematoma expansion is defined as:

(18) $$c_j=max\left\{v_j^0+12.5,1.33\times v_j^0\right\}.$$

Using the trained model, we can predict the probability of hematoma expansion for patient j within the time window $\left[t_0,t_0+T\right]$:

(19) $$P_j=1-e^{-{\int }_{t_0}^{t_0+T}{\int }_c^{\mathrm{\infty }}{\displaystyle \frac{1}{\sqrt{2\pi {\mathrm{\Sigma }}_j\left(t\right)}}{e}^{-\frac{{(v-M_j\left(t\right))}^2}{2{\mathrm{\Sigma }}_j\left(t\right)}}dvdt}}.$$

In practical calculations, we use numerical integration methods to approximate the above integral:

(20) $$P_j=1-e^{-{\sum }_{i=1}^{N_t}{\beta }_j\left(t_i\right)\mathrm{\Delta }t},$$where $N_t$ is the number of time discretization points, $t_i$ is the discrete time point, $\mathrm{\Delta }t$ is the time step, and ${\beta }_j\left(t_i\right)$ is the instantaneous probability that patient j’s hematoma volume exceeds the threshold $c_j$ at time $t_i$.

Hybrid predictive model based on multi-scale kernel transformer architecture for non-stationary gaussian processes

Algorithm architecture design

When dealing with predictive tasks for non-stationary Gaussian processes, traditional modeling methods often face challenges related to the complexity and diversity of time-series data. Non-stationarity implies that the statistical properties of time-series data change over time. Standard Gaussian process models may exhibit high computational complexity and insufficient fitting capability when handling highly nonlinear and multi-scale dependencies. To address these challenges, this chapter proposes an innovative algorithm that integrates a hybrid model with multi-scale kernel functions and a Transformer architecture, aiming to achieve efficient prediction of complex non-stationary time-series data. The specific design is illustrated in the Fig. 1.

The proposed algorithm architecture combines the feature extraction capabilities of multi-scale kernel functions with the sequence modeling capabilities of the Transformer to effectively capture the non-stationary characteristics of the data. The input layer receives time-series data, which is mapped to a high-dimensional feature space through an embedding layer. This embedding incorporates more semantic information, allowing the kernel functions to capture subtle patterns. Standardization ensures feature consistency and comparability, while dimensionality adjustment optimizes the processing effectiveness of the multi-scale kernel functions.

In the multi-scale kernel function layer, data is processed through various kernel functions: the linear kernel function captures linear relationships, the polynomial kernel function identifies complex dependencies, the radial basis function (RBF) kernel function handles noise and short-term fluctuations, and the Matérn kernel function is suitable for modeling local variations. The combination of multiple kernel functions enables multi-scale analysis and modeling of the data. The feature fusion module integrates the outputs from each kernel function to form a comprehensive feature representation, which serves as the informational basis for sequence modeling.

The Transformer encoder layer addresses both short- and long-term dependencies in time-series data. Positional encoding retains the temporal order of the sequence, and the multi-head self-attention mechanism computes dependencies across different time scales in parallel, capturing both short-term fluctuations and long-term trends. A feed-forward neural network applies nonlinear transformations to the attention outputs, enhancing the model’s capability to capture complex features. Layer normalization and dropout stabilize the training process, prevent overfitting, and improve learning efficiency through residual connections, enabling the model to perform excellently on complex datasets.

The output layer maps the encoded high-dimensional features to the prediction space to generate the final results. To address the complexity of non-stationary Gaussian processes, a Gaussian process parameterization module is introduced, allowing the model to dynamically adapt to data changes. This module provides predictions and describes the evolution and uncertainty of the data, offering a basis for further analysis and decision-making.

Experimental design

Performance analysis

In the diagnosis and treatment of hemorrhagic stroke (such as cerebral hemorrhage), the presence or absence of hematoma expansion is a critical metric. Hematoma expansion refers to a significant increase in hematoma volume shortly after the initial bleeding, which usually indicates a more severe prognosis and a higher mortality rate. Using the presence or absence of hematoma expansion as the evaluation criterion, the performance of the proposed model on the test set is shown in Table 2 and Fig. 2.

Further analysis of the confusion matrix (Table 2) details the model’s classification performance on the test set for predicting hematoma expansion. The proposed MSKT-NSGP model achieved a high Recall (Sensitivity) of 91.3%, demonstrating a strong capacity to identify patients genuinely experiencing expansion, which is crucial for minimizing missed high-risk events. Concurrently, the Specificity was 82.3%, indicating correct identification of most patients without expansion, though with a notable false positive rate. The Precision stood at 71.2%, meaning positive predictions were correct approximately 71% of the time. While the high recall is clinically valuable for screening, the presence of 4 false negatives and the moderate precision (implying nearly 29% of positive alerts may be false alarms) necessitate careful consideration. Consequently, MSKT-NSGP shows promise as a clinical decision support aid but requires further prospective validation before potentially replacing clinical judgment.

Overall, the proposed model demonstrated strong predictive capability, achieving an accuracy of 85.5% and an area under the curve (AUC) of 0.87, indicating high discriminative ability (Fig. 2). This effectiveness, particularly in handling complex dependencies and non-stationary characteristics inherent in the time-series data, can be attributed to the model’s architecture. The integration of multi-scale kernel functions for diverse feature extraction, combined with the Transformer’s self-attention mechanism for modeling temporal dynamics, allows the MSKT-NSGP model to excel in this challenging clinical prediction task.

Comparison of prediction accuracy

In terms of prediction accuracy, we evaluated the performance of each algorithm on the test set. Figure 3 presents the comparison of prediction accuracy across different algorithms on multiple datasets. It can be seen that the proposed model consistently exhibits higher prediction accuracy across all test sets, especially for time-series data with highly non-stationary characteristics, significantly outperforming other benchmark algorithms. In contrast, SVGP and MS-CNN have some capability in handling complex time dependencies but show limitations when dealing with strong non-stationarity and data heterogeneity. This is because the sparsity assumption of SVGP limits its fitting ability for highly dynamic data, while MS-CNN, though capable of capturing multi-scale features, encounters bottlenecks in long-term dependency modeling.

VAE-GP has an advantage in capturing nonlinear relationships and can learn complex nonlinear mappings through its encoder network, but its latent space distribution assumptions may not be flexible enough to handle multimodal features, leading to limited prediction accuracy. MDN, with its multimodal output capability, can handle uncertainty, but its single-layer network structure lacks depth and expressiveness compared to the proposed model in handling long-term dependencies and complex dynamic changes.

From Fig. 4, it can be seen that the proposed model outperforms other benchmark models in terms of both mean squared error (MSE) and mean absolute error (MAE), particularly when dealing with complex non-stationary time-series data, demonstrating its superiority in predictive capability.

Computational efficiency and model complexity

Computational efficiency is another critical evaluation metric, especially in processing large-scale time-series data. We compared the parameter count, computational cost, memory usage, training time, and inference speed of each algorithm under the same hardware environment, as shown in the Table 3.

The results show that MSKT-NSGP has a parameter count of 19.97 million and a computational cost of 5.40 GFLOPs, placing it in the mid-high range. The training time is 158 min, and the inference time is 55 milliseconds. The model achieves a significant improvement in handling complex non-stationary time-series data and prediction accuracy by increasing computational overhead. Although it requires higher computation and memory, the inference speed remains relatively good, indicating that MSKT-NSGP achieves an effective balance between accuracy and computational cost, making it particularly suitable for applications that demand high precision.

Capability of uncertainty handling

This experiment aims to evaluate the capability of different models in handling uncertainty, especially in the context of medical time-series data. A series of experiments were designed to quantify the models’ performance in predicting uncertainty, and a comprehensive evaluation was conducted based on the scoring criteria.

Based on the predicted distribution’s mean and variance obtained earlier, the mean prediction error (MPE) and variance calibration (VC) were calculated to assess the accuracy of the predicted mean and the reasonableness of the predicted variance. The difference between the predicted mean and the actual value represents the accuracy of the central prediction of the model; the predicted variance reflects the model’s estimate of uncertainty—the larger the variance, the lower the model’s confidence in the prediction.

Next, the prediction confidence interval at the 95% confidence level for each model was calculated, and the coverage and width of the confidence interval were evaluated using the following formulas:

(30) $$Coverage={\displaystyle \frac{1}{n}\sum _{i=1}^n\mathrm{I}\left(y_i\in \left[{\hat{y}}_i-1.96{\sigma }_i,{\hat{y}}_i+1.96{\sigma }_i\right]\right)}$$where I is the indicator function, which equals 1 if the actual value is within the confidence interval and 0 otherwise. Ideally, the Coverage should be close to 0.95.

(31) $$Width={\displaystyle \frac{1}{n}\sum _{i=1}^{n}2\times 1.96\times {\sigma }_i.}$$

A narrower confidence interval width with Coverage close to 0.95 indicates higher predictive accuracy.

Data heterogeneity and noise handling were then analyzed by introducing noise and variation into the data to evaluate model stability under noise and heterogeneity. The Rate of Noise-induced Error Change (RNH) was calculated as:

(32) $$RNH={\displaystyle \frac{1}{n}\sum _{i=1}^n{\displaystyle \frac{\left|\left({\hat{y}}_i^{noise}-y_i\right)-\left({\hat{y}}_i-y_i\right)\right|}{\left|{\hat{y}}_i-y_i\right|}}}$$where ${\hat{y}}_i^{noise}$ is the prediction value after noise is added. A smaller RNH value indicates stronger robustness of the model against noise and heterogeneity.

The experimental results (Figs. 5 to 7) show that MSKT-NSGP significantly outperforms other models in predicting uncertainty and handling noise. Figure 5 shows that MSKT-NSGP’s confidence interval coverage is close to 0.95, indicating strong accuracy and reliability of its predictions at a high confidence level. In contrast, SVGP and VAE-GP have lower coverage rates, indicating their limitations in handling high-dimensional complex data uncertainty, particularly in dynamic data environments like hemorrhagic stroke.

Figure 6 presents the distribution of confidence interval widths. MSKT-NSGP has narrower confidence intervals, suggesting higher prediction accuracy at high coverage rates. This is due to its multi-scale feature fusion and self-attention mechanism, which effectively capture the multi-scale dynamic features of complex time-series data. In contrast, SVGP and MDN have wider confidence intervals, showing their lack of precision at high confidence levels, possibly due to an inability to effectively combine multi-scale information and dynamic patterns, leading to over- or underestimation.

Figure 7 shows the rate of noise-induced error change (RNH), where MSKT-NSGP has the lowest error change rate, demonstrating stronger robustness in handling data noise and heterogeneity. This is attributed to its combined modeling capability of different feature scales and temporal dependencies, allowing it to maintain stable performance in noisy environments. In contrast, VAE-GP shows increased errors under noise conditions due to the inflexibility of its latent space distribution assumptions when handling multimodal features; while SVGP, despite improved computational efficiency, shows weaker adaptability to abrupt changes in non-stationary and heterogeneous data environments.