Integrating textual clinical data and neuroimaging for enhanced diagnosis of neurological disorders
Abstract
Accurate diagnosis of neurological disorders is challenging due to the heterogeneous nature of patient data, which includes neuroimaging and unstructured clinical narratives. The integration of multimodal information, particularly high-dimensional neuroimaging and textual clinical records, has emerged as a pivotal area of research. Conventional methods often process these modalities independently or utilize basic fusion techniques, resulting in semantic misalignment, loss of modality-specific information, and reduced robustness when data is incomplete. To overcome these challenges, this study introduces a multimodal fusion framework centered on the Modal-Interlaced Fusion Network (MIF-Net) and supported by Semantic Cross-Modality Regularization (SCMR). MIF-Net leverages graph-based message passing, cross-attentional encoding, and gated hierarchical fusion to model inter-modal dependencies, ensuring deep semantic alignment while retaining individual modality attributes. SCMR enhances semantic consistency and robustness through adaptive confidence weighting, projection invariance, entropy-based uncertainty regulation, and semi-supervised consistency learning. Experimental results on benchmark datasets for neurological disorder diagnosis demonstrate marked improvements in diagnostic accuracy, modality robustness, and interpretability. This framework offers a scalable solution for integrating structured and unstructured medical data, contributing to advancements in medical image analysis and neuroinformatics.