Multi-modal Multi-Level Neuroimaging Fusion with Modality-Aware Mask-Guided Attention and Deep Canonical Correlation Analysis to Improve Dementia Risk Prediction

Abstract

Alzheimer's Disease (AD) is a progressive neurodegenerative disorder characterized by structural and molecular changes in the brain. Early diagnosis and accurate subtyping are essential for timely intervention and therapeutic planning. This thesis presents a novel multimodal deep learning framework that integrates T1-weighted MRI and Amyloid PET imaging to improve the diagnosis and stratification of AD. The proposed architecture leverages a two-stage pipeline involving modality-specific feature extraction using ResNet50 backbones, followed by middle fusion enhanced with a Modality-Aware Mask-Guided Attention (MAMGA) mechanism. To address missing modalities and inter-modal misalignment, the model incorporates Random Modality Masking and Deep Canonical Correlation Analysis (DCCA) for cross-modal feature alignment. Experiments on the ADNI dataset demonstrate that the proposed MRI+PET (MAMGA+DCCA) model achieves a balanced accuracy of 0.998 and an AUC-ROC of 0.999 in distinguishing stable normal cognition (sNC) from stable Alzheimer's Disease (sDAT). For the more challenging task of separating stable and progressive MCI (sMCI vs. pMCI), the best-performing fusion model achieved a balanced accuracy of 0.732 and an AUC of 0.789. Extensive ablation studies confirm the contributions of MAMGA, DCCA, and dual-optimizer strategies in enhancing diagnostic robustness. This work highlights the clinical potential of multimodal deep learning frameworks in improving early Alzheimer's detection and stratification.