Incomplete Multimodal Learning for Complex Brain Disorders Prediction

by   Reza Shirkavand, et al.

Recent advancements in the acquisition of various brain data sources have created new opportunities for integrating multimodal brain data to assist in early detection of complex brain disorders. However, current data integration approaches typically need a complete set of biomedical data modalities, which may not always be feasible, as some modalities are only available in large-scale research cohorts and are prohibitive to collect in routine clinical practice. Especially in studies of brain diseases, research cohorts may include both neuroimaging data and genetic data, but for practical clinical diagnosis, we often need to make disease predictions only based on neuroimages. As a result, it is desired to design machine learning models which can use all available data (different data could provide complementary information) during training but conduct inference using only the most common data modality. We propose a new incomplete multimodal data integration approach that employs transformers and generative adversarial networks to effectively exploit auxiliary modalities available during training in order to improve the performance of a unimodal model at inference. We apply our new method to predict cognitive degeneration and disease outcomes using the multimodal imaging genetic data from Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort. Experimental results demonstrate that our approach outperforms the related machine learning and deep learning methods by a significant margin.


Predicting Brain Degeneration with a Multimodal Siamese Neural Network

To study neurodegenerative diseases, longitudinal studies are carried on...

Medical Diagnosis with Large Scale Multimodal Transformers: Leveraging Diverse Data for More Accurate Diagnosis

Multimodal deep learning has been used to predict clinical endpoints and...

Multimodal Data Integration for Oncology in the Era of Deep Neural Networks: A Review

Cancer has relational information residing at varying scales, modalities...

A Sequence Agnostic Multimodal Preprocessing for Clogged Blood Vessel Detection in Alzheimer's Diagnosis

Successful identification of blood vessel blockage is a crucial step for...

Multilevel Modeling with Structured Penalties for Classification from Imaging Genetics data

In this paper, we propose a framework for automatic classification of pa...

Interpretable multimodal fusion networks reveal mechanisms of brain cognition

Multimodal fusion benefits disease diagnosis by providing a more compreh...

Multimodal Neuroimaging Data Integration and Pathway Analysis

With fast advancements in technologies, the collection of multiple types...

Please sign up or login with your details

Forgot password? Click here to reset