Multi-modal Learning based Prediction for Disease

by   Yaran Chen, et al.

Non alcoholic fatty liver disease (NAFLD) is the most common cause of chronic liver disease, which can be predicted accurately to prevent advanced fibrosis and cirrhosis. While, a liver biopsy, the gold standard for NAFLD diagnosis, is invasive, expensive, and prone to sampling errors. Therefore, non-invasive studies are extremely promising, yet they are still in their infancy due to the lack of comprehensive research data and intelligent methods for multi-modal data. This paper proposes a NAFLD diagnosis system (DeepFLDDiag) combining a comprehensive clinical dataset (FLDData) and a multi-modal learning based NAFLD prediction method (DeepFLD). The dataset includes over 6000 participants physical examinations, laboratory and imaging studies, extensive questionnaires, and facial images of partial participants, which is comprehensive and valuable for clinical studies. From the dataset, we quantitatively analyze and select clinical metadata that most contribute to NAFLD prediction. Furthermore, the proposed DeepFLD, a deep neural network model designed to predict NAFLD using multi-modal input, including metadata and facial images, outperforms the approach that only uses metadata. Satisfactory performance is also verified on other unseen datasets. Inspiringly, DeepFLD can achieve competitive results using only facial images as input rather than metadata, paving the way for a more robust and simpler non-invasive NAFLD diagnosis.


page 1

page 3

page 6

page 8

page 9

page 10


PheME: A deep ensemble framework for improving phenotype prediction from multi-modal data

Detailed phenotype information is fundamental to accurate diagnosis and ...

Multi-modal Graph Neural Network for Early Diagnosis of Alzheimer's Disease from sMRI and PET Scans

In recent years, deep learning models have been applied to neuroimaging ...

Self-Attention Equipped Graph Convolutions for Disease Prediction

Multi-modal data comprising imaging (MRI, fMRI, PET, etc.) and non-imagi...

Multi-modal Disease Classification in Incomplete Datasets Using Geometric Matrix Completion

In large population-based studies and in clinical routine, tasks like di...

Intelligent Sight and Sound: A Chronic Cancer Pain Dataset

Cancer patients experience high rates of chronic pain throughout the tre...

Multi-Modal Hypergraph Diffusion Network with Dual Prior for Alzheimer Classification

The automatic early diagnosis of prodromal stages of Alzheimer's disease...

Please sign up or login with your details

Forgot password? Click here to reset