UniBrain: Universal Brain MRI Diagnosis with Hierarchical Knowledge-enhanced Pre-training

by   Jiayu Lei, et al.

Magnetic resonance imaging (MRI) have played a crucial role in brain disease diagnosis, with which a range of computer-aided artificial intelligence methods have been proposed. However, the early explorations usually focus on the limited types of brain diseases in one study and train the model on the data in a small scale, yielding the bottleneck of generalization. Towards a more effective and scalable paradigm, we propose a hierarchical knowledge-enhanced pre-training framework for the universal brain MRI diagnosis, termed as UniBrain. Specifically, UniBrain leverages a large-scale dataset of 24,770 imaging-report pairs from routine diagnostics. Different from previous pre-training techniques for the unitary vision or textual feature, or with the brute-force alignment between vision and language information, we leverage the unique characteristic of report information in different granularity to build a hierarchical alignment mechanism, which strengthens the efficiency in feature learning. Our UniBrain is validated on three real world datasets with severe class imbalance and the public BraTS2019 dataset. It not only consistently outperforms all state-of-the-art diagnostic methods by a large margin and provides a superior grounding performance but also shows comparable performance compared to expert radiologists on certain disease types.


page 3

page 10


Medical Transformer: Universal Brain Encoder for 3D MRI Analysis

Transfer learning has gained attention in medical image analysis due to ...

BrainNPT: Pre-training of Transformer networks for brain network classification

Deep learning methods have advanced quickly in brain imaging analysis ov...

Cross-Domain Self-Supervised Deep Learning for Robust Alzheimer's Disease Progression Modeling

Developing successful artificial intelligence systems in practice depend...

Connectional-Style-Guided Contextual Representation Learning for Brain Disease Diagnosis

Structural magnetic resonance imaging (sMRI) has shown great clinical va...

Classification of Brain Tumours in MR Images using Deep Spatiospatial Models

A brain tumour is a mass or cluster of abnormal cells in the brain, whic...

SpineOne: A One-Stage Detection Framework for Degenerative Discs and Vertebrae

Spinal degeneration plagues many elders, office workers, and even the yo...

Please sign up or login with your details

Forgot password? Click here to reset