Diagnose Like a Radiologist: Hybrid Neuro-Probabilistic Reasoning for Attribute-Based Medical Image Diagnosis

by   Gangming Zhao, et al.

During clinical practice, radiologists often use attributes, e.g. morphological and appearance characteristics of a lesion, to aid disease diagnosis. Effectively modeling attributes as well as all relationships involving attributes could boost the generalization ability and verifiability of medical image diagnosis algorithms. In this paper, we introduce a hybrid neuro-probabilistic reasoning algorithm for verifiable attribute-based medical image diagnosis. There are two parallel branches in our hybrid algorithm, a Bayesian network branch performing probabilistic causal relationship reasoning and a graph convolutional network branch performing more generic relational modeling and reasoning using a feature representation. Tight coupling between these two branches is achieved via a cross-network attention mechanism and the fusion of their classification results. We have successfully applied our hybrid reasoning algorithm to two challenging medical image diagnosis tasks. On the LIDC-IDRI benchmark dataset for benign-malignant classification of pulmonary nodules in CT images, our method achieves a new state-of-the-art accuracy of 95.36% and an AUC of 96.54%. Our method also achieves a 3.24% accuracy improvement on an in-house chest X-ray image dataset for tuberculosis diagnosis. Our ablation study indicates that our hybrid algorithm achieves a much better generalization performance than a pure neural network architecture under very limited training data.


page 2

page 10

page 11

page 12

page 13


Learning Invariant Feature Representation to Improve Generalization across Chest X-ray Datasets

Chest radiography is the most common medical image examination for scree...

Cross Chest Graph for Disease Diagnosis with Structural Relational Reasoning

Locating lesions is important in the computer-aided diagnosis of X-ray i...

Multi-Instance Learning by Utilizing Structural Relationship among Instances

Multi-Instance Learning(MIL) aims to learn the mapping between a bag of ...

Multi-Scale Feature Fusion using Parallel-Attention Block for COVID-19 Chest X-ray Diagnosis

Under the global COVID-19 crisis, accurate diagnosis of COVID-19 from Ch...

Medical idioms for clinical Bayesian network development

Bayesian Networks (BNs) are graphical probabilistic models that have pro...

OXnet: Omni-supervised Thoracic Disease Detection from Chest X-rays

Chest X-ray (CXR) is the most typical medical image worldwide to examine...

Justifying Diagnosis Decisions by Deep Neural Networks

An integrated approach is proposed across visual and textual data to bot...

Please sign up or login with your details

Forgot password? Click here to reset