Improving Chest X-Ray Report Generation by Leveraging Warm-Starting

by   Aaron Nicolson, et al.

Automatically generating a report from a patient's Chest X-Rays (CXRs) is a promising solution to reducing clinical workload and improving patient care. However, current CXR report generators, which are predominantly encoder-to-decoder models, lack the diagnostic accuracy to be deployed in a clinical setting. To improve CXR report generation, we investigate warm-starting the encoder and decoder with recent open-source computer vision and natural language processing checkpoints, such as the Vision Transformer (ViT) and PubMedBERT. To this end, each checkpoint is evaluated on the MIMIC-CXR and IU X-Ray datasets using natural language generation and Clinical Efficacy (CE) metrics. Our experimental investigation demonstrates that the Convolutional vision Transformer (CvT) ImageNet-21K and the Distilled Generative Pre-trained Transformer 2 (DistilGPT2) checkpoints are best for warm-starting the encoder and decoder, respectively. Compared to the state-of-the-art (M2 Transformer Progressive), CvT2DistilGPT2 attained an improvement of 8.3 METEOR. The reports generated by CvT2DistilGPT2 are more diagnostically accurate and have a higher similarity to radiologist reports than previous approaches. By leveraging warm-starting, CvT2DistilGPT2 brings automatic CXR report generation one step closer to the clinical setting. CvT2DistilGPT2 and its MIMIC-CXR checkpoint are available at


page 5

page 8

page 10

page 11

page 12

page 13

page 14

page 15


Clinically Accurate Chest X-Ray Report Generation

The automatic generation of radiology reports given medical radiographs ...

Longitudinal Data and a Semantic Similarity Reward for Chest X-Ray Report Generation

Chest X-Ray (CXR) report generation is a promising approach to improving...

On the Importance of Image Encoding in Automated Chest X-Ray Report Generation

Chest X-ray is one of the most popular medical imaging modalities due to...

Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT models

We propose Retrieval Augmented Generation (RAG) as an approach for autom...

Generating Radiology Reports via Memory-driven Transformer

Medical imaging is frequently used in clinical practice and trials for d...

Utilizing Longitudinal Chest X-Rays and Reports to Pre-Fill Radiology Reports

Despite the reduction in turn-around times in radiology reports with the...

MIMIC-CXR: A large publicly available database of labeled chest radiographs

Chest radiography is an extremely powerful imaging modality, allowing fo...

Please sign up or login with your details

Forgot password? Click here to reset