Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs

by   Somin Wadhwa, et al.

Results from Randomized Controlled Trials (RCTs) establish the comparative effectiveness of interventions, and are in turn critical inputs for evidence-based care. However, results from RCTs are presented in (often unstructured) natural language articles describing the design, execution, and outcomes of trials; clinicians must manually extract findings pertaining to interventions and outcomes of interest from such articles. This onerous manual process has motivated work on (semi-)automating extraction of structured evidence from trial reports. In this work we propose and evaluate a text-to-text model built on instruction-tuned Large Language Models (LLMs) to jointly extract Interventions, Outcomes, and Comparators (ICO elements) from clinical abstracts, and infer the associated results reported. Manual (expert) and automated evaluations indicate that framing evidence extraction as a conditional generation task and fine-tuning LLMs for this purpose realizes considerable (∼20 point absolute F1 score) gains over the previous SOTA. We perform ablations and error analyses to assess aspects that contribute to model performance, and to highlight potential directions for further improvements. We apply our model to a collection of published RCTs through mid-2022, and release a searchable database of structured findings (anonymously for now):


Understanding Clinical Trial Reports: Extracting Medical Entities and Their Relations

The best evidence concerning comparative treatment effectiveness comes f...

Trialstreamer: Mapping and Browsing Medical Evidence in Real-Time

We introduce Trialstreamer, a living database of clinical trial reports....

A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature

We present a corpus of 5,000 richly annotated abstracts of medical artic...

An automated approach to extracting positive and negative clinical research results

Failure is common in clinical trials since the successful failures prese...

Extraction of evidence tables from abstracts of randomized clinical trials using a maximum entropy classifier and global constraints

Systematic use of the published results of randomized clinical trials is...

Semi-Automating Knowledge Base Construction for Cancer Genetics

In this work, we consider the exponentially growing subarea of genetics ...

Inferring Which Medical Treatments Work from Reports of Clinical Trials

How do we know if a particular medical treatment actually works? Ideally...

Please sign up or login with your details

Forgot password? Click here to reset