Validation and Topic-driven Ranking for Biomedical Hypothesis Generation Systems

by   Justin Sybrandt, et al.

Literature underpins research, providing the foundation for new ideas. But as the pace of science accelerates, many researchers struggle to stay current. To expedite their searches, some scientists leverage hypothesis generation (HG) systems, which can automatically inspect published papers to uncover novel implicit connections. With no foreseeable end to the driving pace of research, we expect these systems will become crucial for productive scientists, and later form the basis of intelligent automated discovery systems. Yet, many resort to expert analysis to validate such systems. This process is slow, hard to reproduce, and takes time away from other researchers. Therefore, we present a novel method to validate HG systems, which both scales to large validation sets and does not require expert input. We also introduce a number of new metrics to automatically identify plausible generated hypotheses. Through the study of published, highly cited, and noise predicates, we devise a validation challenge, which allows us to evaluate the performance of a HG system. Using an in-progress system, MOLIERE, as a case-study, we show the utility of our validation and ranking methods. So that others may reproduce our results, we provide our code, validation data, and results at


page 1

page 2

page 3

page 4


Accelerating COVID-19 research with graph mining and transformer-based learning

In 2020, the White House released the, "Call to Action to the Tech Commu...

AGATHA: Automatic Graph-mining And Transformer based Hypothesis generation Approach

Medical research is risky and expensive. Drug discovery, as an example, ...

Large Language Models for Automated Open-domain Scientific Hypotheses Discovery

Hypothetical induction is recognized as the main reasoning type when sci...

MOLIERE: Automatic Biomedical Hypothesis Generation System

Hypothesis generation is becoming a crucial time-saving technique which ...

Literature-based Discovery for Landscape Planning

This project demonstrates how medical corpus hypothesis generation, a kn...

The decline of astronomical research in Venezuela

During the last 15 years the number of astronomy-related papers publishe...

Can Persistent Homology provide an efficient alternative for Evaluation of Knowledge Graph Completion Methods?

In this paper we present a novel method, Knowledge Persistence (𝒦𝒫), for...

Please sign up or login with your details

Forgot password? Click here to reset