Measuring the State of the Art of Automated Pathway Curation Using Graph Algorithms - A Case Study of the mTOR Pathway

08/12/2016
by   Michael Spranger, et al.
0

This paper evaluates the difference between human pathway curation and current NLP systems. We propose graph analysis methods for quantifying the gap between human curated pathway maps and the output of state-of-the-art automatic NLP systems. Evaluation is performed on the popular mTOR pathway. Based on analyzing where current systems perform well and where they fail, we identify possible avenues for progress.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/18/2020

Inspecting state of the art performance and NLP metrics in image-based medical report generation

Several deep learning architectures have been proposed over the last yea...
research
02/26/2021

Methods for the Design and Evaluation of HCI+NLP Systems

HCI and NLP traditionally focus on different evaluation methods. While H...
research
04/21/2023

A Group-Specific Approach to NLP for Hate Speech Detection

Automatic hate speech detection is an important yet complex task, requir...
research
06/21/2018

Experimental Analysis of Distributed Graph Systems

This paper evaluates eight parallel graph processing systems: Hadoop, Ha...
research
01/22/2021

Evaluation Discrepancy Discovery: A Sentence Compression Case-study

Reliable evaluation protocols are of utmost importance for reproducible ...
research
01/13/2021

Robustness Gym: Unifying the NLP Evaluation Landscape

Despite impressive performance on standard benchmarks, deep neural netwo...
research
10/13/2020

With Little Power Comes Great Responsibility

Despite its importance to experimental design, statistical power (the pr...

Please sign up or login with your details

Forgot password? Click here to reset