ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning

by   Ahmed Masry, et al.
York University

Charts are very popular for analyzing data. When exploring charts, people often ask a variety of complex reasoning questions that involve several logical and arithmetic operations. They also commonly refer to visual features of a chart in their questions. However, most existing datasets do not focus on such complex reasoning questions as their questions are template-based and answers come from a fixed-vocabulary. In this work, we present a large-scale benchmark covering 9.6K human-written questions as well as 23.1K questions generated from human-written chart summaries. To address the unique challenges in our benchmark involving visual and logical reasoning over charts, we present two transformer-based models that combine visual features and the data table of the chart in a unified way to answer questions. While our models achieve the state-of-the-art results on the previous datasets as well as on our benchmark, the evaluation also reveals several challenges in answering complex reasoning questions.


page 13

page 14

page 17


Data Interpretation over Plots

Reasoning over plots by question answering (QA) is a challenging machine...

Explicit Knowledge-based Reasoning for Visual Question Answering

We describe a method for visual question answering which is capable of r...

Declarative Question Answering over Knowledge Bases containing Natural Language Text with Answer Set Programming

While in recent years machine learning (ML) based approaches have been t...

OpenCQA: Open-ended Question Answering with Charts

Charts are very popular to analyze data and convey important insights. P...

V-PROM: A Benchmark for Visual Reasoning Using Visual Progressive Matrices

One of the primary challenges faced by deep learning is the degree to wh...

Non-monotonic Logical Reasoning Guiding Deep Learning for Explainable Visual Question Answering

State of the art algorithms for many pattern recognition problems rely o...

Benchmark Visual Question Answer Models by using Focus Map

Inferring and Executing Programs for Visual Reasoning proposes a model f...

Please sign up or login with your details

Forgot password? Click here to reset