Two is Better than Many? Binary Classification as an Effective Approach to Multi-Choice Question Answering

by   Deepanway Ghosal, et al.
University of Michigan

We propose a simple refactoring of multi-choice question answering (MCQA) tasks as a series of binary classifications. The MCQA task is generally performed by scoring each (question, answer) pair normalized over all the pairs, and then selecting the answer from the pair that yield the highest score. For n answer choices, this is equivalent to an n-class classification setup where only one class (true answer) is correct. We instead show that classifying (question, true answer) as positive instances and (question, false answer) as negative instances is significantly more effective across various models and datasets. We show the efficacy of our proposed approach in different tasks – abductive reasoning, commonsense question answering, science question answering, and sentence completion. Our DeBERTa binary classification model reaches the top or close to the top performance on public leaderboards for these tasks. The source code of the proposed approach is available at


page 1

page 2

page 3

page 4


BUCA: A Binary Classification Approach to Unsupervised Commonsense Question Answering

Unsupervised commonsense reasoning (UCR) is becoming increasingly popula...

CS1QA: A Dataset for Assisting Code-based Question Answering in an Introductory Programming Course

We introduce CS1QA, a dataset for code-based question answering in the p...

SemEval-2019 Task 8: Fact Checking in Community Question Answering Forums

We present SemEval-2019 Task 8 on Fact Checking in Community Question An...

A Semantic-based Method for Unsupervised Commonsense Question Answering

Unsupervised commonsense question answering is appealing since it does n...

Question answering: from partitions to Prolog

We implement Groenendijk and Stokhof's partition semantics of questions ...

Context-guided Triple Matching for Multiple Choice Question Answering

The task of multiple choice question answering (MCQA) refers to identify...

Prompting as Probing: Using Language Models for Knowledge Base Construction

Language Models (LMs) have proven to be useful in various downstream app...

Code Repositories


Our EMNLP 2022 paper on MCQA

view repo

Please sign up or login with your details

Forgot password? Click here to reset