Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering

by   Jeroen Offerijns, et al.

For the field of education, being able to generate semantically correct and educationally relevant multiple choice questions (MCQs) could have a large impact. While question generation itself is an active research topic, generating distractors (the incorrect multiple choice options) receives much less attention. A missed opportunity, since there is still a lot of room for improvement in this area. In this work, we train a GPT-2 language model to generate three distractors for a given question and text context, using the RACE dataset. Next, we train a BERT language model to answer MCQs, and use this model as a filter, to select only questions that can be answered and therefore presumably make sense. To evaluate our work, we start by using text generation metrics, which show that our model outperforms earlier work on distractor generation (DG) and achieves state-of-the-art performance. Also, by calculating the question answering ability, we show that larger base models lead to better performance. Moreover, we conducted a human evaluation study, which confirmed the quality of the generated questions, but showed no statistically significant effect of the QA filter.


page 1

page 2

page 3

page 4


Addressing Semantic Drift in Question Generation for Semi-Supervised Question Answering

Text-based Question Generation (QG) aims at generating natural and relev...

Exploring Automated Distractor and Feedback Generation for Math Multiple-choice Questions via In-context Learning

Multiple-choice questions (MCQs) are ubiquitous in almost all levels of ...

Learning to Answer by Learning to Ask: Getting the Best of GPT-2 and BERT Worlds

Automatic question generation aims at the generation of questions from a...

A BERT-based Distractor Generation Scheme with Multi-tasking and Negative Answer Training Strategies

In this paper, we investigate the following two limitations for the exis...

Transformer-based End-to-End Question Generation

Question Generation (QG) is an important task in Natural Language Proces...

Adversarial and Safely Scaled Question Generation

Question generation has recently gained a lot of research interest, espe...

Mask and Cloze: Automatic Open Cloze Question Generation using a Masked Language Model

Open cloze questions have been attracting attention for both measuring t...

Please sign up or login with your details

Forgot password? Click here to reset