Mask and Cloze: Automatic Open Cloze Question Generation using a Masked Language Model

by   Shoya Matsumori, et al.

Open cloze questions have been attracting attention for both measuring the ability and facilitating the learning of L2 English learners. In spite of its benefits, the open cloze test has been introduced only sporadically on the educational front, largely because it is burdensome for teachers to manually create the questions. Unlike the more commonly used multiple choice questions (MCQ), open cloze questions are in free form and thus teachers have to ensure that only a ground truth answer and no additional words will be accepted in the blank. To help ease this burden, we developed CLOZER, an automatic open cloze question generator. In this work, we evaluate CLOZER through quantitative experiments on 1,600 answers and show statistically that it can successfully generate open cloze questions that only accept the ground truth answer. A comparative experiment with human-generated questions also reveals that CLOZER can generate OCQs better than the average non-native English teacher. Additionally, we conduct a field study at a local high school to clarify the benefits and hurdles when introducing CLOZER. The results demonstrate that while students found the application useful for their language learning. Finally, on the basis of our findings, we proposed several design improvements.


page 1

page 3

page 8

page 9

page 10

page 11


Generating Answer Candidates for Quizzes and Answer-Aware Question Generators

In education, open-ended quiz questions have become an important tool fo...

Generating Followup Questions for Interpretable Multi-hop Question Answering

We propose a framework for answering open domain multi-hop questions in ...

O.D.E.S. : An Online Dynamic Examination System based on a CMS Wordpress plugin

This paper describes the online dynamic examination application plugin n...

Import test questions into Moodle LMS

The purpose of the study is to highlight the theoretical and methodologi...

Better Distractions: Transformer-based Distractor Generation and Multiple Choice Question Filtering

For the field of education, being able to generate semantically correct ...

QAScore – An Unsupervised Unreferenced Metric for the Question Generation Evaluation

Question Generation (QG) aims to automate the task of composing question...

Leveraging Human Feedback to Scale Educational Datasets: Combining Crowdworkers and Comparative Judgement

Machine Learning models have many potentially beneficial applications in...

Please sign up or login with your details

Forgot password? Click here to reset