Automatic Short Math Answer Grading via In-context Meta-learning

by   Mengxue Zhang, et al.

Automatic short answer grading is an important research direction in the exploration of how to use artificial intelligence (AI)-based tools to improve education. Current state-of-the-art approaches use neural language models to create vectorized representations of students responses, followed by classifiers to predict the score. However, these approaches have several key limitations, including i) they use pre-trained language models that are not well-adapted to educational subject domains and/or student-generated text and ii) they almost always train one model per question, ignoring the linkage across a question and result in a significant model storage problem due to the size of advanced language models. In this paper, we study the problem of automatic short answer grading for students' responses to math questions and propose a novel framework for this task. First, we use MathBERT, a variant of the popular language model BERT adapted to mathematical content, as our base model and fine-tune it for the downstream task of student response grading. Second, we use an in-context learning approach that provides scoring examples as input to the language model to provide additional context information and promote generalization to previously unseen questions. We evaluate our framework on a real-world dataset of student responses to open-ended math questions and show that our framework (often significantly) outperforms existing approaches, especially for new questions that are not seen during training.


page 1

page 2

page 3

page 4


Context Matters: A Strategy to Pre-train Language Model for Science Education

This study aims at improving the performance of scoring student response...

Large Language Model-based System to Provide Immediate Feedback to Students in Flipped Classroom Preparation Learning

This paper proposes a system that uses large language models to provide ...

Code Soliloquies for Accurate Calculations in Large Language Models

High-quality conversational datasets are integral to the successful deve...

Algebra Error Classification with Large Language Models

Automated feedback as students answer open-ended math questions has sign...

Using Language Models to Detect Alarming Student Responses

This article details the advances made to a system that uses artificial ...

Using language models in the implicit automated assessment of mathematical short answer items

We propose a new way to assess certain short constructed responses to ma...

ChatGPT and Software Testing Education: Promises Perils

Over the past decade, predictive language modeling for code has proven t...

Please sign up or login with your details

Forgot password? Click here to reset