Get It Scored Using AutoSAS – An Automated System for Scoring Short Answers

by   Yaman Kumar, et al.

In the era of MOOCs, online exams are taken by millions of candidates, where scoring short answers is an integral part. It becomes intractable to evaluate them by human graders. Thus, a generic automated system capable of grading these responses should be designed and deployed. In this paper, we present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS). We propose and explain the design and development of a system for SAS, namely AutoSAS. Given a question along with its graded samples, AutoSAS can learn to grade that prompt successfully. This paper further lays down the features such as lexical diversity, Word2Vec, prompt, and content overlap that plays a pivotal role in building our proposed model. We also present a methodology for indicating the factors responsible for scoring an answer. The trained model is evaluated on an extensively used public dataset, namely Automated Student Assessment Prize Short Answer Scoring (ASAP-SAS). AutoSAS shows state-of-the-art performance and achieves better results by over 8 some of the question prompts as measured by Quadratic Weighted Kappa (QWK), showing performance comparable to humans.


page 1

page 2

page 3

page 4


Modeling and Analyzing Scorer Preferences in Short-Answer Math Questions

Automated scoring of student responses to open-ended questions, includin...

Automatic Short Answer Grading and Feedback Using Text Mining Methods

Automatic grading is not a new approach but the need to adapt the latest...

Rubric-Specific Approach to Automated Essay Scoring with Augmentation Training

Neural based approaches to automatic evaluation of subjective responses ...

Balancing Cost and Quality: An Exploration of Human-in-the-loop Frameworks for Automated Short Answer Scoring

Short answer scoring (SAS) is the task of grading short text written by ...

"What is on your mind?" Automated Scoring of Mindreading in Childhood and Early Adolescence

In this paper we present the first work on the automated scoring of mind...

Joint Multi-Domain Learning for Automatic Short Answer Grading

One of the fundamental challenges towards building any intelligent tutor...

Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees

Automated Scoring (AS), the natural language processing task of scoring ...

Please sign up or login with your details

Forgot password? Click here to reset