Tail-to-Tail Non-Autoregressive Sequence Prediction for Chinese Grammatical Error Correction

by   Piji Li, et al.

We investigate the problem of Chinese Grammatical Error Correction (CGEC) and present a new framework named Tail-to-Tail (TtT) non-autoregressive sequence prediction to address the deep issues hidden in CGEC. Considering that most tokens are correct and can be conveyed directly from source to target, and the error positions can be estimated and corrected based on the bidirectional context information, thus we employ a BERT-initialized Transformer Encoder as the backbone model to conduct information modeling and conveying. Considering that only relying on the same position substitution cannot handle the variable-length correction cases, various operations such substitution, deletion, insertion, and local paraphrasing are required jointly. Therefore, a Conditional Random Fields (CRF) layer is stacked on the up tail to conduct non-autoregressive sequence prediction by modeling the token dependencies. Since most tokens are correct and easily to be predicted/conveyed to the target, then the models may suffer from a severe class imbalance issue. To alleviate this problem, focal loss penalty strategies are integrated into the loss functions. Moreover, besides the typical fix-length error correction datasets, we also construct a variable-length corpus to conduct experiments. Experimental results on standard datasets, especially on the variable-length datasets, demonstrate the effectiveness of TtT in terms of sentence-level Accuracy, Precision, Recall, and F1-Measure on tasks of error Detection and Correction.


page 1

page 2

page 3

page 4


FastCorrect: Fast Error Correction with Edit Alignment for Automatic Speech Recognition

Error correction techniques have been used to refine the output sentence...

Sequence-to-Action: Grammatical Error Correction with Action Guided Sequence Generation

The task of Grammatical Error Correction (GEC) has received remarkable a...

uChecker: Masked Pretrained Language Models as Unsupervised Chinese Spelling Checkers

The task of Chinese Spelling Check (CSC) is aiming to detect and correct...

Spelling Error Correction with Soft-Masked BERT

Spelling error correction is an important yet challenging task because a...

An Error-Guided Correction Model for Chinese Spelling Error Correction

Although existing neural network approaches have achieved great success ...

Progressive Multi-task Learning Framework for Chinese Text Error Correction

Chinese Text Error Correction (CTEC) aims to detect and correct errors i...

Pseudo-Bidirectional Decoding for Local Sequence Transduction

Local sequence transduction (LST) tasks are sequence transduction tasks ...

Please sign up or login with your details

Forgot password? Click here to reset