MELM: Data Augmentation with Masked Entity Language Modeling for Cross-lingual NER

by   Ran Zhou, et al.
Alibaba Group
Nanyang Technological University

Data augmentation for cross-lingual NER requires fine-grained control over token labels of the augmented text. Existing augmentation approach based on masked language modeling may replace a labeled entity with words of a different class, which makes the augmented sentence incompatible with the original label sequence, and thus hurts the performance.We propose a data augmentation framework with Masked-Entity Language Modeling (MELM) which effectively ensures the replacing entities fit the original labels. Specifically, MELM linearizes NER labels into sentence context, and thus the fine-tuned MELM is able to predict masked tokens by explicitly conditioning on their labels. Our MELM is agnostic to the source of data to be augmented. Specifically, when MELM is applied to augment training data of the source language, it achieves up to 3.5 F1 score improvement for cross-lingual NER. When unlabeled target data is available and MELM can be further applied to augment pseudo-labeled target data, the performance gain reaches 5.7 outperforms multiple baseline methods for data augmentation.


page 1

page 2

page 3

page 4


A Dual-Contrastive Framework for Low-Resource Cross-Lingual Named Entity Recognition

Cross-lingual Named Entity Recognition (NER) has recently become a resea...

ConNER: Consistency Training for Cross-lingual Named Entity Recognition

Cross-lingual named entity recognition (NER) suffers from data scarcity ...

Zero-Resource Cross-Lingual Named Entity Recognition

Recently, neural methods have achieved state-of-the-art (SOTA) results i...

CoLaDa: A Collaborative Label Denoising Framework for Cross-lingual Named Entity Recognition

Cross-lingual named entity recognition (NER) aims to train an NER system...

Remedying BiLSTM-CNN Deficiency in Modeling Cross-Context for NER

Recent researches prevalently used BiLSTM-CNN as a core module for NER i...

ACLM: A Selective-Denoising based Generative Data Augmentation Approach for Low-Resource Complex NER

Complex Named Entity Recognition (NER) is the task of detecting linguist...

PALI at SemEval-2021 Task 2: Fine-Tune XLM-RoBERTa for Word in Context Disambiguation

This paper presents the PALI team's winning system for SemEval-2021 Task...

Please sign up or login with your details

Forgot password? Click here to reset