Enhancing Cross-lingual Prompting with Mask Token Augmentation

by   Meng Zhou, et al.

Prompting shows promising results in few-shot scenarios. However, its strength for multilingual/cross-lingual problems has not been fully exploited. Zhao and Schütze (2021) made initial explorations in this direction by presenting that cross-lingual prompting outperforms cross-lingual finetuning. In this paper, we conduct empirical analysis on the effect of each component in cross-lingual prompting and derive Universal Prompting across languages, which helps alleviate the discrepancies between source-language training and target-language inference. Based on this, we propose a mask token augmentation framework to further improve the performance of prompt-based cross-lingual transfer. Notably, for XNLI, our method achieves 46.54 training examples per class, significantly better than 34.99


page 1

page 2

page 3

page 4


Enhancing Cross-lingual Transfer by Manifold Mixup

Based on large-scale pre-trained multilingual representations, recent cr...

Is Prompt-Based Finetuning Always Better than Vanilla Finetuning? Insights from Cross-Lingual Language Understanding

Multilingual pretrained language models (MPLMs) have demonstrated substa...

Do Multi-Lingual Pre-trained Language Models Reveal Consistent Token Attributions in Different Languages?

During the past several years, a surge of multi-lingual Pre-trained Lang...

Colexifications for Bootstrapping Cross-lingual Datasets: The Case of Phonology, Concreteness, and Affectiveness

Colexification refers to the linguistic phenomenon where a single lexica...

Cross-lingual Dysarthria Severity Classification for English, Korean, and Tamil

This paper proposes a cross-lingual classification method for English, K...

A Neighbourhood Framework for Resource-Lean Content Flagging

We propose a novel interpretable framework for cross-lingual content fla...

Adversarial Neural Networks for Cross-lingual Sequence Tagging

We study cross-lingual sequence tagging with little or no labeled data i...

Please sign up or login with your details

Forgot password? Click here to reset