Efficiently Aligned Cross-Lingual Transfer Learning for Conversational Tasks using Prompt-Tuning

by   Lifu Tu, et al.

Cross-lingual transfer of language models trained on high-resource languages like English has been widely studied for many NLP tasks, but focus on conversational tasks has been rather limited. This is partly due to the high cost of obtaining non-English conversational data, which results in limited coverage. In this work, we introduce XSGD, a parallel and large-scale multilingual conversation dataset that we created by translating the English-only Schema-Guided Dialogue (SGD) dataset (Rastogi et al., 2020) into 105 other languages. XSGD contains approximately 330k utterances per language. To facilitate aligned cross-lingual representations, we develop an efficient prompt-tuning-based method for learning alignment prompts. We also investigate two different classifiers: NLI-based and vanilla classifiers, and test cross-lingual capability enabled by the aligned prompts. We evaluate our model's cross-lingual generalization capabilities on two conversation tasks: slot-filling and intent classification. Our results demonstrate the strong and efficient modeling ability of NLI-based classifiers and the large cross-lingual transfer improvements achieved by our aligned prompts, particularly in few-shot settings.


Marvelous Agglutinative Language Effect on Cross Lingual Transfer Learning

As for multilingual language models, it is important to select languages...

Multi2WOZ: A Robust Multilingual Dataset and Conversational Pretraining for Task-Oriented Dialog

Research on (multi-domain) task-oriented dialog (TOD) has predominantly ...

Cross-lingual Intermediate Fine-tuning improves Dialogue State Tracking

Recent progress in task-oriented neural dialogue systems is largely focu...

BUFFET: Benchmarking Large Language Models for Few-shot Cross-lingual Transfer

Despite remarkable advancements in few-shot generalization in natural la...

DuRecDial 2.0: A Bilingual Parallel Corpus for Conversational Recommendation

In this paper, we provide a bilingual parallel human-to-human recommenda...

Zero-shot Cross-lingual Conversational Semantic Role Labeling

While conversational semantic role labeling (CSRL) has shown its usefuln...

InfoCTM: A Mutual Information Maximization Perspective of Cross-Lingual Topic Modeling

Cross-lingual topic models have been prevalent for cross-lingual text an...

Please sign up or login with your details

Forgot password? Click here to reset