Adapting BigScience Multilingual Model to Unseen Languages

04/11/2022

∙

We benchmark different strategies of adding new languages (German and Korean) into the BigScience's pretrained multilingual language model with 1.3 billion parameters that currently supports 13 languages. We investigate the factors that affect the language adaptability of the model and the trade-offs between computational costs and expected performance.

READ FULL TEXT

Adapting BigScience Multilingual Model to Unseen Languages

HiJoNLP at SemEval-2022 Task 2: Detecting Idiomaticity of Multiword Expressions using Multilingual Pretrained Language Models

Multichannel Generative Language Model: Learning All Possible Factorizations Within and Across Channels

Is Multilingual BERT Fluent in Language Generation?

ALBERTI, a Multilingual Domain Specific Language Model for Poetry Analysis

Multilingual Detection of Check-Worthy Claims using World Languages and Adapter Fusion

GPT-SW3: An Autoregressive Language Model for the Nordic Languages

Methods Included: Standardizing Computational Reuse and Portability with the Common Workflow Language

Adapting BigScience Multilingual Model to Unseen Languages

Related Research

HiJoNLP at SemEval-2022 Task 2: Detecting Idiomaticity of Multiword Expressions using Multilingual Pretrained Language Models

Multichannel Generative Language Model: Learning All Possible Factorizations Within and Across Channels

Is Multilingual BERT Fluent in Language Generation?

ALBERTI, a Multilingual Domain Specific Language Model for Poetry Analysis

Multilingual Detection of Check-Worthy Claims using World Languages and Adapter Fusion

GPT-SW3: An Autoregressive Language Model for the Nordic Languages

Methods Included: Standardizing Computational Reuse and Portability with the Common Workflow Language