FarsTail: A Persian Natural Language Inference Dataset

by   Hossein Amirkhani, et al.

Natural language inference (NLI) is known as one of the central tasks in natural language processing (NLP) which encapsulates many fundamental aspects of language understanding. With the considerable achievements of data-hungry deep learning methods in NLP tasks, a great amount of effort has been devoted to develop more diverse datasets for different languages. In this paper, we present a new dataset for the NLI task in the Persian language, also known as Farsi, which is one of the dominant languages in the Middle East. This dataset, named FarsTail, includes 10,367 samples which are provided in both the Persian language as well as the indexed format to be useful for non-Persian researchers. The samples are generated from 3,539 multiple-choice questions with the least amount of annotator interventions in a way similar to the SciTail dataset. A carefully designed multi-step process is adopted to ensure the quality of the dataset. We also present the results of traditional and state-of-the-art methods on FarsTail including different embedding methods such as word2vec, fastText, ELMo, BERT, and LASER, as well as different modeling approaches such as DecompAtt, ESIM, HBMP, ULMFiT, and cross-lingual transfer approach to provide a solid baseline for the future research. The best obtained test accuracy is 78.13 current methods to be useful for real-world NLP applications in different languages. The dataset is available at https://github.com/dml-qom/FarsTail.


Choosing Transfer Languages for Cross-Lingual Learning

Cross-lingual transfer, where a high-resource transfer language is used ...

Mukayese: Turkish NLP Strikes Back

Having sufficient resources for language X lifts it from the under-resou...

OYXOY: A Modern NLP Test Suite for Modern Greek

This paper serves as a foundational step towards the development of a li...

Dataset Geography: Mapping Language Data to Language Users

As language technologies become more ubiquitous, there are increasing ef...

On Evaluation of Bangla Word Analogies

This paper presents a high-quality dataset for evaluating the quality of...

Evaluating Language Tools for Fifteen EU-official Under-resourced Languages

This article presents the results of the evaluation campaign of language...

Transition to Adulthood for Young People with Intellectual or Developmental Disabilities: Emotion Detection and Topic Modeling

Transition to Adulthood is an essential life stage for many families. Th...

Please sign up or login with your details

Forgot password? Click here to reset