RuBioRoBERTa: a pre-trained biomedical language model for Russian language biomedical text mining

04/08/2022

∙

by   Alexander Yalunin, et al.

∙

0

∙

share

This paper presents several BERT-based models for Russian language biomedical text mining (RuBioBERT, RuBioRoBERTa). The models are pre-trained on a corpus of freely available texts in the Russian biomedical domain. With this pre-training, our models demonstrate state-of-the-art results on RuMedBench - Russian medical language understanding benchmark that covers a diverse set of tasks, including text classification, question answering, natural language inference, and named entity recognition.