Revisiting Round-Trip Translation for Quality Estimation

04/29/2020
by   Jihyung Moon, et al.
0

Quality estimation (QE) is the task of automatically evaluating the quality of translations without human-translated references. Calculating BLEU between the input sentence and round-trip translation (RTT) was once considered as a metric for QE, however, it was found to be a poor predictor of translation quality. Recently, various pre-trained language models have made breakthroughs in NLP tasks by providing semantically meaningful word and sentence embeddings. In this paper, we employ semantic embeddings to RTT-based QE. Our method achieves the highest correlations with human judgments, compared to previous WMT 2019 quality estimation metric task submissions. While backward translation models can be a drawback when using RTT, we observe that with semantic-level metrics, RTT-based QE is robust to the choice of the backward translation system. Additionally, the proposed method shows consistent performance for both SMT and NMT forward translation systems, implying the method does not penalize a certain type of model.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/31/2021

Quality Estimation Using Round-trip Translation with Sentence Embeddings

Estimating the quality of machine translation systems has been an ongoin...
research
09/17/2021

Classification-based Quality Estimation: Small and Efficient Models for Real-world Applications

Sentence-level Quality estimation (QE) of machine translation is traditi...
research
03/15/2022

Compressing Sentence Representation for Semantic Retrieval via Homomorphic Projective Distillation

How to learn highly compact yet effective sentence representation? Pre-t...
research
02/22/2019

OpenKiwi: An Open Source Framework for Quality Estimation

We introduce OpenKiwi, a Pytorch-based open source framework for transla...
research
10/21/2022

SIT at MixMT 2022: Fluent Translation Built on Giant Pre-trained Models

This paper describes the Stevens Institute of Technology's submission fo...
research
05/31/2021

Verdi: Quality Estimation and Error Detection for Bilingual

Translation Quality Estimation is critical to reducing post-editing effo...
research
06/12/2019

Does BLEU Score Work for Code Migration?

Statistical machine translation (SMT) is a fast-growing sub-field of com...

Please sign up or login with your details

Forgot password? Click here to reset