MT Evaluation

MT: Machine Translation

Overview

Machine translation evaluation studies how well translations preserve meaning and meet linguistic, cultural, and practical needs, helping researchers compare systems and identify failures. Approaches range from reference-based metrics such as BLEU and chrF to neural metrics such as COMET, quality estimation without references, and human assessment using error taxonomies like MQM. Benchmarks increasingly test low-resource languages, specialized domains, document context, cultural nuance, speech, and images, while LLM judges and targeted error checks offer ways to assess qualities that simple text overlap misses. A central challenge is that automatic scores can disagree with human judgments and overlook valid variation, so reliable evaluation must balance scalable metrics with context-sensitive human review.

Latest papers

CardsList

181 papers10 papers in the last four weeks, down 41% on the four weeks before. 0.1% of all new papers.

  1. Rethinking Round-Trip Translation for Machine Translation Evaluation

    Sep 15, 2022Terry Yue Zhuo, Qiongkai Xu, Xuanli He +1MT EvaluationMachine Translation