MT Evaluation

Machine translation evaluation studies how well translations preserve meaning and meet linguistic, cultural, and practical needs, helping researchers compare systems and identify failures. Approaches range from reference-based metrics such as BLEU and chrF to neural metrics such as COMET, quality estimation without references, and human assessment using error taxonomies like MQM. Benchmarks increasingly test low-resource languages, specialized domains, document context, cultural nuance, speech, and images, while LLM judges and targeted error checks offer ways to assess qualities that simple text overlap misses. A central challenge is that automatic scores can disagree with human judgments and overlook valid variation, so reliable evaluation must balance scalable metrics with context-sensitive human review.