MT Evaluation

MT: Machine Translation

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 106

All topics

Studying quantization trade-offs for efficient inference deployment in machine translation
Jul 31, 2026Jim Zhao, Sohir Maskey, Koen Oostermeijer +2LLM Quantization LLM Inference Acceleration
University of Basel · Aleph Alpha Research · Graphcore
Contrastive ESA: Human Evaluation of Multiple Translations at Once
Jul 29, 2026Vilém Zouhar, Roman Grundkiewicz, Sara Rajaee +6Human-in-the-Loop Annotation MT Evaluation
ETH Zurich · Microsoft · University of Amsterdam +6
Evaluating Prompt Scope and Demonstration Similarity in Local LLM Machine Translation
Jul 28, 2026Mihael ArcanIn-Context Example Selection Multilingual Language Model Evaluation
Home Lab, Galway, Ireland
Systematic Analysis of Large Language Models and Transformer-Based Machine Translation for English-Tamil and Tamil-English Across Diverse Datasets
Jul 27, 2026Sriharshaa S, Sangeetha Sivanesan, Jaya Nirmala SMultilingual Language Models Low-Resource MT
SASTRA Deemed to be University · National Institute of Technology
On the Systematic Challenges of Culturally Loaded Machine Translation: Dream of the Red Chamber as the Cultural Lens
Jul 22, 2026Yiming Wang, Jiayuan DiMultilingual Language Model Evaluation Machine Translation
School of Foreign Languages, East China University of Science and Technology
Can an Old Dog Be Taught New Tricks? Taking LLMs Beyond Sentence Level Translation
Jul 15, 2026Alaina BrandtLLM Prompting Document-Level MT
When Simpler Is Better: Evaluating Translation Pipelines for Medieval Latin Manuscripts
Jul 4, 2026Nguyen Kim Hai Bui, Md. Easin Arafat, Tamás Gábor Orosz +1VLM Evaluation Low-Resource MT
Eötvös Loránd University · King Fahd University of Petroleum and Minerals
MetaHOPE: A Metaphor-Oriented Evaluation Framework for Analysing MT and LLM Translation Errors
Jul 1, 2026Jiahui Liang, Lifeng HanLLM Evaluation Machine Translation
Centre for Linguistics, Humanities, Leiden University, NL · LIACS, Leiden University, NL · BDS, Leiden University Medical Centre, NL
Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer
Jun 29, 2026M. K. ArabovMultilingual Language Models Arabic NLP
Kazan Federal University, Institute of Computational Mathematics and Information Technologies, Department of Data Analysis, Kazan, Russia
Which Tokens Need Context? A Reference-Based Analysis of Translation Responsibility Using Fertility and Entropy
Jun 28, 2026Ramakrishna Appicharla, Baban Gain, Santanu Pal +1Machine Translation MT Evaluation
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India · 2Wipro AI Lab45, Bengaluru, India
AI translation of literary texts is "fine", but readers still prefer human translations
Jun 24, 2026Yves Ferstler, Adam Podoxin, Ty Brassington +5Machine Translation MT Evaluation
Université du Québec à Montréal · Simon Fraser University · Microsoft
Machine Translation and Post-Editing: Comparative Evaluation of Different MT Systems and Post-Editor Groups in Specialised Translation
Jun 22, 2026Joachim Minder, Alexandra Mestivier, Natalie KüblerMachine Translation MT Evaluation
ALTAE, CLILLAC-ARP · Université Paris Cité, ALTAE, F-75013 Paris, France · ALTAE
Evaluating Large Language Models for Hausa and Fongbe Machine Translation: Benchmarks, Failures, and Metric Reliability
Jun 20, 2026Mahounan Pericles Adjovi, Roald Eiselen, Prasenjit MitraLLM Evaluation Low-Resource MT
Carnegie Mellon University Africa, Kigali, Rwanda · North-West University, South Africa
Rubric-as-Experts: Case-Specific MQM Rubrics for Translation Quality Evaluation
Jun 19, 2026Weilu Xu, Yunzhi Shen, Xinye Wang +2LLM Evaluation Rubric-Based Evaluation
National Key Laboratory for Novel Software Technology, Nanjing University
CzechDocs: A Multiway Parallel Dataset of Formatted Documents for Minority Languages in Czechia
Jun 18, 2026Josef Jon, Ondřej BojarDocument-Level MT Machine Translation
Charles University, Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics Prague, Czech Republic
Speaking in Self-Assessing Tongues: On the Verbalized Confidence of LLMs in Machine Translation
Jun 15, 2026Ali Marashian, Alexis Palmer, Katharina von der WenseConfidence Estimation in Language Models Language Model Calibration
University of Colorado Boulder, USA · Johannes Gutenberg University Mainz, Germany
How Far Can Machine Translation Quality Take You? Extrinsic Discourse Evaluation in Goal-Oriented Setups
Jun 15, 2026Wafaa Mohammed, Kata Naszadi, Vlad NiculaeMT Evaluation
Language Technology Lab, University of Amsterdam
Evaluative Judgement in Teaching AI-based Translation: A Class-room Case Study of AI-Mediated Translation and Post-Editing
Jun 13, 2026Gokhan DogruMachine Translation MT Evaluation
Universitat Pompeu Fabra Barcelona, Spain
A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation
Jun 13, 2026Yulin Xue, Siqi Ouyang, Lei LiSpeech Translation Simultaneous Speech Translation
Carnegie Mellon University
Who Brought Easter Eggs to Eid? Auditing LLM-Generated Cultural Translation of Math Word Problems Across Languages and Regions
Jun 9, 2026Parisa Suchdev, Juniper LovatoMultilingual Language Model Evaluation Cultural Bias in Language Models
Computational Ethics Lab · Department of Computer Science, University of Vermont · Vermont Complex Systems Center, University of Vermont +1
Emotion Profiling in LLM-Based Literary Translation: Systematic Shifts Across MT and Post-Editing
Jun 8, 2026Antonio Castaldo, Johanna Monti, Sheila CastilhoMachine Translation MT Evaluation
University of Naples “L’Orientale” · University of Pisa · Antonio Castaldo1,2 +3
Beyond Accuracy: Community Perspectives on Machine Translation
Jun 8, 2026Yujun Wang, Ehud Reiter, Shimei Pan +2Trust in AI Machine Translation
The Aberdeen NLP Research Group University of Aberdeen, UK · University of Maryland, Baltimore County, USA · University of Technology Nuremberg, Germany
HydraQE: OSU's Submission for the IWSLT 2026 Speech Translation Metrics Shared Task
Jun 7, 2026Kevin Krahn, Eric Fosler-LussierSpeech Translation Speech Translation Evaluation
Dept. of Computer Science and Engineering The Ohio State University
A Komi-Yazva--Russian Parallel Corpus and Evaluation Protocol for Zero- and Few-Shot LLM Translation
Jun 4, 2026Petr ParshakovLLM Evaluation Low-Resource MT
HSE University, Perm, Russia · School of Management SKOLKOVO, Moscow, Russia
Ouvia: A User-centered Framework for Measuring Usability of Speech Translation in Real-World Communication Scenarios
Jun 4, 2026Giuseppe Attanasio, Beatrice Savoldi, Daniel Chechelnitsky +4Speech Translation Speech Translation Evaluation
Instituto de Telecomunicações · Instituto Superior Técnico · Fondazione Bruno Kessler +3
ComplexityMT: Benchmarking the Interaction Between Text Complexity and Machine Translation
Jun 3, 2026Joseph Marvin Imperial, Junhong Liang, Belal Shoer +9Machine Translation MT Evaluation
University of Bath · National University Philippines · MBZUAI +7
SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translation
Jun 2, 2026Zeno Testa, Antonino Furnari, Lorenzo Baraldi +1Sign Language Translation Video QA
University of Modena and Reggio Emilia Modena, Italy · University of Catania Catania, Italy · University of Granada, CITIC & DaSCI Institute, Spain
Beyond "To whom it may concern": Tailoring Machine Translation to Audience and Intent
Jun 2, 2026Raphael Merx, Ekaterina Vylomova, Trevor CohnLLM Evaluation Machine Translation
1The University of Melbourne · 2Google
Translating Classical Poetry into Modern Prose
Jun 1, 2026Chalamalasetti Kranti, Sowmya VajjalaLLM Evaluation Machine Translation
Computational Linguistics, University of Potsdam, Germany · National Research Council, Ottawa, Canada
From Outliers to Errors: Auditing Pali-to-English LLM Translations with Multi-Reference Adjudication
May 31, 2026Máté Metzger, Nadnapang Phophichit, Hansa DhammahasoLLM Auditing Machine Translation
Independent Researcher, Hungary · Nibbana Meditation Centre, Thailand

Jul 31, 2026, cs.CLJ/K move · Enter open · S save

Studying quantization trade-offs for efficient inference deployment in machine translation

Jim Zhao, Sohir Maskey, Koen Oostermeijer, Douglas Orr+1

University of Basel · Aleph Alpha Research · Graphcore

LLM Quantization LLM Inference Acceleration Document-Level MT Machine Translation MT Evaluation

Open paper AskPDF arXiv

Deploying large language models in realistic server environments poses challenges, as the system needs to provide high-quality responses with low latency. Quantization is a common approach to reduce the memory footprint and improve inference efficiency, yet its impact on latency and throughput is rarely evaluated under controlled, orchestration-level workloads. In this work we study the quantization trade-offs of two translation model families, EuroLLM \citep{martins2025eurollm} and Hy-MT2 \citep{zheng2026hy} across five models ranging from 1.7B to 22B for efficient deployment on a single A100 or H100 GPU. We demonstrate that combining a document-chunking strategy with W4A8 or W8A8 quantization improves the latency-throughput Pareto-curve under a wide range of workloads. Furthermore, since standard machine translation (MT) benchmarks rely on isolated sentences and fail to capture long-context dynamics, we introduce a document-level evaluation from WMT24++ to assess how text chunking strategies affect translation quality under quantization. Our results reveal that standard segment-level evaluation can fail to predict the interaction between quantization and long-context document translation. While Hy-MT2 remains robust under quantization, EuroLLM shows strong sensitivity and translation quality collapses rapidly for all considered quantization formats. Overall, our experiments show that the trade-off between inference efficiency and translation quality depends not only on the quantization format, but also on the choice of text chunking strategy.

Momentum

Related topics

Latest papers 106