MT Evaluation

MT: Machine Translation

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 106

All topics

Model-Based Quality Assessment for Massively Multilingual Parallel Data
May 29, 2026Abdelaziz M. A. Ibrahim, Zihao Li, Jörg Tiedemann +1Sentence Embeddings MT Evaluation
University of Jyväskylä · University of Helsinki · ELLIS Institute Finland +1
Translation Analytics for Freelancers II: Benchmarking Local LLMs for Confidential Translation Workflows
May 29, 2026Yuri Balashov, Rex VanHorn, Mingxi Xu +1Multilingual Language Model Evaluation On-Device Language Model Inference
University of Georgia, Athens, Georgia, USA
Unlocking Fine-Grained Translation Quality Estimation in LRMs through Mutually Boosting Implicit and Explicit Reasoning
May 29, 2026Renfei Dang, Xinye Wang, Zhejian Lai +5MT Evaluation Large Reasoning Models
National Key Laboratory for Novel Software Technology, Nanjing University · Huawei Translation Services Center, Beijing, China
AfriScience-MT: Towards Decolonizing Science in Africa through Text Translation
May 28, 2026Idris Abdulmumin, Tajuddeen Gwadabe, Shamsuddeen Hassan Muhammad +11Scientific Communication Low-Resource MT
Data Science for Social Impact, University of Pretoria · Masakhane Research Foundation · Imperial College London +9
Comparative Evaluation of Machine Translation Systems on Images with Text
May 28, 2026Blai Puchol, Sergio Gómez González, Miguel Domingo +1VLM Evaluation Machine Translation
PRHLT Research Center - Universitat Polit`ecnica de Val`encia, Camino de Vera, s/n - Valencia · ValgrAI - Valencian Graduate School and Research Network for Artificial Intelligence, Camino de Vera s/n
HardMTBench: Stress-Testing Chinese-English Translation on Knowledge-Intensive Domains
May 27, 2026Zheng Li, Mao Zheng, Mingyang Song +1Machine Translation MT Evaluation
Large Language Model Department, Tencent
Why We Need Speech to Evaluate Speech Translation
May 27, 2026Maike Züfle, Danni Liu, Vilém Zouhar +1Speech Translation Speech Prosody
Karlsruhe Institute of Technology · ETH Zurich
IFMTBench: A Comprehensive Benchmark for Multilingual Translation Instruction Following
May 27, 2026Mingrui Sun, Mao Zheng, Zheng Li +1Multilingual Language Model Evaluation Machine Translation
Large Language Model Department, Tencent
Cultural Fidelity in English-to-Hindi Translation: A Preservation-Fluency Frontier for Gender Recoverability
May 26, 2026Samyak Savi, Chavi Gupta, Shreyas Gantayet +2Cultural Alignment Machine Translation
BITS Pilani
Testing the Deliteralization Hypothesis in Human and Machine Translation
May 25, 2026Malik Marmonier, Rachel Bawden, Benoît SagotMachine Translation MT Evaluation
Inria, Paris, France
Beyond Literal Translation: Evaluating Cultural Effectiveness in Social Media UGC
May 25, 2026Linjuan Wu, Ruiqi Zhang, Xinze Lyu +7Cultural Alignment Machine Translation
Zhejiang University · Xiaohongshu Inc. · Fudan University.
Quantifying the Impact of Translation Errors on Multilingual LLM Evaluation
May 24, 2026Klaudia-Doris Thellmann, Bernhard Stadler, Michael Färber +1Multilingual Language Model Evaluation MT Evaluation
TUD Dresden University of Technology · 2InfAI e.V. · 3Amazon +1
ROC Analysis for Evaluating Translation Quality Estimation Systems
May 23, 2026Evelyn Y. Garland, Carola F. BergerMT Evaluation
Acta Language Services, LLC · CFB Scientific Translations LLC
Metaphors in Literary Post-Editing: Opening Pandora's Box?
May 20, 2026Aletta G. Dorst, Mayra O. Nas, Katinka ZevenMachine Translation MT Evaluation
Leiden University Centre for Linguistics
Ancient Greek to Modern Greek Machine Translation: A Novel Benchmark and Fine-Tuning Experiments on LLMs and NMT Models
May 18, 2026Spyridon Mavromatis, Sokratis Sofianopoulos, Prokopis Prokopidis +1LLM Fine-Tuning Low-Resource MT
National and Kapodistrian University of Athens, Department of Informatics and Telecommunications · Institute for Language and Speech Processing, Athena RC
PaliBench: A Multi-Reference Blueprint for Classical Language Translation Benchmarks
May 16, 2026Máté Metzger, Nadnapang PhophichitLLM Evaluation Machine Translation
Independent Researcher · International Buddhist Studies College · Mahachulalongkornrajavidyalaya University
ForMaT: Dataset for Visually-Grounded Multilingual PDF Translation
May 15, 2026Michał Ciesiółka, Dawid Wiśniewski, Adrian Charkiewicz +1Machine Translation MT Evaluation
Laniqo, Pozna´n, Poland · Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Pozna´n, Poland · Pozna´n University of Technology
CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs
May 15, 2026Kamil Guttmann, Zofia Fraś, Artur Nowakowski +1Small Language Models MT Evaluation
Laniqo, Pozna´n, Poland · Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Pozna´n, Poland
Fluency and Faithfulness in Human and Machine Literary Translation
May 14, 2026Sarah Griebel, Ted UnderwoodMachine Translation MT Evaluation
School of Information Sciences University of Illinois Urbana–Champaign
AI-assisted cultural heritage dissemination: Comparing NMT and glossary-augmented LLM translation in rock art documents
May 14, 2026Vicent Briva-Iglesias, María Ferre-FernándezLLM Prompting Cultural Heritage
SALIS, CTTS, ADAPT Centre · Dublin City University · Universidad de Almer´ıa
Creativity Bias: How Machine Evaluation Struggles with Creativity in Literary Translations
May 13, 2026Kyo Gerrits, Rik van Noord, Ana Guerberof ArenasCreativity Assessment LLM-as-a-Judge
Centre for Language and Cognition, University of Groningen
ATD-Trans: A Geographically Grounded Japanese-English Travelogue Translation Dataset
May 13, 2026Shohei Higashiyama, Hiroki Ouchi, Atsushi Fujita +1Machine Translation MT Evaluation
National Institute of Information and Communications Technology · Nara Institute of Science and Technology
Dynamic Meta-Metrics: Source-Sentence Conditioned Weighting for MT Evaluation
May 9, 2026Luke Zhang, Justin Vasselli, Aditya Khan +2Metric Learning MT Evaluation
University of Toronto, Canada · Nara Institute of Science and Technology, Japan · Ontario Tech University, Canada
Why do Large Language Models Fail in Low-resource Translation? Unraveling the Token Dynamics of Large Language Models for Machine Translation
May 8, 2026Shenbin Qian, Yves ScherrerMultilingual Language Model Evaluation Low-Resource MT
Language Technology Group, Department of Informatics University of Oslo, Norway
Data Contamination in Neural Hieroglyphic Translation: A Reproducibility Study
May 8, 2026Ammar Toutou, Abdelrahman Harb, Christine BastaBenchmark Contamination Low-Resource MT
Computer Science and Engineering, Alamein International University (AIU), Egypt · HiTZ Center, University of the Basque Country, Spain · Faculty of Computers and Data Science, Alexandria University, Egypt
Nsanku: Evaluating Zero-Shot Translation Performance of LLMs for Ghanaian Languages
May 5, 2026Stephen E. Moore, Mich-Seth Owusu, Akwasi Asare +13Multilingual Language Model Evaluation Open-Weight Language Models
Department of Mathematics, University of Cape Coast, Cape Coast, Ghana · Ghana Natural Language Processing, Cape Coast, Cape Coast, Ghana · Khaya AI LTD Ghana
VIDA: A Dataset for Visually Dependent Ambiguity in Multimodal Machine Translation
May 3, 2026Jingheng Pan, Xintong Wang, Longyue Wang +3Large Vision-Language Models Multimodal Model Evaluation
Department of Informatics, Universität Hamburg · Alibaba Group
Is Textual Similarity Invariant under Machine Translation? Evidence Based on the Political Manifesto Corpus
May 1, 2026Daria Boratyn, Damian Brzyski, Albert Leśniak +5Semantic Textual Similarity Text Embedding Evaluation
aJagiellonian Center for Quantitative Political Science, Jagiellonian University, Kraków, Poland · bAGH University, Kraków, Poland · cJagiellonian Center for Digital Humanities, Jagiellonian University, Kraków, Poland
Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues
Apr 30, 2026Muhammad Dehan Al Kautsar, Saeed Almheiri, Momina Ahsan +13Multilingual Language Model Evaluation Arabic NLP
Mohamed bin Zayed University of Artificial Intelligence · IBM Research AI · American University in the Emirates
Beyond Semantics: Measuring Fine-Grained Emotion Preservation in Small Language Model-Based Machine Translation
Apr 30, 2026Dawid Wisniewski, Igor CzudySmall Language Models Machine Translation
Pozna´n University of Technology

May 29, 2026, cs.CLJ/K move · Enter open · S save

Model-Based Quality Assessment for Massively Multilingual Parallel Data

Abdelaziz M. A. Ibrahim, Zihao Li, Jörg Tiedemann, Shaoxiong Ji

University of Jyväskylä · University of Helsinki · ELLIS Institute Finland+1

Sentence Embeddings MT Evaluation

Open paper AskPDF arXiv

Large-scale multilingual bitext often contains two distinct problems: non-parallel sentence pairs and low-quality translations. We decompose model-based assessment for such data into two independent components: parallelism assessment with multilingual embeddings and reference-free quality estimation (QE). For parallelism, we benchmark four embedding models on FLORES-200 and BOUQuET retrieval tasks, covering 6,654 source--target directions in our target language-pair inventory. For QE, we evaluate nine reference-free evaluators on professional FLORES-200 translations across 41,412 ordered source--target directions. Results show that no model is universally reliable across translation directions. Naive QE ensembles dilute strong model signals, while documented target-language coverage is strongly associated with higher QE scores. Overall, these findings suggest that multilingual parallel-data assessment is best approached as a direction-aware routing and calibration problem, where no single universal metric is expected to suffice across all languages.

Momentum

Related topics

Latest papers 106