cs.CLNov 10, 2026

Pterodactyl: Pushing the Limits of Efficient Automated Translation Evaluation

Authors: Evelyn Y. Garland, Carola F. Berger

Organizations: Group · Acta Language Services, LLC · CFB Scientific Translations LLC

Abstract

In this paper, we present 1020 Group's submission to the WMT26 shared task on automated quality evaluation. Our system, Pterodactyl, was engineered specifically for subtasks 2 and 3, that is, segment-level quality score prediction and the detection of error-free segments. Pterodactyl is a highly customizable, ultra-low cost system optimized for binary classification in translation triage and data curation. It is based on an LLM-as-a-judge approach, with prompt engineering and model selection guided by ROC analysis. We also developed several variants of Pterodactyl that were specifically adapted for subtasks 3.

Figures & tables

Explore similar work

CardsList
  1. FACET at WMT 2026 Automated Translation Quality Evaluation Task

    Sep 8, 2026Ahrii Kim, Chanjun Park, Seong-heum KimMT EvaluationMachine Translation

  2. AskQE: Question Answering as Automatic Evaluation for Machine Translation

    Apr 15, 2025Dayeon Ki, Kevin Duh, Marine CarpuatMT Evaluation

  3. CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs

    May 15, 2026Kamil Guttmann, Zofia Fraś, Artur Nowakowski +1Small Language ModelsMT Evaluation