We present PatMT2 (Patent Machine Translation, v2), a two-stage LoRA pipeline that adapts the Hy-MT2-7B dedicated translation model to four-language (Chinese/English/Japanese/Korean) patent translation. On six translation directions, the Stage-2 checkpoint reaches a macro-average BLEU of 43.39 (BF16) to 43.57 (AWQ INT4), improving over the Hy-MT2-7B base by up to +8.8 BLEU and also exceeding Qwen3-32B and several external systems on the same average. We then measure the cost of adaptation: probing 38 languages absent from the fine-tuning data, the model still produces correct, fluent output on 34, while four low-resource languages collapse into distinct, reproducible failure modes. We treat these as diagnostic cases rather than retention rates, and report their mechanisms without attributing them to a single cause. Paired bootstrap tests confirm the translation gains over all baselines (p < 0.001).
Figures & tables
Figure 1: Two-stage fine-tuning pipeline for PatMT2-7B. Stage 1 trains on short-text data; Stage 2 adds long-text mixed with skill-retention short-text. Both stages use attention-only LoRA adapters on the Hy-MT2-7B base.
Parameter
Stage 1
Stage 2
Base model
Hy-MT2-7B
Stg1 merged
LoRA rank/alpha
32/64
32/64
Dropout
0.1
0.1
Target modules
q/k/v/o
q/k/v/o
Learning rate
5×10−5
2×10−5
Schedule
cosine
cosine
Table 1: Two-stage training configuration.
Model
z → e
e → z
z → j
j → z
z → k
k → z
Avg
HunYuan-MT-Chimera
34.19
32.73
32.16
33.46
28.28
30.13
31.83
Hy-MT2-7B (base)
36.67
32.87
35.23
37.46
30.49
34.79
34.58
Qwen3-32B (base)
39.56
37.72
38.23
45.59
32.67
39.60
38.90
PatMT-v1 (BF16)
36.83
39.42
41.32
46.70
38.69
43.65
41.10
Qwen3.6-27B-FP8
41.07
39.84
41.16
49.54
35.69
39.25
41.09
PatMT2 BF16
40.31
42.96
48.65
48.38
36.44
43.59
43.39
Table 2: BLEU across six directions (4,000 pairs/dir). Bold = column best.
Figure 2: Multilingual retention probe: 34 of 38 never-fine-tuned languages produce correct output, and 4 fail with distinct mechanisms. One fixed source sentence per language; this figure is a fault-case map, not a retention-rate estimate.
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.
We present PatTransBench, a 12,000-item Chinese-English/Japanese/Korean patent machine translation (PatMT) benchmark with a four-dimensional evaluation protocol. The benchmark comprises 6,000 short-text (sentence/claim) and 6,000 long-text (paragraph) pairs, evenly split across three language pairs and both translation directions, built via IPC-section-stratified sampling and dualthreshold quality gating from upstream patent-family parallel corpora. The evaluation protocol spans literal n-gram overlap (BLEU, ROUGE-L, chrF), cross-lingual semantic textual similarity (CLSTS), 38-probe-language consistency testing, and serving performance under concurrency. A case study on PatMT2-7B and its fp8/AWQ4/AutoRound-4bit variants reveals direction-dependent quantization effects masked by aggregate scores and non-monotonic language-failure patterns across quantization schemes that are invisible to single-metric evaluation. We plan to release the benchmark, protocol, and case-study results upon publication to support reproducible PatMT evaluation.
Xiaowei Su, Xusen Cheng
Renmin University of China, Beijing, China · Beijing Bayuegua Technology Co., Ltd., Beijing, China
Recent multilingual large language models achieve strong translation performance, yet the internal mechanisms responsible for low-resource multilingual translation remain insufficiently understood. In this paper, we study translation circuits in multilingual decoder-only LLMs for Hindi-to-Urdu and Hindi-to-Sindhi (Devanagari). We propose TITAN, a mechanistic framework that combines causal intervention, sparse feature decomposition, and targeted preference optimization to identify and optimize crucial translation components in the transformer. Unlike conventional full-parameter fine-tuning, TITAN restricts optimization to sparse, causally important circuits, enabling scalable and parameter-efficient adaptation while updating fewer than 5% of model parameters. Experimental results show that TITAN achieves BLEU improvements of 33.75% on Hindi-to-Urdu and 45.86% on Hindi-to-Sindhi over the strongest multilingual baselines. Human expert evaluation further confirms substantial gains in adequacy and fluency across both language pairs. Additional analyses reveal that multilingual translation is implemented through sparse, structured circuits operating over a shared semantic subspace rather than through an English-centric pivot representation. Overall, our findings demonstrate that mechanistic circuit discovery can directly improve low-resource multilingual translation while providing interpretable insights into multilingual transformer behavior.
Atai Waris Khan, Abdullah Mazhar, Raghu Kumar +1
Indraprastha Institute of Information Technology Delhi
Machine Translation (MT) for low-resource languages remains far behind that of high-resource languages, and the gap is widest in specialised domains, where parallel data is scarce or entirely absent. We present EnSiTa, a trilingual multi-domain parallel dataset and benchmark for English, Sinhala and Tamil. EnSiTa provides human post-edited training data for seven domains, plus manually translated test sets for those and one additional domain, all produced by professional translators under a multi-year, rigorously quality-controlled process. Using this dataset, we conduct an extensive study of domain-specific MT for all six language directions, fine-tuning a from-scratch Transformer, a pre-trained translation model (NLLB-600M), and decoder-only LLMs (Gemma 3 family, 1B-12B, and TranslateGemma) across training-data sizes, model scales, and in-domain, cross-domain, multilingual and multi-domain settings. To the best of our knowledge, this is the most extensive systematically documented multi-domain parallel data creation and benchmarking effort for low-resource MT. Our data and models will be publicly released.
Surangika Ranathunga, Nisansa de Silva, Aloka Fernando +8
Massey University · University of Moratuwa · National University of Singapore +3