PATMT2: Two-stage domain adaptation for multilingual patent translation

Xiaowei Su; Xusen Cheng

cs.CLOct 12, 2026

PATMT2: Two-stage domain adaptation for multilingual patent translation

Authors: Xiaowei Su, Xusen Cheng

Organizations: Renmin University of China, Beijing, China · Beijing Bayuegua Technology Co., Ltd., Beijing, China

Abstract

We present PatMT2 (Patent Machine Translation, v2), a two-stage LoRA pipeline that adapts the Hy-MT2-7B dedicated translation model to four-language (Chinese/English/Japanese/Korean) patent translation. On six translation directions, the Stage-2 checkpoint reaches a macro-average BLEU of 43.39 (BF16) to 43.57 (AWQ INT4), improving over the Hy-MT2-7B base by up to +8.8 BLEU and also exceeding Qwen3-32B and several external systems on the same average. We then measure the cost of adaptation: probing 38 languages absent from the fine-tuning data, the model still produces correct, fluent output on 34, while four low-resource languages collapse into distinct, reproducible failure modes. We treat these as diagnostic cases rather than retention rates, and report their mechanisms without attributing them to a single cause. Paired bootstrap tests confirm the translation gains over all baselines (p < 0.001).

Figures & tables

Figure 1: Two-stage fine-tuning pipeline for PatMT2-7B. Stage 1 trains on short-text data; Stage 2 adds long-text mixed with skill-retention short-text. Both stages use attention-only LoRA adapters on the Hy-MT2-7B base.

Parameter	Stage 1	Stage 2
Base model	Hy-MT2-7B	Stg1 merged
LoRA rank/alpha	32/64	32/64
Dropout	0.1	0.1
Target modules	q/k/v/o	q/k/v/o
Learning rate	$5{\times}10^{-5}$	$2{\times}10^{-5}$
Schedule	cosine	cosine

Table 1: Two-stage training configuration.

Model	z $\to$ e	e $\to$ z	z $\to$ j	j $\to$ z	z $\to$ k	k $\to$ z	Avg
HunYuan-MT-Chimera	34.19	32.73	32.16	33.46	28.28	30.13	31.83
Hy-MT2-7B (base)	36.67	32.87	35.23	37.46	30.49	34.79	34.58
Qwen3-32B (base)	39.56	37.72	38.23	45.59	32.67	39.60	38.90
PatMT-v1 (BF16)	36.83	39.42	41.32	46.70	38.69	43.65	41.10
Qwen3.6-27B-FP8	41.07	39.84	41.16	49.54	35.69	39.25	41.09
PatMT2 BF16	40.31	42.96	48.65	48.38	36.44	43.59	43.39

Table 2: BLEU across six directions (4,000 pairs/dir). Bold = column best.

Figure 2: Multilingual retention probe: 34 of 38 never-fine-tuned languages produce correct output, and 4 fail with distinct mechanisms. One fixed source sentence per language; this figure is a fault-case map, not a retention-rate estimate.

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

PatTransBench: an ipc-stratified, multi-dimensional benchmark for patent machine translation evaluation
Oct 12, 2026Xiaowei Su, Xusen ChengMT Evaluation
Renmin University of China, Beijing, China · Beijing Bayuegua Technology Co., Ltd., Beijing, China
Translation Circuit Discovery and Targeted Optimization for Low-Resource Multilingual Translation
Oct 15, 2026Atai Waris Khan, Abdullah Mazhar, Raghu Kumar +1Multilingual Language Models Low-Resource MT
Indraprastha Institute of Information Technology Delhi
EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation
Aug 25, 2026Surangika Ranathunga, Nisansa de Silva, Aloka Fernando +8Machine Translation Low-Resource MT
Massey University · University of Moratuwa · National University of Singapore +3

Oct 12, 2026J/K move · Enter open · S save

PatTransBench: an ipc-stratified, multi-dimensional benchmark for patent machine translation evaluation

Xiaowei Su, Xusen Cheng

Renmin University of China, Beijing, China · Beijing Bayuegua Technology Co., Ltd., Beijing, China

MT Evaluation

Open paper AskPDF arXiv

We present PatTransBench, a 12,000-item Chinese-English/Japanese/Korean patent machine translation (PatMT) benchmark with a four-dimensional evaluation protocol. The benchmark comprises 6,000 short-text (sentence/claim) and 6,000 long-text (paragraph) pairs, evenly split across three language pairs and both translation directions, built via IPC-section-stratified sampling and dualthreshold quality gating from upstream patent-family parallel corpora. The evaluation protocol spans literal n-gram overlap (BLEU, ROUGE-L, chrF), cross-lingual semantic textual similarity (CLSTS), 38-probe-language consistency testing, and serving performance under concurrency. A case study on PatMT2-7B and its fp8/AWQ4/AutoRound-4bit variants reveals direction-dependent quantization effects masked by aggregate scores and non-monotonic language-failure patterns across quantization schemes that are invisible to single-metric evaluation. We plan to release the benchmark, protocol, and case-study results upon publication to support reproducible PatMT evaluation.

Abstract

Machine Translation

Explore similar work

PatTransBench: an ipc-stratified, multi-dimensional benchmark for patent machine translation evaluation

Translation Circuit Discovery and Targeted Optimization for Low-Resource Multilingual Translation

EnSiTa - A Trilingual Multi-Domain Parallel Dataset and Benchmark for Domain-Specific Machine Translation