Translating historical cultural heritage poses profound challenges for machine translation systems due to the acute semantic asymmetry across eras and cultures, where terse classical syntax frequently obscures domain-specific cultural knowledge. While Large Language Models have demonstrated remarkable prowess in generic translation benchmarks, they struggle to negotiate Culturally-Specific Items and diachronic shifts in domain-rich historical texts, such as Yellow River cultural records. To bridge this critical gap, we introduce YRC-Bench, the first human-curated, multi-parallel benchmark dedicated to historical heritage translation across Classical Chinese, Modern Chinese, and nine target foreign languages. YRC-Bench establishes a tri-level parallel corpus incorporating modern explanatory glosses as explicit semantic bridges and operationalizes translation strategies through paired conserved versus substituted CSI variants categorized by cultural translatability. Systematic evaluations of nine leading LLMs across bidirectional translation directions reveal that leveraging modern Chinese glosses - especially human-curated ones - dramatically elevates both lexical fidelity and semantic adequacy. Furthermore, multidimensional human preference audits by native experts unveil distinct cross-cultural preferences between Sinographic-tradition and non-Sinographic language groups.
Figures & tables
Figure 1: The Tri-Level Architecture and CSI Operationalization in YRC-Bench
Figure 2: Data Distribution of YRC-Bench Dataset
Model
LZH → X
X → LZH
chrF++
COMET
chrF++
COMET
GPT-4o
43.12
0.785
35.67
0.812
Claude 3.5
44.05
0.792
36.81
0.825
Qwen2.5-72B
41.88
0.764
34.22
0.798
DeepSeek-V2
40.52
0.751
33.15
0.784
Table 1: Baseline Direct Translation (DT) performance averaged across all target languages. Best results in bold.
Figure 3: Ablation of Semantic Bridging Efficacy Across Models
Model
LZH-X
X-LZH
BLEU
ChrF++
BERTScore
COMET
BLEU
ChrF++
BERTScore
COMET
Qwen3.8-27B
DT
26.8670
41.7430
0.8374
0.778124
20.5774
16.8792
0.8443
0.788221
HGB
30.2943
45.6224
0.8533
0.804146
37.9433
28.7248
0.8861
0.824925
MGB
27.3879
42.9405
0.8416
0.791304
21.3519
17.5757
0.8393
0.786278
Gemma3-27B
DT
21.0019
36.0892
0.8128
0.736109
11.6991
11.3138
0.8243
0.768620
HGB
25.7769
41.0833
0.8344
0.775935
18.3080
16.1042
0.8539
0.800608
Table 2: Translation performance comparison of four models under three experimental settings (DT, HGB, and MGB) in both LZH-X and X-LZH translation directions. Best results in bold.
Figure 4: Ablation of CSI Resolution Efficacy Across Models
Setting
Acc.
Comp.
Cult.
Flu.
Style
Direction: LZH → X
DT
2.45
2.88
1.95
3.42
2.15
MGB
3.52
3.81
3.10
3.75
3.22
HGB
4.35
4.62
4.45
4.30
4.18
Direction: X → LZH
DT
1.85
2.55
1.62
2.85
1.55
Table 3: Average multidimensional human audit scores (1-5 scale) across all target languages. Human-Gloss Bridged (HGB) exhibits overwhelming superiority, particularly rescuing the Culture and Style dimensions which suffer fatal degradation under Direct Translation (DT). Best results in bold.
Dimension
X → LZH
LZH → X
Count ( T+M / T )
Win%
Count ( T+M / T )
Win%
Accuracy
1012 / 428
69.6%
958 / 475
66.4%
Completeness
830 / 345
66.3%
815 / 352
65.7%
Culture
610 / 126
68.3%
414 / 201
58.2%
Fluency
865 / 444
64.1%
869 / 483
63.1%
Style
924 / 416
67.1%
762 / 361
63.6%
Table 4: Multidimensional pairwise preference breakdown between bridged ( T+M ) and unbridged ( T ) outputs. Win% denotes the preference share for T+M .
Figure 5: Net Human Preference Win-Rate of Bridged (T+M) over Direct (T) across Target Languages
Tradition
Forced Condition
Possible Condition
Conserved (%)
Substituted (%)
Conserved (%)
Substituted (%)
Sinographic
57.83±21.19
42.17±21.19
66.38±7.55
33.62±7.55
Non-Sinographic
78.56±18.80
21.44±18.80
20.76±7.32
79.24±7.32
Table 5: Expert translation strategy choices (%) under Forced and Possible CSI conditions across Sinographic (JA, KO, VI) and Non-Sinographic traditions.
Figure 6: Translation Strategy Preference Share (%) under Forced and Possible Conditions
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Model(DT)
LZH-X
X-LZH
BLEU
ChrF++
BERTScore
COMET
BLEU
ChrF++
BERTScore
COMET
Qwen3-8B
16.4975
32.4854
0.8008
0.711960
12.2196
11.5365
0.8101
0.759374
Qwen3.8-27B
26.8670
41.7430
0.8374
0.778124
20.5774
16.8792
0.8443
0.788221
Seed-X-PPO
16.0290
41.8972
0.8083
0.728496
6.8269
8.9710
0.7342
0.750249
Seed-X-Instruct
14.6113
31.3028
0.7834
0.730740
5.9654
9.5262
0.7247
0.720437
Qwen3.7
26.2831
41.1017
0.8373
0.777979
23.9789
19.5319
0.8444
0.789696
Appendix
Table 6: Translation performance comparison of nine models under the DT setting in both LZH-X and X-LZH translation directions. Best results in bold.
Model(HGB)
LZH-X
X-LZH
BLEU
ChrF++
BERTScore
COMET
BLEU
ChrF++
BERTScore
COMET
Qwen3-8B
12.2149
24.3329
0.7734
0.645202
29.0430
23.8852
0.8632
0.814734
Qwen3.8-27B
30.2943
45.6224
0.8533
0.804146
37.9433
28.7248
0.8861
0.824925
Seed-X-PPO
10.8524
33.0707
0.7287
0.679762
7.1642
11.8485
0.6925
0.691651
Seed-X-Instruct
10.7300
32.1508
0.7247
0.667093
8.9870
13.6161
0.7075
0.693047
Qwen3.7
29.7738
44.9143
0.8520
0.799462
42.4629
32.8775
0.8873
0.829411
Appendix
Table 7: Translation performance comparison of nine models under the HGB setting in both LZH-X and X-LZH translation directions. Best results in bold.
Figure 7: Heatmap results comparing translation performance across different evaluation metrics and translation directions.
Figure 8: Direct Translation (DT) and Human-Gloss Bridged performance (HGB) across different models in LZH–X and X–LZH translation directions.
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.
Large language model (LLM)-based machine translation has advanced cross-cultural communication, yet it still struggles with culture-loaded words (CLWs) in ancient Chinese texts. The challenge extends beyond lexical alignment to deciding when and how culture-dependent knowledge should be explicated for readers lacking relevant background. Literal translation often preserves surface forms while missing underlying concepts, whereas over-explicitation harms conciseness and readability. To address this problem, we formulate CLW translation as a selective explicitation task and propose MACAT, a Multi-Agent Culture-Aware Translation framework that dynamically identifies culturally salient phrases and injects concise explanatory knowledge when necessary. MACAT further incorporates a quality-aware reranking module for candidate selection and a multi-round evaluation agent that assesses translations across terminological precision, readability, fidelity, cultural preservation, and cultural explicitation. Experiments on traditional Chinese medicine (TCM) classics and the Analects show that, under a unified GPT-5.4 evaluation setting, MACAT consistently outperforms both the backbone model and general-purpose MT baselines on 100 TCM documents and a 20-chapter subset of the Analects.
Xiaoqi He, Kaixin Lan, Mu You +3
NLP2CT Lab, Department of Computer and Information Science, University of Macau · Institute of International Language Services Studies, Macau Millennium College
Historical archives are invaluable for multidisciplinary research but remain inaccessible globally due to severe language barriers. While manual translation is prohibitively expensive and time-consuming, simply applying existing machine translation models is inadequate due to unique linguistic nuances and the temporal bias inherent in scarce human-labeled corpora. To address these challenges, we propose HERIT, a systematic domain-adaptation pipeline that leverages retrieval-augmented generation (RAG) to generate high-quality pseudo-labeled data from abundant Hanja-Korean corpora. This approach substantially expands the training dataset, mitigating both data scarcity and temporal overfitting across different historical periods. Extensive automated and human evaluations demonstrate that HERIT consistently outperforms baselines. Finally, we translate previously untranslated portions of the archives using our model, thereby enabling global access to these resources for researchers.
Dayeon Jeong, Kyeongpil Kang
Graduate School of Data Science, Kangwon National University · Department of Computer Science and Engineering, Kangwon National University
Large language models (LLMs) have achieved strong performance in general machine translation, yet their ability in culture-aware scenarios remains poorly understood. To bridge this gap, we introduce CanMT, a Culture-Aware Novel-Driven Parallel Dataset for Machine Translation, together with a theoretically grounded, multi-dimensional evaluation framework for assessing cultural translation quality. Leveraging CanMT, we systematically evaluate a wide range of LLMs and translation systems under different translation strategy constraints. Our findings reveal substantial performance disparities across models and demonstrate that translation strategies exert a systematic influence on model behavior. Further analysis shows that translation difficulty varies across types of culture-specific items, and that a persistent gap remains between models' recognition of culture-specific knowledge and their ability to correctly operationalize it in translation outputs. In addition, incorporating reference translations is shown to substantially improve evaluation reliability in LLM-as-a-judge, underscoring their essential role in assessing culture-aware translation quality. The corpus and code are available at CanMT.
Zekun Yuan, Yangfan Ye, Xiaocheng Feng +5
Harbin Institute of Technology · Peng Cheng Laboratory · Huawei Technologies Co., Ltd