Large Language Models (LLMs) remain unreliable for extremely low-resource languages whose writing systems conflict with high-resource pretraining priors. We study Bai-Chinese translation in the Bowen script, where Chinese characters are frequently borrowed phonetically rather than semantically, causing severe literal-translation hallucinations. We propose MAS-Bai, a linguistically informed adaptive multi-agent framework for Bowen-script Bai-Chinese translation. MAS-Bai routes each input through either a lightweight retrieval-translation path or a deeper philological path with etymological and syntactic agents, and uses cycle-consistency with a lightweight reverse translator as an inference-time reliability signal. We construct a 6,536-sentence Bai-Chinese corpus and evaluate on a 1,536-sentence test set. MAS-Bai achieves 16.23 BLEU and 62.15 COMET, improving over Standard RAG by 8.11 BLEU and 6.85 COMET. Forward LoRA, expert ablations and controlled routing experiments distinguish the roles of supervised adaptation, Bai-specific expertise and conditional routing. The largest gains occur on Bowen phonetic loans and divergent Bai word order, while human evaluation and reverse-verifier analyses support the automatic results.
Figures & tables
Figure 1: Architecture of MAS-Bai. The Meta-Agent routes inputs by Bowen-loan and syntax-marker counts to either a retrieval–translation path or a philological expert path. Cycle-consistency failures trigger topology modification for up to T=3 iterations.
Mismatch signal
Topology update
Bowen-loan markers missing or altered in x^′
Activate or re-run Aety and require explicit phonetic-loan glosses.
SOV or postpositional structure not recovered
Activate or re-run Asyn and require a reordered intermediate representation.
Both lexical and syntactic mismatches occur
Switch to the full quality-oriented pathway: Aety+Asyn+Aretr+Atrans .
Low semantic similarity without lexical/syntactic mismatch
Refresh retrieved examples and re-run Aretr and Atrans .
Repeated failure after T iterations
Return the candidate with the highest C(x,x^′) .
Table 1: Topology modification policy in the self-evolution loop.
Table 2: Main results on the Bai–Chinese test set. Qwen2.5-72B denotes Qwen-2.5-72B-Instruct. COMET scores are scaled by 100.
Configuration
BLEU
ChrF++
COMET
MAS-Bai (Full)
16.23
35.74
62.15
Ablation of Philological Agents
w/o Etymologist
11.45
28.12
51.50
w/o Syntactician
13.08
30.55
55.88
Ablation of Adaptive Routing
Static MAS
14.51
32.19
58.23
Table 3: Ablation study of MAS-Bai components.
Routing policy
BLEU
COMET
Avg. tokens
Uniform random
14.85
59.10
1012
Static MAS
14.51
58.23
1250
MAS-Bai routing
16.23
62.15
775
COMET oracle
16.65
62.80
–
Table 4: Routing-policy comparison. All variants use the same backbone, expert pool, prompts, retrieval base, decoding settings and cycle-consistency configuration. Avg. tokens is the mean cumulative number of input and output tokens across all invoked agents and refinement rounds, counted with the Qwen-2.5 tokenizer. The COMET oracle is retrospective and is included only as a diagnostic upper bound.
Subset
n
Std. RAG
MAS-Bai
Δ
Full test set
1,536
55.30
62.15
+6.85
Register subsets
Dialogue-like
512
60.15
64.20
+4.05
Formal narrative
484
56.40
62.80
+6.40
Lyric / verse
540
49.75
59.60
+9.85
Linguistic phenomenon subsets
Table 5: COMET scores by register and linguistic phenomenon. COMET scores are scaled by 100.
Method
Adeq. ↑
Flu. ↑
Halluc. ↓
GPT-4o (0-shot)
1.84
4.12
76.0%
Standard RAG
3.25
3.95
38.0%
MAS-Bai
4.42
4.55
8.0%
Table 6: Human evaluation on 100 test sentences. Adequacy and fluency are averaged across three annotators.
Source
Units
Tokens
Role
Published Dabenqv collection
4,986
∼ 75K
Existing Bai–Chinese sentence pairs
Handwritten Bowen manuscript
1,552
∼ 12K
Bai sentences translated into Chinese
Candidate units
6,538
∼ 87K
Before final alignment and quality control
Final aligned corpus
6,536
–
After sentence-level quality control
Table A1: Corpus source statistics. Token counts are source-side estimates recorded during data collection.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A1: Distribution of computational pathways chosen by the Meta-Agent for simple and complex inputs.
Pathway Class
Precision
Recall
F1
Support
Simple (Efficiency)
87.6
90.2
88.9
102
Complex (Quality)
89.5
86.7
88.1
98
Overall Accuracy
88.5
200
Appendix
Table A2: Meta-Agent routing accuracy against human expert annotations. Metrics are percentages.
Figure A2: COMET score versus token consumption for adaptive MAS-Bai and Static MAS.
Setting
Training pairs
BLEU
COMET
Circularity control
No self-evolution
–
12.87
54.91
Shared verifier
5,000
16.23
62.15
Disjoint verifier
1,000
15.88
61.34
Verifier strength variation, fixed 5,000-pair retrieval base
Weak verifier
250
14.12
57.85
Appendix
Table A3: Cycle-consistency reliability under verifier-data separation and verifier-strength variation.
τ
T
BLEU
COMET
0.80
3
15.98
61.85
0.85
3
16.23
62.15
0.90
3
16.15
61.95
0.85
5
17.27
62.18
Appendix
Table A4: Sensitivity to the cycle-consistency acceptance threshold and iteration cap.
Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.
Large language model (LLM)-based machine translation has advanced cross-cultural communication, yet it still struggles with culture-loaded words (CLWs) in ancient Chinese texts. The challenge extends beyond lexical alignment to deciding when and how culture-dependent knowledge should be explicated for readers lacking relevant background. Literal translation often preserves surface forms while missing underlying concepts, whereas over-explicitation harms conciseness and readability. To address this problem, we formulate CLW translation as a selective explicitation task and propose MACAT, a Multi-Agent Culture-Aware Translation framework that dynamically identifies culturally salient phrases and injects concise explanatory knowledge when necessary. MACAT further incorporates a quality-aware reranking module for candidate selection and a multi-round evaluation agent that assesses translations across terminological precision, readability, fidelity, cultural preservation, and cultural explicitation. Experiments on traditional Chinese medicine (TCM) classics and the Analects show that, under a unified GPT-5.4 evaluation setting, MACAT consistently outperforms both the backbone model and general-purpose MT baselines on 100 TCM documents and a 20-chapter subset of the Analects.
Xiaoqi He, Kaixin Lan, Mu You +3
NLP2CT Lab, Department of Computer and Information Science, University of Macau · Institute of International Language Services Studies, Macau Millennium College
Large Language Models (LLMs) have achieved strong performance across many downstream tasks, yet their effectiveness in extremely low-resource machine translation remains limited. Standard adaptation techniques typically rely on large-scale parallel data or extensive fine-tuning, which are infeasible for the long tail of underrepresented languages. In this work, we investigate a more constrained question: in data-scarce settings, to what extent can linguistically similar pivot languages and few-shot demonstrations provide useful guidance for on-the-fly adaptation in LLMs? We study a data-efficient experimental setup that combines linguistically related pivot languages with few-shot in-context examples, without any parameter updates, and evaluate translation behavior under controlled conditions. Our analysis shows that while pivot-based prompting can yield improvements in certain configurations, particularly in settings where the target language is less well represented in the model's vocabulary, the gains are often modest and sensitive to few shot example construction. For closely related or better represented varieties, we observe diminishing or inconsistent gains. Our findings provide empirical guidance on how and when inference-time prompting and pivot-based examples can be used as a lightweight alternative to fine-tuning in low-resource translation settings.
While large language models (LLMs) have achieved remarkable success in general language tasks, their performance on Chouxiang Language, a representative subcultural language in the Chinese internet context, remains largely unexplored. In this paper, we introduce Mouse, a specialized benchmark designed to evaluate the capabilities of LLMs on NLP tasks involving Chouxiang Language across six tasks. Experimental results show that, current state-of-the-art (SOTA) LLMs exhibit clear limitations on multiple tasks, while performing well on tasks that involve contextual semantic understanding. In addition, we further discuss the reasons behind the generally low performance of SOTA LLMs on Chouxiang Language, examine whether the LLM-as-a-judge approach adopted for translation tasks aligns with human judgments and values, and analyze the key factors that influence Chouxiang translation. Our study aims to promote further research in the NLP community on multicultural integration and the dynamics of evolving internet languages. Our code and data are publicly available.
Dianqing Lin, Tian Lan, Jiali Zhu +7
College of Computer Science, Inner Mongolia University, China