From Unverifiable Domain Text to Cross-Lingual RLVR via Code-Switched Task Synthesis

Cheonbok Park; Jihwan Kim; Shinyoung Joo; Joosung Lee; Hwiyeol Jo; Jeonghoon Kim; Jaegul Choo

cs.CLOct 18, 2026

From Unverifiable Domain Text to Cross-Lingual RLVR via Code-Switched Task Synthesis

Authors: Cheonbok Park, Jihwan Kim, Shinyoung Joo, Joosung Lee, Hwiyeol Jo, Jeonghoon Kim, Jaegul Choo

Organizations: NAVER Cloud · KAIST AI

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) can improve reasoning in large language models, but specialized non-English domains expose a data mismatch. Raw English domain text is often available, while target-language RLVR tasks with verifiable rewards are scarce. We introduce code-switched task synthesis for cross-lingual RLVR, a data-centric approach that derives target-language and code-switched RLVR task variants from otherwise unverifiable English domain text. Using fill-in-the-middle multiple-choice QA as a seed task format, we synthesize automatically verifiable tasks from domain passages and reshape them into cross-lingual variants. These variants preserve the same evidence-answer relation but change how English and target-language evidence are presented during RLVR. On Korean medical benchmarks and cybersecurity benchmarks across four target languages, code-switched training outperforms matched monolingual and bilingual-mixture baselines on average target-language performance while maintaining comparable English performance. Additional analyses provide evidence of better target-language input understanding and stronger cross-lingual alignment in both model outputs and internal representations. These results suggest that code-switched training helps target-language inputs access domain reasoning learned more strongly in English.

Figures & tables

Figure 1: Data construction pipeline for cross-lingual RLVR from raw English domain corpora. (a) We first extract educational passages from domain raw corpora, identify answer spans, and generate distractors to synthesize English FIM-MCQA seed tasks. We then translate each seed task into the target language and construct code-switched variants: option-side code-switching, which pairs passages and answer options across languages, and passage-level code-switching, which mixes English and target-language evidence within the passage in both directions. (b) The resulting training variants used for controlled RLVR comparisons.

	Qwen2.5-3B-Instruct					Llama-3.2-3B-Instruct
Bench.	RLVR-Base	+En	+Tgt	+BiMono	+Ours	RLVR-Base	+En	+Tgt	+BiMono	+Ours
KorMedMQ-D	35.06	35.34	35.29	35.34	36.67	21.10	27.93	27.47	28.79	29.31
MedMCQA*	34.52	33.96	34.03	33.72	34.86	24.26	27.12	27.36	29.65	29.22
KMMLU-Pro-M	35.45	35.76	36.21	36.06	36.29	22.84	24.77	24.09	25.08	26.36
Avg (Tgt)	35.01	35.02	35.18	35.04	35.94	22.73	26.61	26.31	27.84	28.30
Avg (En)	51.42	52.19	52.97	52.96	52.13	51.65	56.64	56.89	57.21	57.26

Table 1: Performance on medical benchmarks across RLVR data configurations. RLVR-Base denotes RLVR training using only the auxiliary data pool. The remaining variants include the same auxiliary pool and additionally use English data ( +En ), target-language data ( +Tgt ), bilingual monolingual data ( +BiMono ), or our proposed code-switched data ( +Ours ). The asterisk (*) indicates that the benchmark is translated into the respective language. The gray Avg (En) row reports average English performance (details in Appendix C.1 ). Bold indicates the best result within each model, and green shading highlights our proposed method ( +Ours ).

	SmolLM3-3B
	Korean					Swahili
Bench.	RLVR-Base	+En	+Tgt	+BiMono	+Ours	RLVR-Base	+En	+Tgt	+BiMono	+Ours
SecEval*	44.3	50.1	49.8	51.1	57.3	25.8	26.8	28.6	25.9	28.3
CyberMetric*	65.8	70.6	80.2	72.4	80.0	54.0	53.9	54.2	54.5	55.0
CTI-MCQ*	40.6	43.2	45.5	43.8	47.1	41.9	42.1	43.2	43.4	45.0
Avg (Tgt)	50.23	54.63	58.50	55.77	61.47	40.57	40.93	42.00	41.27	42.77

Table 2: Comprehensive performance on cybersecurity benchmarks across RLVR data configurations for four target languages spanning different scripts and resource levels: Korean and Swahili (top), and Hindi and Japanese (bottom). All other notations and formatting follow Table 1 .

Variant	Target Avg	English Avg
RLVR-Base	50.23	64.87
+ En	54.63	67.00
+ Tgt	58.50	67.83
+ BiMono	55.77	66.80
+ Opt	56.30	67.20
+ Pass	59.37	67.07

Table 3: Component ablation on Korean cybersecurity benchmarks (SmolLM3-3B). Values report the macro-average accuracy across benchmarks in the Target Language (Korean) and English.

(English, Korean)	( $\checkmark$ , $\checkmark$ ) $\uparrow$	( $\checkmark$ , $\times$ ) $\downarrow$	( $\times$ , $\checkmark$ ) $\downarrow$
Base	15.1	28.2	13.0
En-only	28.1	30.0	7.3
Bi-Mono	29.2	29.8	7.1
Ours	34.8	27.1	6.7

Table 4: Cross-lingual consistency comparison on cybersecurity benchmarks. This table aggregates the outcomes of English and Korean question pairs across cybersecurity domain benchmarks.

\checkmark

denotes a correct answer, and

\times

denotes an incorrect answer. Bold indicates the most consistent results.

Figure 2: Reasoning performance ratio vs. in-domain Korean-to-English translation quality for (a) CyberMetric (SmolLM3-3B) and (b) KMMLU-Pro-Medical (Llama-3.2-3B-Instruct). Dashed lines show linear trends;

r

denotes Pearson correlation.

Figure 3: Layer-wise English–Korean representation alignment on CyberMetric, measured at the opening <think> marker. Our method shows stronger middle-to-upper layer alignment than the non-code-switched baselines. The gray area marks middle layers 12–24.

Medical
	vs. En-only		vs. Tgt-only
Win Rate(%)	Ours	En-only	Ours	Tgt-only
Qwen2.5	48.3	40.0	47.5	42.5
Llama3.2	47.0	38.8	43.3	42.0
Cyber
SmolLM3	36.3	31.7	58.7	21.7

Table 5: Pairwise judge win rates to measure cross-lingual reasoning consistency using GPT-5.5. The target language is Korean. The reported results are aggregated across benchmarks within each domain (evaluating 100 samples per benchmark). Our method demonstrates consistent improvement across all models and baselines.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	$n$	Source
Cybersecurity	207,048	Primus-Seed, Primus-FineWeb
Medical	158,714	Biomed-Enriched
Total	365,762

Appendix

Table 6: Synthesized FIM-MCQA data pool per domain. Note that for each domain-target language pair, an identical number of samples is constructed across all training variants, matching the exact counts reported in the table. The subsample used for training is described in Appendix B.1 .

Domain	Model	Pool	Filtered
Medical	Qwen2.5-3B-Instruct	158,714	50,476
Medical	Llama-3.2-3B-Instruct	158,714	44,294
Cybersecurity	SmolLM3-3B	207,048	43,705

Appendix

Table 7: Number of Effective Samples after the difficulty filtering stage.

Benchmark	Korean (Acc/Flu/Term)			COMET
Benchmark	GPT-5.5	Human	$r$	KO	SW
SecEval	4.94/4.82/4.70	4.25/4.18/4.12	0.81	0.852	0.803
CyberMetric	4.92/4.75/4.67	4.21/4.14/4.08	0.83	0.864	0.810
CTI-MCQ	4.88/4.74/4.66	4.16/4.10/4.02	0.79	0.851	0.808

Appendix

Table 8: Translation quality evaluation across cybersecurity benchmarks. GPT-5.5 and human scores represent assessments on a 20-instance subsample for Korean (evaluated on a 1-5 scale).

r

denotes the Pearson correlation between LLM and human judgments. COMET scores are reported for the full evaluation sets in Korean and Swahili.

Target Language
	Qwen2.5-3B-Instruct						Llama-3.2-3B-Instruct
Bench.	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours
KorMedMQ-D	34.50	35.06	35.34	35.29	35.34	36.67	19.02	21.10	27.93	27.47	28.79	29.31
MedMCQA*	33.24	34.52	33.96	34.03	33.72	34.86	19.85	24.26	27.12	27.36	29.65	29.22
KMMLU-Pro-M	35.20	35.45	35.76	36.21	36.06	36.29	20.45	22.84	24.77	24.09	25.08	26.36
Avg (Tgt)	34.31	35.01	35.02	35.18	35.04	35.94	19.77	22.73	26.61	26.31	27.84	28.30

Appendix

Table 9: Comprehensive performance on medical benchmarks across RLVR data configurations. Base denotes the original instruction-tuned model without RLVR. RLVR-Base denotes RLVR training using only the auxiliary data pool. The remaining variants include the same auxiliary pool and additionally use English data ( +En ), target-language data ( +Tgt ), bilingual monolingual data ( +BiMono ), or our proposed code-switched data ( +Ours ). English rows are shown in gray to indicate source-language evaluation, which is reported as a retention check rather than the primary target-language transfer metric. The asterisk (*) indicates that the benchmark is translated into the respective language. The best performance within each model is highlighted in bold .

	SmolLM3-3B
	Korean						Swahili
Bench.	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours
Target Language
SecEval*	40.9	44.3	50.1	49.8	51.1	57.3	26.9	25.8	26.8	28.6	25.9	28.3
CyberMetric*	63.6	65.8	70.6	80.2	72.4	80.0	53.0	54.0	53.9	54.2	54.5	55.0
CTI-MCQ*	33.7	40.6	43.2	45.5	43.8	47.1	41.4	41.9	42.1	43.2	43.4	45.0

Appendix

Table 10: Comprehensive performance on cybersecurity benchmarks across RLVR data configurations. Target Language indicates Korean for the left section and Swahili for the right section. Base denotes the original instruction-tuned model without RLVR. RLVR-Base denotes RLVR training using only the auxiliary data pool. The remaining variants include the same auxiliary pool and additionally use English data ( +En ), target-language data ( +Tgt ), bilingual monolingual data ( +BiMono ), or our proposed code-switched data ( +Ours ). English rows are shown in gray to indicate source-language evaluation, which is reported as a retention check rather than the primary target-language transfer metric. The asterisk (*) indicates that the benchmark is translated into the respective language. The best performance within each model is highlighted in bold .

Variant	SecEval	CyberMetric	CTI-MCQ	Avg
RLVR-Base	$43.9_{\pm 1.0}$	$65.5_{\pm 0.7}$	$40.5_{\pm 0.6}$	$50.0_{\pm 0.5}$
+En	$51.5_{\pm 0.6}$	$70.6_{\pm 0.6}$	$43.8_{\pm 0.7}$	$55.3_{\pm 0.5}$
+Tgt	$50.8_{\pm 0.5}$	$\mathbf{80.0}_{\pm 0.4}$	$45.2_{\pm 0.6}$	$58.7_{\pm 0.4}$
+BiMono	$51.4_{\pm 0.4}$	$73.3_{\pm 0.4}$	$43.7_{\pm 0.6}$	$56.1_{\pm 0.5}$
+Ours	$\mathbf{57.5}_{\pm 0.1}$	$\mathbf{80.0}_{\pm 0.4}$	$\mathbf{46.9}_{\pm 0.5}$	$\mathbf{61.5}_{\pm 0.2}$

Appendix

Table 11: Multi-seed results for SmolLM3-3B on Korean cybersecurity benchmarks, evaluated in the target language. Values report mean

\pm

standard deviation over three random seeds. Best results are bolded.

Target Language
	SmolLM3-3B (Korean)
Bench.	RLVR-Base	+En	+Tgt	+BiMono	+Opt	+Pass	+Ours
SecEval*	44.3	50.1	49.8	51.1	52.0	54.7	57.3
CyberMetric*	65.8	70.6	80.2	72.4	71.0	77.4	80.0
CTI-MCQ*	40.6	43.2	45.5	43.8	45.9	46.0	47.1
Avg (Tgt)	50.23	54.63	58.50	55.77	56.30	59.37	61.47

Appendix

Table 12: Component ablation on Korean cybersecurity benchmarks with SmolLM3-3B. Opt-only and Pass-only extend Bi-Mono with option-side and passage-level code-switching, respectively, while Ours combines both components. All variants follow the same experimental setup. Asterisks denote target-language versions of the benchmarks.

SecEval
(English, Korean)	( $\checkmark$ , $\checkmark$ ) $\uparrow$	( $\checkmark$ , $\times$ ) $\downarrow$	( $\times$ , $\checkmark$ ) $\downarrow$
Base	8.3	39.2	5.4
En-only	15.3	41.8	3.4
Bi-Mono	16.3	41.1	3.7
Ours	23.8	38.8	2.5
CyberMetric

Appendix

Table 13: Detailed cross-lingual consistency comparison for each benchmark. Bold indicates the highest consistency.

Medical
		vs. En-only			vs. Tgt-only
	Benchmark	Ours%	En-only%	tie%	Ours%	Tgt-only%	tie%
Qwen2.5-3B	MedQA	47.0	36.0	1.0	48.0	41.0	0.0
	MedMCQA	50.0	40.0	3.0	46.0	47.0	1.0
	KMMLU-Pro-Medical	50.0	37.0	0.0	54.0	36.0	0.0
	KorMedMCQA-Doctor	46.0	47.0	0.0	42.0	46.0	2.0

Appendix

Table 14: Detailed pairwise judge win rate results using GPT-5.5. The target language is Korean. 100 samples are evaluated per benchmark. Bold indicates a higher win rate.

Panel of Figure 4 — Figure 4: Qualitative example — Medical (Part 1/2). Same source passage shown across 3 variants. [MASK] marks the masked answer span; the ground-truth option ( $\checkmark$ ) is what fills it.

Panel of Figure 5 — Figure 5: Qualitative example — Medical (Part 2/2). Continuation of the 3 variants.

Panel of Figure 6 — Figure 6: Qualitative example — Cybersecurity (Part 1/2). Same source passage shown across 4 variants. [MASK] marks the masked answer span; the ground-truth option ( $\checkmark$ ) is what fills it.

Panel of Figure 7 — Figure 7: Qualitative example — Cybersecurity (Part 2/2). Continuation of the 4 variants.

Panel of Figure 8 — Figure 8: The prompt template used in Stage 1 to simultaneously extract educational passages and identify mask spans for the Cybersecurity domain.

Panel of Figure 9 — Figure 9: The prompt template used in Stage 1 to simultaneously extract educational passages and identify mask spans for the Medical domain.

Panel of Figure 10 — Figure 10: The prompt template used in Stage 3 to generate plausible, length-matched distractors for the masked spans in the Cybersecurity domain.

Panel of Figure 11 — Figure 11: The prompt template used in Stage 3 to generate plausible, length-matched distractors for the masked spans in the Medical domain.

Panel of Figure 12 — Figure 12: The prompt template used for context-aware translation into the target language. The prompt enforces strict constraints to preserve code snippets, system identifiers, and [MASK] markers while preventing unnatural mid-word language mixing.

Panel of Figure 13 — Figure 13: The prompt template used to translate the multiple-choice options into the target language. Similar to the passage translation, it leverages the source document as context to ensure terminology consistency and requires a strict JSON array output.

Panel of Figure 14 — Figure 14: The prompt template used to translate evaluation benchmark multiple-choice questions and options into the target language.

Panel of Figure 15 — Figure 15: The prompt template used for translating source language text into English.

Panel of Figure 16 — Figure 16: The prompt template used for generating span-level code-switched passages.

Panel of Figure 17 — Figure 17: The prompt template for GEMBA-DA used to measure translation quality. The variables {src_full} and {tgt_full} represent the full names of the source and target languages, while {src} and {tgt} are substituted with the original text and the model’s translation, respectively.

Panel of Figure 18 — Figure 18: The prompt template used for the reasoning trace alignment experiment.

Panel of Figure 19 — Figure 19: The prompt template used for both the LLM-as-a-judge (GPT-5.5) and the human evaluators to assess translation quality.

Cross-Lingual Transfer

Cross-lingual transfer enables models to apply knowledge or capabilities learned in one language to tasks in another.

See all 256 papers

To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models
Feb 13, 2026Haoqing Wang, Xiang Long, Ziheng Li +3RL for Language Models Gradient Interference
Samsung Research, Beijing, China · Peking University
ReasonXL: A Large-Scale Corpus for Language-Specific Reasoning in LLMs
Apr 14, 2026Daniil Gurgurov, Tom Röhr, Sebastian von Rohrscheidt +3Cross-Lingual Reasoning in Language Models Multilingual Language Models
German Research Center for Artificial Intelligence (DFKI), Saarland University · Berliner Hochschule für Technik
GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings
Aug 13, 2026Konstantin Dobler, Federico Scozzafava, Jonathan Janke +2Cross-Lingual Reasoning in Language Models RL for Language Model Reasoning
Apple · 1Hasso Plattner Institute & ELLIS Unit Potsdam (work done during Apple internship)

Feb 13, 2026J/K move · Enter open · S save

To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models

Haoqing Wang, Xiang Long, Ziheng Li, Yilong Xu+2

Samsung Research, Beijing, China · Peking University

RL for Language Models Gradient Interference Reinforcement Learning with Verifiable Rewards Multi-Task RL Model Merging

Open paper AskPDF arXiv

Reinforcement Learning with Verifiable Rewards (RLVR) plays a key role in stimulating the explicit reasoning capability of Large Language Models (LLMs). We can achieve expert-level performance in some specific domains via RLVR, such as coding or math. When a general multi-domain expert-level model is required, we need to carefully consider the collaboration of RLVR across different domains. The current state-of-the-art models mainly employ two different training paradigms for multi-domain RLVR: mixed multi-task RLVR and separate RLVR followed by model merging. However, most of the works did not provide a detailed comparison and analysis about these paradigms. To this end, we choose multiple commonly used high-level tasks (e.g., math, coding, science, instruction following, and agent) as our target domains and design extensive qualitative and quantitative experiments using open-source datasets. We find the RLVR across domains exhibits small mutual interferences, and reasoning-intensive domains have mutually synergistic effects. Furthermore, we analyze the internal mechanisms from the perspectives of information constraints, model prediction behavior and self-verification. Our homepage is at https://github.com/Mosi-AI/M2RL.

Abstract

Cross-Lingual Transfer

Explore similar work

To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models

ReasonXL: A Large-Scale Corpus for Language-Specific Reasoning in LLMs

GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings