From Unverifiable Domain Text to Cross-Lingual RLVR via Code-Switched Task Synthesis

Cheonbok Park; Jihwan Kim; Shinyoung Joo; Joosung Lee; Hwiyeol Jo; Jeonghoon Kim; Jaegul Choo

cs.CLOct 18, 2026

From Unverifiable Domain Text to Cross-Lingual RLVR via Code-Switched Task Synthesis

Authors: Cheonbok Park, Jihwan Kim, Shinyoung Joo, Joosung Lee, Hwiyeol Jo, Jeonghoon Kim, Jaegul Choo

Organizations: NAVER Cloud · KAIST AI

Abstract

Reinforcement Learning with Verifiable Rewards (RLVR) can improve reasoning in large language models, but specialized non-English domains expose a data mismatch. Raw English domain text is often available, while target-language RLVR tasks with verifiable rewards are scarce. We introduce code-switched task synthesis for cross-lingual RLVR, a data-centric approach that derives target-language and code-switched RLVR task variants from otherwise unverifiable English domain text. Using fill-in-the-middle multiple-choice QA as a seed task format, we synthesize automatically verifiable tasks from domain passages and reshape them into cross-lingual variants. These variants preserve the same evidence-answer relation but change how English and target-language evidence are presented during RLVR. On Korean medical benchmarks and cybersecurity benchmarks across four target languages, code-switched training outperforms matched monolingual and bilingual-mixture baselines on average target-language performance while maintaining comparable English performance. Additional analyses provide evidence of better target-language input understanding and stronger cross-lingual alignment in both model outputs and internal representations. These results suggest that code-switched training helps target-language inputs access domain reasoning learned more strongly in English.

Figures & tables

Figure 1: Data construction pipeline for cross-lingual RLVR from raw English domain corpora. (a) We first extract educational passages from domain raw corpora, identify answer spans, and generate distractors to synthesize English FIM-MCQA seed tasks. We then translate each seed task into the target language and construct code-switched variants: option-side code-switching, which pairs passages and answer options across languages, and passage-level code-switching, which mixes English and target-language evidence within the passage in both directions. (b) The resulting training variants used for controlled RLVR comparisons.

	Qwen2.5-3B-Instruct					Llama-3.2-3B-Instruct
Bench.	RLVR-Base	+En	+Tgt	+BiMono	+Ours	RLVR-Base	+En	+Tgt	+BiMono	+Ours
KorMedMQ-D	35.06	35.34	35.29	35.34	36.67	21.10	27.93	27.47	28.79	29.31
MedMCQA*	34.52	33.96	34.03	33.72	34.86	24.26	27.12	27.36	29.65	29.22
KMMLU-Pro-M	35.45	35.76	36.21	36.06	36.29	22.84	24.77	24.09	25.08	26.36
Avg (Tgt)	35.01	35.02	35.18	35.04	35.94	22.73	26.61	26.31	27.84	28.30
Avg (En)	51.42	52.19	52.97	52.96	52.13	51.65	56.64	56.89	57.21	57.26

Table 1: Performance on medical benchmarks across RLVR data configurations. RLVR-Base denotes RLVR training using only the auxiliary data pool. The remaining variants include the same auxiliary pool and additionally use English data ( +En ), target-language data ( +Tgt ), bilingual monolingual data ( +BiMono ), or our proposed code-switched data ( +Ours ). The asterisk (*) indicates that the benchmark is translated into the respective language. The gray Avg (En) row reports average English performance (details in Appendix C.1 ). Bold indicates the best result within each model, and green shading highlights our proposed method ( +Ours ).

	SmolLM3-3B
	Korean					Swahili
Bench.	RLVR-Base	+En	+Tgt	+BiMono	+Ours	RLVR-Base	+En	+Tgt	+BiMono	+Ours
SecEval*	44.3	50.1	49.8	51.1	57.3	25.8	26.8	28.6	25.9	28.3
CyberMetric*	65.8	70.6	80.2	72.4	80.0	54.0	53.9	54.2	54.5	55.0
CTI-MCQ*	40.6	43.2	45.5	43.8	47.1	41.9	42.1	43.2	43.4	45.0
Avg (Tgt)	50.23	54.63	58.50	55.77	61.47	40.57	40.93	42.00	41.27	42.77

Table 2: Comprehensive performance on cybersecurity benchmarks across RLVR data configurations for four target languages spanning different scripts and resource levels: Korean and Swahili (top), and Hindi and Japanese (bottom). All other notations and formatting follow Table 1 .

Variant	Target Avg	English Avg
RLVR-Base	50.23	64.87
+ En	54.63	67.00
+ Tgt	58.50	67.83
+ BiMono	55.77	66.80
+ Opt	56.30	67.20
+ Pass	59.37	67.07

Table 3: Component ablation on Korean cybersecurity benchmarks (SmolLM3-3B). Values report the macro-average accuracy across benchmarks in the Target Language (Korean) and English.

(English, Korean)	( $\checkmark$ , $\checkmark$ ) $\uparrow$	( $\checkmark$ , $\times$ ) $\downarrow$	( $\times$ , $\checkmark$ ) $\downarrow$
Base	15.1	28.2	13.0
En-only	28.1	30.0	7.3
Bi-Mono	29.2	29.8	7.1
Ours	34.8	27.1	6.7

Table 4: Cross-lingual consistency comparison on cybersecurity benchmarks. This table aggregates the outcomes of English and Korean question pairs across cybersecurity domain benchmarks.

\checkmark

denotes a correct answer, and

\times

denotes an incorrect answer. Bold indicates the most consistent results.

Figure 2: Reasoning performance ratio vs. in-domain Korean-to-English translation quality for (a) CyberMetric (SmolLM3-3B) and (b) KMMLU-Pro-Medical (Llama-3.2-3B-Instruct). Dashed lines show linear trends;

r

denotes Pearson correlation.

Figure 3: Layer-wise English–Korean representation alignment on CyberMetric, measured at the opening <think> marker. Our method shows stronger middle-to-upper layer alignment than the non-code-switched baselines. The gray area marks middle layers 12–24.

Medical
	vs. En-only		vs. Tgt-only
Win Rate(%)	Ours	En-only	Ours	Tgt-only
Qwen2.5	48.3	40.0	47.5	42.5
Llama3.2	47.0	38.8	43.3	42.0
Cyber
SmolLM3	36.3	31.7	58.7	21.7

Table 5: Pairwise judge win rates to measure cross-lingual reasoning consistency using GPT-5.5. The target language is Korean. The reported results are aggregated across benchmarks within each domain (evaluating 100 samples per benchmark). Our method demonstrates consistent improvement across all models and baselines.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	$n$	Source
Cybersecurity	207,048	Primus-Seed, Primus-FineWeb
Medical	158,714	Biomed-Enriched
Total	365,762

Appendix

Table 6: Synthesized FIM-MCQA data pool per domain. Note that for each domain-target language pair, an identical number of samples is constructed across all training variants, matching the exact counts reported in the table. The subsample used for training is described in Appendix B.1 .

Domain	Model	Pool	Filtered
Medical	Qwen2.5-3B-Instruct	158,714	50,476
Medical	Llama-3.2-3B-Instruct	158,714	44,294
Cybersecurity	SmolLM3-3B	207,048	43,705

Appendix

Table 7: Number of Effective Samples after the difficulty filtering stage.

Benchmark	Korean (Acc/Flu/Term)			COMET
Benchmark	GPT-5.5	Human	$r$	KO	SW
SecEval	4.94/4.82/4.70	4.25/4.18/4.12	0.81	0.852	0.803
CyberMetric	4.92/4.75/4.67	4.21/4.14/4.08	0.83	0.864	0.810
CTI-MCQ	4.88/4.74/4.66	4.16/4.10/4.02	0.79	0.851	0.808

Appendix

Table 8: Translation quality evaluation across cybersecurity benchmarks. GPT-5.5 and human scores represent assessments on a 20-instance subsample for Korean (evaluated on a 1-5 scale).

r

denotes the Pearson correlation between LLM and human judgments. COMET scores are reported for the full evaluation sets in Korean and Swahili.

Target Language
	Qwen2.5-3B-Instruct						Llama-3.2-3B-Instruct
Bench.	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours
KorMedMQ-D	34.50	35.06	35.34	35.29	35.34	36.67	19.02	21.10	27.93	27.47	28.79	29.31
MedMCQA*	33.24	34.52	33.96	34.03	33.72	34.86	19.85	24.26	27.12	27.36	29.65	29.22
KMMLU-Pro-M	35.20	35.45	35.76	36.21	36.06	36.29	20.45	22.84	24.77	24.09	25.08	26.36
Avg (Tgt)	34.31	35.01	35.02	35.18	35.04	35.94	19.77	22.73	26.61	26.31	27.84	28.30

Appendix

Table 9: Comprehensive performance on medical benchmarks across RLVR data configurations. Base denotes the original instruction-tuned model without RLVR. RLVR-Base denotes RLVR training using only the auxiliary data pool. The remaining variants include the same auxiliary pool and additionally use English data ( +En ), target-language data ( +Tgt ), bilingual monolingual data ( +BiMono ), or our proposed code-switched data ( +Ours ). English rows are shown in gray to indicate source-language evaluation, which is reported as a retention check rather than the primary target-language transfer metric. The asterisk (*) indicates that the benchmark is translated into the respective language. The best performance within each model is highlighted in bold .

	SmolLM3-3B
	Korean						Swahili
Bench.	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours	Base	RLVR-Base	+En	+Tgt	+BiMono	+Ours
Target Language
SecEval*	40.9	44.3	50.1	49.8	51.1	57.3	26.9	25.8	26.8	28.6	25.9	28.3
CyberMetric*	63.6	65.8	70.6	80.2	72.4	80.0	53.0	54.0	53.9	54.2	54.5	55.0
CTI-MCQ*	33.7	40.6	43.2	45.5	43.8	47.1	41.4	41.9	42.1	43.2	43.4	45.0

Appendix

Table 10: Comprehensive performance on cybersecurity benchmarks across RLVR data configurations. Target Language indicates Korean for the left section and Swahili for the right section. Base denotes the original instruction-tuned model without RLVR. RLVR-Base denotes RLVR training using only the auxiliary data pool. The remaining variants include the same auxiliary pool and additionally use English data ( +En ), target-language data ( +Tgt ), bilingual monolingual data ( +BiMono ), or our proposed code-switched data ( +Ours ). English rows are shown in gray to indicate source-language evaluation, which is reported as a retention check rather than the primary target-language transfer metric. The asterisk (*) indicates that the benchmark is translated into the respective language. The best performance within each model is highlighted in bold .

Variant	SecEval	CyberMetric	CTI-MCQ	Avg
RLVR-Base	$43.9_{\pm 1.0}$	$65.5_{\pm 0.7}$	$40.5_{\pm 0.6}$	$50.0_{\pm 0.5}$
+En	$51.5_{\pm 0.6}$	$70.6_{\pm 0.6}$	$43.8_{\pm 0.7}$	$55.3_{\pm 0.5}$
+Tgt	$50.8_{\pm 0.5}$	$\mathbf{80.0}_{\pm 0.4}$	$45.2_{\pm 0.6}$	$58.7_{\pm 0.4}$
+BiMono	$51.4_{\pm 0.4}$	$73.3_{\pm 0.4}$	$43.7_{\pm 0.6}$	$56.1_{\pm 0.5}$
+Ours	$\mathbf{57.5}_{\pm 0.1}$	$\mathbf{80.0}_{\pm 0.4}$	$\mathbf{46.9}_{\pm 0.5}$	$\mathbf{61.5}_{\pm 0.2}$

Appendix

Table 11: Multi-seed results for SmolLM3-3B on Korean cybersecurity benchmarks, evaluated in the target language. Values report mean

\pm

standard deviation over three random seeds. Best results are bolded.

Target Language
	SmolLM3-3B (Korean)
Bench.	RLVR-Base	+En	+Tgt	+BiMono	+Opt	+Pass	+Ours
SecEval*	44.3	50.1	49.8	51.1	52.0	54.7	57.3
CyberMetric*	65.8	70.6	80.2	72.4	71.0	77.4	80.0
CTI-MCQ*	40.6	43.2	45.5	43.8	45.9	46.0	47.1
Avg (Tgt)	50.23	54.63	58.50	55.77	56.30	59.37	61.47

Appendix

Table 12: Component ablation on Korean cybersecurity benchmarks with SmolLM3-3B. Opt-only and Pass-only extend Bi-Mono with option-side and passage-level code-switching, respectively, while Ours combines both components. All variants follow the same experimental setup. Asterisks denote target-language versions of the benchmarks.

SecEval
(English, Korean)	( $\checkmark$ , $\checkmark$ ) $\uparrow$	( $\checkmark$ , $\times$ ) $\downarrow$	( $\times$ , $\checkmark$ ) $\downarrow$
Base	8.3	39.2	5.4
En-only	15.3	41.8	3.4
Bi-Mono	16.3	41.1	3.7
Ours	23.8	38.8	2.5
CyberMetric

Appendix

Table 13: Detailed cross-lingual consistency comparison for each benchmark. Bold indicates the highest consistency.

Medical
		vs. En-only			vs. Tgt-only
	Benchmark	Ours%	En-only%	tie%	Ours%	Tgt-only%	tie%
Qwen2.5-3B	MedQA	47.0	36.0	1.0	48.0	41.0	0.0
	MedMCQA	50.0	40.0	3.0	46.0	47.0	1.0
	KMMLU-Pro-Medical	50.0	37.0	0.0	54.0	36.0	0.0
	KorMedMCQA-Doctor	46.0	47.0	0.0	42.0	46.0	2.0

Appendix

Table 14: Detailed pairwise judge win rate results using GPT-5.5. The target language is Korean. 100 samples are evaluated per benchmark. Bold indicates a higher win rate.

Panel of Figure 4 — Figure 4: Qualitative example — Medical (Part 1/2). Same source passage shown across 3 variants. [MASK] marks the masked answer span; the ground-truth option ( $\checkmark$ ) is what fills it.

Panel of Figure 5 — Figure 5: Qualitative example — Medical (Part 2/2). Continuation of the 3 variants.

Panel of Figure 6 — Figure 6: Qualitative example — Cybersecurity (Part 1/2). Same source passage shown across 4 variants. [MASK] marks the masked answer span; the ground-truth option ( $\checkmark$ ) is what fills it.

Panel of Figure 7 — Figure 7: Qualitative example — Cybersecurity (Part 2/2). Continuation of the 4 variants.

Panel of Figure 8 — Figure 8: The prompt template used in Stage 1 to simultaneously extract educational passages and identify mask spans for the Cybersecurity domain.

Panel of Figure 9 — Figure 9: The prompt template used in Stage 1 to simultaneously extract educational passages and identify mask spans for the Medical domain.

Panel of Figure 10 — Figure 10: The prompt template used in Stage 3 to generate plausible, length-matched distractors for the masked spans in the Cybersecurity domain.

Panel of Figure 11 — Figure 11: The prompt template used in Stage 3 to generate plausible, length-matched distractors for the masked spans in the Medical domain.

Panel of Figure 12 — Figure 12: The prompt template used for context-aware translation into the target language. The prompt enforces strict constraints to preserve code snippets, system identifiers, and [MASK] markers while preventing unnatural mid-word language mixing.

Panel of Figure 13 — Figure 13: The prompt template used to translate the multiple-choice options into the target language. Similar to the passage translation, it leverages the source document as context to ensure terminology consistency and requires a strict JSON array output.

Panel of Figure 14 — Figure 14: The prompt template used to translate evaluation benchmark multiple-choice questions and options into the target language.

Panel of Figure 15 — Figure 15: The prompt template used for translating source language text into English.

Panel of Figure 16 — Figure 16: The prompt template used for generating span-level code-switched passages.

Panel of Figure 17 — Figure 17: The prompt template for GEMBA-DA used to measure translation quality. The variables {src_full} and {tgt_full} represent the full names of the source and target languages, while {src} and {tgt} are substituted with the original text and the model’s translation, respectively.

Panel of Figure 18 — Figure 18: The prompt template used for the reasoning trace alignment experiment.

Panel of Figure 19 — Figure 19: The prompt template used for both the LLM-as-a-judge (GPT-5.5) and the human evaluators to assess translation quality.

Cross-Lingual Transfer

Cross-lingual transfer enables models to apply knowledge or capabilities learned in one language to tasks in another.

See all 256 papers

Feb 13, 2026

To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models

Reinforcement Learning with Verifiable Rewards (RLVR) plays a key role in stimulating the explicit reasoning capability of Large Language Models (LLMs). We can achieve expert-level performance in some specific domains via RLVR, such as coding or math. When a general multi-domain expert-level model is required, we need to carefully consider the collaboration of RLVR across different domains. The current state-of-the-art models mainly employ two different training paradigms for multi-domain RLVR: mixed multi-task RLVR and separate RLVR followed by model merging. However, most of the works did not provide a detailed comparison and analysis about these paradigms. To this end, we choose multiple commonly used high-level tasks (e.g., math, coding, science, instruction following, and agent) as our target domains and design extensive qualitative and quantitative experiments using open-source datasets. We find the RLVR across domains exhibits small mutual interferences, and reasoning-intensive domains have mutually synergistic effects. Furthermore, we analyze the internal mechanisms from the perspectives of information constraints, model prediction behavior and self-verification. Our homepage is at https://github.com/Mosi-AI/M2RL.

Haoqing Wang, Xiang Long, Ziheng Li +3

Samsung Research, Beijing, China · Peking University

RL for Language Models Gradient Interference Reinforcement Learning with Verifiable Rewards Multi-Task RL Model Merging

Ask PDF

Apr 14, 2026

ReasonXL: A Large-Scale Corpus for Language-Specific Reasoning in LLMs

Despite advances in multilingual capabilities, most large language models (LLMs) remain English-centric in their training and, crucially, in the reasoning traces they produce: even for non-English problems, they predominantly reason in English, leaving non-English users with intermediate reasoning they cannot follow. We address this disparity with two contributions. (a) We introduce

\texttt{ReasonXL}

, the first large-scale parallel corpus of cross-domain reasoning traces in five European languages (English, German, French, Italian, and Spanish): over 2.5M aligned samples (11B tokens) per language, each comprising prompt, reasoning trace, and final output, enabling direct supervision of language-specific reasoning. (b) Using

\texttt{ReasonXL}

, we demonstrate that LLMs can be adapted to reason entirely in a target language through a simple two-stage pipeline of supervised fine-tuning (SFT) and reinforcement learning with verifiable rewards (RLVR), matching or exceeding the base models' English-reasoning performance in two of four languages and recovering most of it in the remaining two.

Daniil Gurgurov, Tom Röhr, Sebastian von Rohrscheidt +3

German Research Center for Artificial Intelligence (DFKI), Saarland University · Berliner Hochschule für Technik

Cross-Lingual Reasoning in Language Models Multilingual Language Models

Ask PDF

Aug 13, 2026

GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings

Reinforcement Learning with Verifiable Rewards (RLVR), often optimized with Group Relative Policy Optimization (GRPO), has become a central recipe for improving the reasoning capabilities of pretrained language models but current studies remain heavily English-centric. We conduct a large-scale empirical study of multilingual and non-English GRPO across a wide range of base models, training languages, and different reasoning language rewards. We find that training to reason in the native language often leaves only a small gap to training for English reasoning. We further observe strong crosslingual transfer: training in one language often improves performance in many others. However, specific trends are highly model- and language-dependent. In some cases, training in a particular language induces severe regressions on out-of-domain capabilities in other languages. Our analysis shows that RLVR beyond English can provide broad crosslingual gains, but also requires broad evaluation to detect language-specific regressions.

Konstantin Dobler, Federico Scozzafava, Jonathan Janke +2

Apple · 1Hasso Plattner Institute & ELLIS Unit Potsdam (work done during Apple internship)

Cross-Lingual Reasoning in Language Models RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Group Relative Policy Optimization Cross-Lingual Transfer Multilingual Language Model Evaluation

Ask PDF

Abstract

Cross-Lingual Transfer

Explore similar work

To Mix or To Merge: Toward Multi-Domain Reinforcement Learning for Large Language Models

ReasonXL: A Large-Scale Corpus for Language-Specific Reasoning in LLMs

GRPO Beyond English: A Large-Scale Study of GRPO in Non-English and Multilingual Settings