Take three frontier mixture-of-experts models (Alibaba, OpenAI, NVIDIA; 3.6-4.0B active parameters each) and fine-tune them to reason in a low-resource language. On accuracy benchmarks almost nothing happens, and the benchmark itself is noise at this scale: changing only the random seed moves the score by 7.7 points, more than every data and recipe effect we measured. That null is our first result. The real changes live where accuracy cannot see. Base models never think in Greek: 0 of 1,000 reasoning traces, even when the question is Greek, so the model answers correctly while reasoning in a form its user cannot read, audit, or correct. After supervised fine-tuning (SFT), every released checkpoint reasons in the language of the question on ~98% of items, one family at 3x fewer tokens, with judged grammaticality improving on all four models and general ability within a few points of each base: nothing was forgotten, and fluency was gained. We propose six behavioural dimensions that make such changes measurable, each gated to reject any metric that correlates with output length, and we report how our own instruments lied: six failures, each caught by a control. What SFT cannot do is fix its own defects: a quarter of answers skip the requested format, answers leak into the reasoning channel, and an explicit "think in English" is obeyed under half the time. Reinforcement learning with verifiable rewards, pre-registered before training, fixes the first two outright (fallback 24% to 2.5%, leak 3.5% to 0.0%, both against a flat random-reward control) and moves the third (+9.1pp), while the Greek reasoning habit survives an accuracy-only gradient untouched. We release five checkpoints. The instruments, the controls and the pre-registration travel to any low-resource language; Greek is the case that let us measure them.
Figures & tables
lab
base model
total
active
routing
fine-tuned release
Alibaba
Qwen3.6-35B-A3B
36.0B
3.97B
8 of 256
Sophea-Qwen3.6-v1
OpenAI
Gpt-OSS-20B
20.9B
3.60B
4 of 32
Sophea-OSS-v1
NVIDIA
NemotronH-30B-A3B
31.6B
3.58B
6 of 128
Sophea-Nemo-3-Nano-v1
NVIDIA
Nemotron-3.5-Lightning-30B-A3B
31.6B
3.58B
6 of 128
Sophea-Nemo-3.5-Lightning-v1
Table 1: Total and routed-expert parameters counted from the released tensors; active-per-token is non-routed +k/E of routed. Gpt-OSS ships MXFP4-packed experts, so its figures are vendor-reported. NemotronH (Nano) and Nemotron-3.5-Lightning are Mamba/MoE hybrids (Mamba, MoE and attention layers of 52; the state-space line of Gu and Dao [14] , Dao and Gu [8] and the vendor’s Nemotron-H report [ 21 ] ); the other two are MoE transformers. The right-most column names the fine-tuned Sophea reasoning release each base produces; all four SFT releases are language-matched, and a fifth release, Sophea-Qwen3.6-v1.1, is the RLVR refinement of the Qwen row (§ 15.1 ). Throughout, base refers to the vendor checkpoint and fine-tuned to its Sophea reasoning model. Lightning is the next-generation release in the Nemotron line; Nano and Lightning share the 31.6B / 6-of-128 routing profile and are reported together as the Nemotron family (Nano is the same-generation member of that family).
domain (reasoning half)
rows
share
mathematics
22,608
38.2%
science
11,064
18.7%
world knowledge
5,884
10.0%
deductive logic
5,151
8.7%
medical
4,898
8.3%
commonsense
3,904
6.6%
Table 2: The reasoning half is math-heavy and thin on the axes we evaluate hardest: commonsense is 6.6% and logic 8.7% of it.
name
trained on
rows
Base
nothing (the released model)
—
Reasoning
reasoning rows only
59,107
Subset
a subset of the same rows
15,607
Two-Phase
reasoning rows, then + the direct half
59,107 +
One-Phase
one pass over both halves at once
59,107 +
Table 3: Two-Phase continues the same adapter from Reasoning onto the hybrid mix; One-Phase never separates the two halves. The three recipes fail in different ways (§ 7 ).
Figure 1: fallback% (the rate at which a model never emits the requested answer line) by recipe and by language-matched checkpoint. It separates “cannot reason” from “will not answer in the requested form”: every recipe failure in the paper is visible here before it is visible anywhere else. The Qwen recipes are the side experiment of § 7 ; the Gpt-OSS language-matched arm’s pre-repair Greek-lane rate (70%) is the family defect at its largest (its traces are present and on-language; the model simply does not close with the line the scorer is told to read), and the released checkpoint’s format-repair dose cuts it to 26% (§ 11 ), still the highest of the releases. The dashed reference is a base model on the English control ( 3.4% ), showing the failure is not inherent to the benchmark.
Figure 2: Left: one configuration, three seeds, nothing else changed. Right: every accuracy effect we measured over the project, against that range. All of them fit inside it.
seed
mean
logic
fallback
trace Greek
42
76.2
56.2
10%
1.00
43
68.7
35.6
41%
1.00
44
76.4
54.0
3%
1.00
Table 4: sd=4.4 pp, range =7.7 pp. Note the last column.
Figure 3: Every accuracy effect we measured, against the 7.7 -point seed-to-seed band (shaded). Attribute-level effects (data selection and corpus versions) sit inside the band we would have been tempted to narrate. The only effect that survives is the replicated phase-2 comparison of § 7 ( +6.9 pp, 15 arms, permutation p=0.0008 ): it is inside the band geometrically, but it is estimated across 15 independently trained arms rather than read off a single run, which is precisely the difference the floor teaches. The language-matched accuracy cost ( −6.9 pp) is drawn as an open (unfilled) marker on purpose: it is a single-run measurement inside the floor, disclosed rather than interpreted, and not a recipe property. The equal magnitudes of the +6.9 and the −6.9 are a coincidence: different lanes, different arms.
phase-1 rows
selection
mean
62,562
all
69.5
15,607
top-by-structure
69.5
15,607
uniform random
69.0
Table 5: Selection buys −0.5 pp ( 0.31σ ); a 4× smaller pool costs nothing. The 62,562 is the reasoning count of the pre-gate pool the experiment was run against ( 62,562 reasoning, 60,214 direct); the 59,107 quoted everywhere else is the post-gate train half ( 59,107 reasoning, 58,985 direct), so the table’s all row and the paper’s reasoning-half count differ by the 3,455 rows the decontamination/dedup gate and the val reserve removed.
Figure 4: The same two checkpoints on four dimensions, the base and the structure-selected reasoning-only arm ( Subset , the “Reason-16k” of Figure 11 ), on the 1,000 -item probe lane. Only the first is accuracy.
anchored accuracy
fallback %
arm
math
cs
logic
math
cs
logic
↑ higher is better
↓ lower is better
Qwen base (Greek think)
92.9
82.4
42.6
3.3
0.5
0.2
Sophea-Qwen3.6-v1
83.3
75.8
37.3
18.2
22.9
46.3
Gpt-OSS base (Greek think)
90.7
63.2
32.2
2.4
5.2
0.7
before repair (reference)
74.6
53.7
25.2
34.9
83.5
77.4
Table 6: Per-domain results on the Greek think lane; here and in every table, ↑ marks metrics where higher is better and ↓ where lower is better (math = the two math slices, commonsense = HellaSwag+Winogrande, logic = ProofWriter macro-recall). Rows are grouped by family, each release indented under its own base, so every legitimate accuracy delta is a within-block comparison and the cross-base reading (the one an earlier draft of this paper fell for) is structurally discouraged. Every release now carries its own base row: the NemotronH base dump is the same vintage as the Qwen base dump, and the Nemotron-3.5 base lane, absent until this revision, was generated with the identical lane settings and scored with the same scorer as every other row. The same base is also measured on the NLU retention suite (Table 14 , Greek macro 57.5 ), a direct-mode instrument whose numbers are not comparable to this think-lane table. The Nano release also appears in the retention suite (Table 13 ), and its fallback shape is the mirror of Gpt-OSS’s: negligible on math and commonsense ( 1.7 / 1.8% ) but 39.9% on logic, so its 9.8 logic macro-recall is substantially a format floor on the one domain where its answer form breaks. Accuracy and fallback move together: every release keeps the base’s domain ordering, and the Sophea-OSS format-repair dose pulls the commonsense fallback back toward math’s level ( 83.5→25.7 ) while cutting logic’s by a third ( 77.4→50.9 ). The pre-repair row is reference only; the release is the repaired arm. Conditioning on rows that emit the requested answer line (excluding the fallback-scored path) shrinks the apparent deficits sharply: the Qwen release reads 95.8/78.8/39.5 against its base’s 96.0/82.8/42.7 , and the same condition makes Gpt-OSS pre-repair 74.6→92.2 / 53.7→60.2 / 25.2→36.5 and Sophea-OSS-v1 78.6→94.4 / 53.3→58.2 / 26.5→30.0 (math/commonsense/logic), so a substantial share of every raw gap is format compliance, not reasoning.
Greek fid.
EN ctrl
switches
leak
arm
(%) ↑
(%) ↑
/100 sent. ↓
(%) ↓
Qwen base
0.0
100
16.4
15.6
Qwen lang-matched
98.0
100
0.0
3.5
Qwen one-dir. †
98.7
99.3 §
0.0
1.5
Gpt-OSS base
0.0
100
9.7
0.5
Gpt-OSS lang-matched
98.1
100 ‡
0.0
10.3
Table 7: Language-matching across the three families: all four released checkpoints, the two measurable bases, and the one-directional reference arm. Greek fid. : Greek-trace fidelity on the Greek benchmark; both bases sit at exactly 0.0 ( 0/5,156 traces reach ratio 0.9 ), and all four language-matched checkpoints land at the same level ( 97.4 – 98.1% ). EN ctrl : English-control compliance, 100% for every arm. Switches : in-question language switches per 100 sentences on the Greek lane; the bases switch 16.4 (Qwen) and 9.7 (Gpt-OSS) times, every fine-tuned arm (the one-directional reference included) zero. The base fidelity and switch cells are one instrument run: both base dumps rescored with the current scorer on the same day (an earlier draft printed 19.0 for the Qwen base from an earlier dump vintage). Leak : answer-channel leak, the one place families differ ( 10.3% Sophea-OSS-v1, 5.2% Lightning, 3.5% Qwen, 0.0% Nano, the only arm at zero; base leak is measured on the English control, fine-tuned leak on the Greek think lane). † Arms of the earlier one-directional recipe, shown for reference (a different recipe, not releases); the Qwen one-directional row is the recipe-evolution comparison: it locks against explicit language instructions (Table 10 ), not against English questions. § Measured on the 1,900 -item paired probe ( 99.3% English, 0.0% Greek, 0.7% too short to score); this arm was never run on the 1,100 -item control the other EN-ctrl cells use. ‡ Measured on the pre-repair language-matched arm: the repair dose adds 2% trace-less answer-format rows and does not touch trace language, but the released arm’s own English control has not been generatively re-run. The recipe generalises to every family tried; the leak’s size does not.
Figure 5: Median trace length in tokens (each family’s own tokenizer, 5,156 matched items), base against the fine-tuned release. The word-level shortening must repay Greek’s 2.3 – 2.5× token fertility before it becomes a serving saving: Qwen’s does, NemotronH’s breaks even, Gpt-OSS’s does not.
Figure 6: Overrun = trace ≥3× the median trace for that item across arms. Easy = at least 80% of arms answer correctly. Cells pool the arms of each column; hard-item think-mode rates are compressed by generation-cap truncation (§ 5 ), so the easy row carries the section’s claims.
recipe
mean
empty
fallback
One-Phase
66.1
23.6%
12%
Two-Phase
64.4–69.9
0.0–1.3%
33–40%
reasoning-only
73.6
0.0%
2–12%
Table 8: single-phase = one pass over both halves; two-phase = reasoning-only then hybrid; reasoning-only = the reasoning half alone. Empty-trace rate is measured in think mode, fallback% in direct mode; means are over all arms of each recipe. Fallback ranges are over the individual arms of each recipe (2–12% across the three reasoning-only arms; 33–40% across the two-phase arms); the three seed replicates of § 4 widen it further (3–41% across seeds 42–44, all three of which are among the six phase-1-only arms above), so the fuller picture is 2–41% when seed noise is admitted.
Figure 7: Each point is an independently trained arm. Corpus version varies within both groups.
Figure 8: The four withdrawn findings of this section, drawn: the effect as first believed (open circle) against the same quantity after its control (filled). Three flip sign; the un-anchored scorer artifact ( +29.8 pp, 18σ , one probe) collapses entirely: its endpoint is drawn at zero because the prose reports the artifact removed by anchored scoring, not as a measured residual. Rows are different instruments and lanes, so magnitudes compare within a row, not across rows; the other pairs’ numbers appear in this section’s prose. The section’s fifth failure (§ 8.5 ) is a scorer defect rather than an effect-size withdrawal, so it has no before/after pair to draw.
arm
human
machine
Δ
Base
95.6
90.7
−4.9 ( −3.14σ )
Subset
94.0
95.1
+1.1
Reasoning
93.6
95.3
+1.7
Two-Phase
87.6
90.4
+2.8
Table 9: The base is ∼ 5 pp worse on our translations; every fine-tune is slightly better.
Figure 9: Trace-language composition per lane, Qwen line: the share of traces that are Greek (character ratio ≥0.9 ), mixed, or English ( ≤0.1 ), on the same instrument as Table 7 ’s fidelity column. Greek-question rows are the full 5,156 -item benchmark; English-question rows are the 1,100 -item control (base) and the 1,900 -item probe (matched). Two honesty notes drawn rather than hidden: the base’s Greek-question traces are not mostly pure English: only 4.2% are; the mass is mixed script with English scaffolding (median ratio 0.33 , § 5 ); and the one-directional arm’s Greek-question lane decomposes as 98.7 Greek / 1.3 mixed / 0.0 English on the same instrument ( n=5,156 , uninstructed); its English-question lane, generated later on the same 1,900 -item probe, measures 99.3% English (Table 7 ). Uninstructed defaults only; the instructed override is Table 10 .
model, instruction
English traces
Greek traces
base, “reason in English”
100.0%
0.0%
base, “reason in Greek”
72.0%
0.0%
fine-tune, “reason in English”
0.0%
98.7%
fine-tune, “reason in Greek”
0.0%
98.7%
matched ft, “reason in English”
44.8%
24.8%
matched ft, “reason in Greek”
0.0%
98.5%
Table 10: The one-directional fine-tune’s traces are identical under both instructions, and it does not matter how the instruction is phrased. Rows classified as neither/mixed scripts ( 28% of the base under the Greek instruction) are omitted, so rows need not sum to 100 . The matched ft rows, measured after the language-matched re-training of § 10 , show the override partially returning; its mixed share under the English instruction is 30.4% .
released checkpoint
“in English” ↑
“in Greek” ↑
(Greek Q)
(English Q)
Sophea-Qwen3.6-v1
44.8%
83.7%
Sophea-Qwen3.6-v1.1 (RLVR)
53.9%
85.7%
Sophea-OSS-v1
62.5%
93.3%
Sophea-Nemo-3.5-Lightning-v1
0.0%
92.7%
Sophea-Nemo-3-Nano-v1
0.0%
87.5%
Table 11: Instructed-override compliance on the five released checkpoints: the share of traces in the instructed language when the instruction contradicts the question ( n=1,000 Greek-question and n=1,100 English-question items; same scorer as Table 10 ). Every release obeys “switch to Greek”; only the Qwen line and Gpt-OSS obey “switch to English”, and the RLVR refinement widens Qwen’s opening ( 44.8→53.9% , § 15.1 ). Two instrument notes, both checked: the Nemotron dumps do not separate trace from answer, so the mandatory Greek answer line falls inside the scored text: re-scoring with that line stripped leaves both Nemotron arms at 0.0% (the Qwen control moves 44.8→49.5% , the expected direction); and the Nemotron arms return 5 – 9% of rows too short to score against Qwen’s 0% , consistent with the low reasoning ceiling reported in § 13 .
arm ( n=1,100 )
accuracy
reasons in EN
Qwen base / ft
94.0 / 96.2
100% / 100%
Gpt-OSS base / ft
95.5 / 96.1
100% / 100%
NemotronH base / ft
92.7 / 93.7
100% / 100%
Qwen language-matched
94.4
100%
Gpt-OSS language-matched
91.9
100%
Table 12: English questions get English traces from every arm measured. The fine-tunes in the first three rows are the dual-mode / reasoning-recipe checkpoints (§ 7 ), kept deliberately: they are the recipe-class evidence that one-directional training does not lock English questions . The last two rows are the language-matched checkpoints: the released Qwen checkpoint, re-run on this exact control ( 94.4 , +0.4 over its base, 100% English traces, consistent with its 1,900 -item probe above), and the Gpt-OSS language-matched arm evaluated after merging (the pre-repair arm; the released, format-repaired checkpoint’s control has not been generatively re-run, Table 7 ). Together the rows confirm the language-matching result is not Qwen-specific; none of this retests the hard one-directional lock of Table 10 , which remains a property of training in a single output direction.
Greek macro
base
language-matched
+ format repair
Gpt-OSS
61.2
53.9
58.0
macro deltas vs base (points):
Sophea-Qwen3.6-v1
−0.01 Greek / +0.08 English
Sophea-OSS-v1
−3.2 Greek / +1.0 English †
Sophea-Nemo-3-Nano-v1
+3.8 Greek / −2.0 English
Sophea-Nemo-3.5-Lightning-v1
+1.7 Greek / −1.1 English
Table 13: Forgetting on the Titan-1 suite. Top: Gpt-OSS on the nine Greek non-reasoning benchmarks, base against the language-matched fine-tune, before and after the format-repair dose: the language-matched fine-tune initially costs −7.3 points, and the repair dose recovers 4.1 of them, so the residual is −3.2 not zero. Below: the macro deltas this paper reports per family, fine-tuned arm vs its own base; the format-repaired Gpt-OSS is the fine-tuned arm we stand behind . Qwen is flat in both languages; Nano gains Greek (+3.8) against the family’s low 0.49 Nano base (mostly catch-up, not forgetting); Lightning is read against its own base, measured separately (Greek macro 57.5, well above Nano’s 48.6), and gains +1.7; an earlier +10.6 reading against the shared Nano base is superseded by that measurement. Per benchmark against binomial standard error; a macro delta is a direction, not a test. † On English the pre-repair language-matched Gpt-OSS lost −7.7 points ( 75.7→68.0 ); re-measured on the repaired release, the English macro is 76.7 , +1.0 above base: the repair dose recovered both lanes, so the pre-repair loss was format behaviour throughout. Macro deltas are computed on unrounded macros; Table 14 prints one decimal, so its Qwen columns round to 75.7→75.6 Greek / 85.1→85.2 English.
Qwen
Gpt-OSS
Nemotron
benchmark
base
ft
base
ft
Nano base
Nano ft
Ltng. base
Ltng. ft
Greek (nine benchmarks, the Greek macro of Table 13 ):
ARC-Challenge-el
91.8
92.0
77.1
68.6
54.2
59.0
68.7
72.8
ARC-Easy-el
96.9
97.1
85.4
79.9
62.3
70.1
75.8
81.4
Belebele-el
92.7
93.7
86.0
81.3
67.0
78.0
81.3
83.3
GreekMMLU
83.8
83.0
66.3
61.3
58.8
53.3
64.2
62.2
Table 14: Every benchmark behind the macros of Table 13 (Titan-1 suite, non-reasoning mode; accuracy %). Qwen ft is Sophea-Qwen3.6-v1 (language-matched); Gpt-OSS ft is Sophea-OSS-v1, the format-repaired release (its pre-repair intermediate is in Table 13 ); Nano ft is Sophea-Nemo-3-Nano-v1, the language-matched Nano release; Lightning ft is Sophea-Nemo-3.5-Lightning-v1 (language-matched); the two Nemotron generations share the same 31.6B / 6-of-128 routing profile (Table 1 ) and each is read against its own base column. (The earlier one-directional recipe arm is reported only in § 13 ’s recipe history, not as a release.) The per-benchmark view localises the Gpt-OSS story: before the repair the loss concentrated on the instruction-format-heavy reading benchmarks (Belebele-el −17.7 , ARC-Challenge −13.1 el / −19.1 en, MMLU-en −16.0 ) while HellaSwag and Winogrande gained in both languages, and the repair dose recovers most where the loss was format (Belebele-el regains +13.0 of the 17.7 ). English retention on the released Gpt-OSS checkpoint recovers to +1.0 above its base; Lightning retains English within 1.1 points of its own base.
Figure 10: Every per-benchmark retention delta behind Table 14 : fine-tuned release minus its own base (Lightning against the Lightning base; Sophea-OSS-v1 is the format-repaired release), Titan-1 suite, non-reasoning mode. Per-benchmark binomial sigma applies and a delta is a direction, not a test (Table 13 ); the value is the pattern: the Qwen column is near-white in both languages, the Sophea-OSS-v1 residual concentrates on the knowledge-heavy benchmarks (MMLU-el −10.1 ) while HellaSwag gains in both languages on every family, and MMLU is the largest English loss for all three non-Qwen releases.
Figure 11: Left: accuracy per axis on the 5,156-item benchmark for the Qwen recipe arms (the cross-family releases are Table 6 ), with the seed-to-seed noise floor shaded around the Qwen base. Right: the same arms’ cost in words per correct answer (token-denominated cost is Figure 5 ). Math uses the 250 human-translated items only (§ 8 ); logic is macro-recall (§ 3 ).
Supported by our evidence
basis
Drop the hybrid phase 2; train reasoning-only
+6.9 pp over 15 runs, p=0.0008 (§ 7 ); and the direct mode it sacrifices costs nothing measurable on direct-mode NLU (E3, § 11 )
Generate exploratory traces, not explanations
structure 0.27→0.68 on the same questions
Forbid formulaic openings in the generator
24/24 identical openings without it
Report fallback% beside accuracy
separates reasoning from format compliance
Run the seed control before any ablation
7.7 pp (§ 4 )
Probe trace-language steerability per checkpoint
same recipe, four checkpoints, 62.5 / 44.8 / 0 / 0 (§ 10.1 )
Table 15: Supported and unsupported configuration choices, each with its evidential basis.
Greek
fall-
override
reverse
arm
fid. % ↑
acc ↑
back % ↓
leak % ↓
el → EN % ↑
en → EL % ↑
SFT checkpoint (before)
97.98
73.7
24.1
3.53
44.8
83.7
random reward (control)
98.06
74.0
22.1
3.61
44.1
83.8
correct+format+term.
98.22
77.2
2.5
0.00
48.8
84.7
+ language
98.02
77.2
5.6
0.04
45.9
84.5
+ override
98.27
77.0
2.8
0.02
53.9
85.7
Table 16: The pre-registered RLVR round, scored on the held-out instruments (Greek think lane n=5,156 ; override lanes n=1,000 / 1,100 ), same scorer, same day. The before row is the released checkpoint’s frozen dump rescored with the current scorer; its full same-day regeneration reproduced all 5,156 responses bit-identically (greedy decoding is deterministic here), and the plain-mode lanes to the decimal ( 44.8 / 83.7 ): the before column is not a provenance caveat but a verified constant. The control row is the entire admissibility argument: on every axis the random-reward arm reproduces the baseline, so nothing below can be elicitation. Accuracy is shown for completeness only: +3.3 – 3.5 is inside the ±7.7 seed floor of § 4 and is claimed by nobody.
Recent advances in large language models (LLMs) have demonstrated that reinforcement fine-tuning of pretrained base models can lead to significant gains in reasoning performance at inference time. In this work, we theoretically analyze why reinforcement fine-tuning induces better reasoning ability than purely supervised fine-tuning (SFT) methods. We model chain-of-thought (CoT) reasoning as a pathfinding problem on graphs and compare the popular method of reinforcement learning with verifiable rewards (RLVR) against traditional SFT. We prove that SFT, when trained on golden shortest paths without negative examples, fails to learn how to efficiently backtrack. In contrast, an RLVR-trained model can learn how to efficiently backtrack from dead ends using only outcome reward. This leads to an exponential separation in inference-time compute between the two methods, and demonstrates that RLVR leads the model to learn the location of difficult decisions in a reasoning chain, ultimately allowing for better allocation of inference-time compute. Finally, we show that the reasoning traces of an RLVR model can be distilled to train a base model to backtrack efficiently as well.
Stanley Wei, Juno Kim
1Princeton University · University of California, Berkeley.
Large language models exhibit substantial performance variation across languages, even when solving semantically equivalent tasks. Existing analyses often treat this phenomenon as an observational disparity caused by differences in pretraining data, tokenization, or benchmark coverage. We study a complementary hypothesis: high-resource languages (HRLs) may more reliably elicit latent computations useful for task-specific (i.e. mathematical) reasoning, while lower-resource languages (LRLs) may under-activate those computations despite expressing the same task. To test this hypothesis, we introduce a mechanistic intervention framework for identifying and transferring task-relevant sparse latent features across languages. Using sparse autoencoders over residual-stream activations, we isolate features enriched in successful HRL task-specific reasoning while filtering out source-language and generic-generation features. We then construct steering directions from these features and inject them during LRL inference. The resulting interventions test whether the selected features are functionally involved in the observed reasoning gap: suppressing them should impair source-language reasoning, while activating them should partially recover target-language reasoning beyond random and non-task controls. Our framework reframes some cross-lingual reasoning gaps as failures of mechanism elicitation rather than capability absence, and offers a causally testable route to feature-mediated transfer without translation, fine-tuning, or changing the user-facing language.
Minju Song, Hyeon Hwang, Junhyun Lee +1
Korea University · Hankuk University of Foreign Studies · Noah’s Farm +1
Reasoning Language Models (RLMs) achieve their strongest performance when they reason in English, the language for which reasoning-oriented training data is most abundant. However, reasoning trace is a clue for model interpretability and safety, and useful in practice for both the model users and for model developers. Thus, it is desirable to be able to develop a model that reasons in a language of the user's choice, while still maintaining strong reasoning performance. To this end, we study the feasibility of training a model that reasons in Japanese. We develop a Japanese-reasoning variant of Qwen-3-Swallow-8B, which is a Japanese LLM continually pretrained from Qwen-3-8B, with GRPO and evaluate it across coding, math, and science benchmarks. The study shows that reasoning-language control is feasible by training a Japanese continually pretrained model with GRPO. However, its performance is at best on par with strong English-reasoning baselines on several benchmarks. We also evaluate the trained model on Japanese cultural benchmarks and observe that the model's performance is worse than the baseline models, suggesting that the reasoning in Japanese does not immediately improve performance on culturally relevant tasks for free.