Sparse autoencoder (SAE) features are increasingly used to explain and steer language-model behavior, but it remains unclear whether a feature found in one language context plays the same causal role when processing prompts in another language. We study this question using translation-initiation features (Wu et al., 2026). We reproduce the SAE feature discovery method from Wu et al. in Gemma 2 and extend it to multilingual settings that vary prompt language, source language, and target language. We then test whether features that recur across settings affect translation behavior by amplifying or ablating their activations during inference. We also examine whether the method can be applied to Gemma 3. In both models, we observe an identical finding: although we can find more than 20 features that activate frequently across all discovery settings, causal validation shows that nearly all have small or inconsistent effects. In contrast, one feature -- Gemma 2's (L10, 5717) and Gemma 3's (L20, 2456) -- consistently improves COMET scores when amplified and degrades them when ablated across 23 language settings. These results show that feature recurrence can overstate cross-lingual transfer, while identifying a language-agnostic translation-initiation direction in Gemma 2 and Gemma 3.
Figures & tables
Aspect
Wu et al. (2026)
Ours
Models
Gemma 2
Gemma 2, 3
Discovery settings
en2zh_en
en2zh_en , zh2en_zh , zh2ja_zh , zh2ja_en
Test settings
4; English source/prompt
23; non-English sources and no-English settings
Table 1: Comparison of experimental scope.
Setting
Gemma 2
Gemma 3
en2zh_en
42
45
zh2en_zh
53
104
zh2ja_zh
83
206
zh2ja_en
59
65
Intersection (all 4)
22
22
Table 2: Number of consistent features discovered under each setting.
Feature
Amplification Δ Comet
Ablation Δ Comet
en2zh_en
zh2en_zh
zh2ja_zh
zh2ja_en
en2zh_en
zh2en_zh
zh2ja_zh
zh2ja_en
( L 0, 8920)
+0.10
+0.37
-0.78
-0.14
-0.27
-0.30
+0.38
+0.84
( L 1, 12054)
-0.11
+0.16
-1.03
-0.35
-0.02
-0.21
+0.21
+0.44
( L 2, 15089)
+0.06
-0.29
-0.92
+0.31
+0.08
-0.20
+0.44
-0.18
( L 6, 13460)
-0.72
-0.46
-0.17
-0.21
-0.16
+0.15
-0.26
+0.75
( L 7, 3704)
+0.11
-0.14
-0.06
-0.32
-0.44
+0.07
-0.55
-0.09
Table 3: Gemma 2 : causal effects of the 22 features recovered in all four discovery settings. Values are changes in Comet relative to baseline; green/red cells mark effects with absolute value ≥1.0 . Exactly one feature, (L10, 5717) , shows the expected translation-switch signature (consistent positive amplification, consistent negative ablation); the remaining 21 features survived the same recurrence-based selection in the same settings, yet produce effects that are small, inconsistent in sign across settings, or both. Corresponding chrF++ results are reported in Appendix E .
Feature
Amplification Δ Comet
Ablation Δ Comet
en2zh_en
zh2en_zh
zh2ja_zh
zh2ja_en
en2zh_en
zh2en_zh
zh2ja_zh
zh2ja_en
( L 4, 2491)
-0.21
-0.40
+0.13
-0.83
+0.67
+0.21
+0.58
+0.86
( L 6, 40)
+0.29
-0.02
+0.05
+0.18
+0.09
-0.08
+0.08
+0.01
( L 6, 236)
+0.22
-0.07
+0.42
+0.15
-0.17
-0.14
+0.37
+0.15
( L 11, 6005)
+0.54
+0.22
+0.73
+0.18
-0.16
-0.26
+0.05
-0.11
( L 12, 273)
-3.14
-1.39
-2.05
-6.41
-19.72
-10.49
-17.94
-18.67
Table 4: Gemma 3 : causal effects of the 22 features recovered in all four discovery settings, following the same protocol and criterion as Table 3 . Exactly one feature, (L20, 2456) , shows the expected translation-switch signature (consistent positive amplification, consistent negative ablation). Note that layer indices are not comparable across models. Corresponding chrF++ results are reported in Appendix E .
Setting
Gemma 2 — ( L 10, 5717)
Gemma 3 — ( L 20, 2456)
Base.
Amp. Δ
Abl. Δ
Base.
Amp. Δ
Abl. Δ
en2zh_en
63.33
+4.61
− 5.85
74.89
+1.40
− 3.29
zh2en_zh
68.17
+2.33
− 0.49
76.25
+1.45
− 3.49
zh2ja_zh
62.66
+5.22
− 3.96
76.88
+2.23
− 3.78
zh2ja_en
67.34
+3.66
− 4.79
77.75
+0.59
− 4.15
en2ja_en
61.03
+5.59
− 7.49
77.19
+1.04
− 4.03
Table 5: Intervention effects for the single transferable feature in each model, ( L 10, 5717) in Gemma 2 and ( L 20, 2456) in Gemma 3, across the same 23 settings. Values are Comet scores or changes relative to baseline. All effects are significant at one-sided p<0.05 by paired bootstrap over sentence-level scores (1000 resamples) except those marked † . Amplification improves Comet in every setting for Gemma 2, and significantly improves it in 19 of 23 settings for Gemma 3; ablation degrades translation in all settings for Gemma 2 and in 22 of 23 for Gemma 3. The feature therefore transfers across prompt languages and translation directions in both models, with ar2en_ar in Gemma 3 the sole setting showing no effect in either direction. Corresponding chrF++ results are reported in Appendix E .
Figure 1: Dose-response check for intervening on (L10, 5717) in Gemma 2 and (L20, 2456) in Gemma 3 across the four discovery settings. Values are changes in Comet relative to the unmodified baseline ( α=1.0 ).
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Qualitative examples for interventions on (L10, 5717) in the en2zh_en setting. Baseline shows the model’s unmodified output. Amplified ( α=2 ) produces a more direct translation. Ablated ( α=0 ) causes the model to abandon translation entirely, instead generating English assistant-style responses (e.g., summarizing or offering advice about the input), consistent with the interpretation that this feature controls translation-task initiation.
Feature
Δ COMET
( L 4, 5301)
−0.47
( L 5, 697)
−0.17
( L 10, 4392)
+0.65
( L 11, 12945)
−1.06
( L 12, 1041)
−0.23
( L 13, 11248)
−0.25
Appendix
Table 7: Amplification effects for features that are consistent in en2zh_en but absent from the four-way core intersection. Values are changes in COMET under amplification with α=2.0 on en2zh_en . The baseline COMET score is 63.33. None of these features approaches the +4.61 effect of ( L 10, 5717) in the same setting.
Language
Baseline
Amplified
Δ
Chinese
0.024
0.039
+0.014
Arabic
0.007
0.005
-0.002
Russian
0.038
0.039
+0.001
Vietnamese
0.131
0.121
-0.011
Appendix
Table 8: Summarization results (ROUGE-L) on XL-Sum under amplification of ( L 10, 5717).
Language
Baseline
Amplified
Δ
Chinese
0.112
0.112
+0.000
Japanese
0.062
0.041
-0.021
Arabic
0.147
0.159
+0.012
Russian
0.198
0.191
-0.008
Vietnamese
0.251
0.255
+0.004
Appendix
Table 9: NER results (entity-set micro-F1) on WikiANN under amplification of ( L 10, 5717).
Setting
Gemma 2 — ( L 10, 5717)
Gemma 3 — ( L 20, 2456)
Amp. Δ chrF++
Abl. Δ chrF++
Amp. Δ chrF++
Abl. Δ chrF++
en2zh_en
+3.28
− 4.64
+1.14
− 1.40
zh2en_zh
+1.97
− 1.43
+0.64
− 1.97
zh2ja_zh
+2.49
− 2.34
− 0.15
− 1.19
zh2ja_en
+2.04
− 3.40
− 0.43
− 0.71
en2ja_en
+2.96
− 4.56
+0.28
− 1.03
Appendix
Table 11: chrF++ intervention effects for the single transferable feature in each model, ( L 10, 5717) in Gemma 2 and ( L 20, 2456) in Gemma 3, across the same 23 settings. Values are corpus-level chrF++ changes relative to baseline. Amplification generally improves chrF++, while ablation usually decreases it, in both models.