cs.CL · 2609.16967 Copy arXiv ID · Sep 15, 2026 Save Target-Language Generation in Multilingual Models: Activation Steering and Optimal Control Authors: James A. Michaelov , Carmen Amo Alonso , Tyler A. Chang , Roger P. Levy
Organizations: Massachusetts Institute of Technology · Stanford University · University of California San Diego
Abstract Ensuring that multilingual language models generate coherent text in a specific target language is a major issue in multilingual language modeling. We develop an optimal control method for target-language text generation as well as a framework for evaluating the quality of generated text in terms of language adherence, linguistic coherence, and semantic coherence. We find that the proposed method performs at least as well as the prominent difference-in-means activation steering method for the majority of models tested, with substantially less hyperparameter tuning required.
Explore similar work Aug 31, 2026 · Ryo Mitsuhashi, Sabri Boughorbel, Majd Hawasly Multilingual Language Models Code-Switching
Aug 31, 2026 · cs.CL J/K move · Enter open · S save
Ryo Mitsuhashi, Sabri Boughorbel, Majd Hawasly
Multilingual large language models can exhibit unintended code-switching -- unnecessarily alternating between languages during generation. We present a comparative study of three methods that identify language-controlling latents in cross-layer transcoders: activation value-based selection (ValSel), activation frequency-based selection (FreqSel), and LLM-generated latent annotation-based selection (AnnSel). To evaluate the efficacy of these methods in identifying language-controlling latents, we introduce two multilingual benchmarks that exhibit code-switching for fine-grained analysis of language steering across seven languages. Through targeted intervention experiments on Gemma-2-2B and Qwen3-4B, we find that all three methods effectively manipulate generation language, with FreqSel achieving the strongest overall performance, while AnnSel offering interpretable latent selection through explicit language annotations. A knock-out analysis suggests the methods select non-overlapping but each-functional latent subsets, indicating redundancy rather than a single canonical language direction. Code and data can be found at https://github.com/rm-3284/Latent-Mechanism-Multilingual.