In this work, we reveal that Large Language Models (LLMs) possess intrinsic behavioral plasticity-akin to chameleons adapting their coloration to environmental cues-that can be exposed through token-conditional generation and stabilized via reinforcement learning. Specifically, by conditioning generation on carefully selected token prefixes sampled from responses exhibiting desired behaviors, LLMs seamlessly adapt their behavioral modes at inference time (e.g., switching from step-by-step reasoning to direct answering) without retraining. Based on this insight, we propose Token-Conditioned Reinforcement Learning (ToCoRL), a principled framework that leverages RL to internalize this chameleon-like plasticity, transforming transient inference-time adaptations into stable and learnable behavioral patterns. ToCoRL guides exploration with token-conditional generation and keep enhancing exploitation, enabling emergence of appropriate behaviors. Extensive experiments show that ToCoRL enables precise behavioral control without capability degradation. Notably, we show that large reasoning models, while performing strongly on complex mathematics, can be effectively adapted to excel at factual question answering, which was a capability previously hindered by their step-by-step reasoning patterns.
Figures & tables
Figure 1: Like chameleons changing their coloration in response to external stimuli (top), language models can adapt their behavior according to the token prefix (bottom). ⊕ stands for concatenation.
Figure 2: Top : Conditioning response generation on a direct-answer prefix enables LRM to switch from step-by-step reasoning to direct knowledge retrieval, revealing a new perspective on language model behavioral plasticity. Middle : ToCoRL leverages token-conditional generation to guide exploration during RL training, inducing the emergence of new factual answering behaviors. Bottom : After an initial direct answer, the ToCoRL-trained model adopts an emergent recalibrative reasoning behavior for factual problems (see Section 5.2 ), while its complex math problem–solving behavior remains unchanged. We provide concrete examples for further illustration in Fig. 4 .
Accuracy
Avg. length
Med. length
The Normal Gen
18.9%
1255
1128
ToCo Gen
20.7%
891
477
Table 1: The following table shows the accuracy (avg@8), the average response length (Avg. length) and medium response length (Med. length) in token number, when tested on SimpleQA benchmark. Note that we use ‘Normal Gen’ and ‘ToCo Gen’ to denote normal generation and token-conditional generation respectively.
Qwen3-30A3B
SimpleQA
Omnisci
AIME’24
AIME’25
Thinking
18.9
16.1
84.8
80.5
Instruct
22.1
10.5
75.2
61.3
w. ToCoRL
28.3
18.6
89.0
81.5
w. GRPO
23.6
16.7
89.5
81.4
w. Adapt-Think
23.9
16.0
86.9
78.9
Table 2: Evaluation results for experiments based on Qwen3-30B-A3B-2507-Thinking. We use ‘Thinking’ and ‘Instruct’ to denote the corresponding two open-source models, and ‘w.’ to denote the training method used for factual problem queries. ‘Adapt-Think’, ‘I-T GRPO’, and ‘Omnisci’ are abbreviations for Adaptive-Thinking, Instruct-Tuning GRPO, and AA-Omniscience, respectively, and we consistently use these notations in subsequent tables.
Figure 3: For clearer visualization, we plot the curves of these metrics over the first training epoch. Compared to ToCoRL, GRPO extends reasoning length while keeping the original reasoning style. Since unnecessary associations and unverified information persist, longer reasoning yields minor improvements, and response lengths eventually return to their initial levels. Adaptive-Thinking penalizes all reasoning, so evaluation scores rise quickly at first as ineffective context is removed. However, without additional reasoning capacity, the LRM’s potential is underutilized, limiting further gains. Instruct-tuning RL initially shortens reasoning, but the prompt instruction gradually loses control over the behavior.
Qwen3-30A3B
SimpleQA
Omnisci
AIME’24
AIME’25
Base
24.2
12.0
15.1
9.4
SFT
19.6
14.0
56.5
38.7
w. ToCoRL
28.1
15.8
72.1
61.7
w. GRPO
22.6
14.3
72.0
61.9
w. Adapt-Think
24.8
13.8
47.9
30.4
Table 3: Results for experiments based on Qwen3-30B-A3B-SFT, obtained by enabling the base model (denoted as ‘Base’) with a unified reasoning behavior. We refer to it as ‘SFT’ for brevity.
(λ,k)
SimpleQA
Omnisci
AIME’24
AIME’25
(1,6)
28.3
18.6
89.0
81.5
(1,7)
27.5
18.1
88.2
80.6
(1,8)
27.6
18.3
88.5
80.9
(0.5,6)
28.1
18.3
88.3
80.5
(2,6)
27.2
17.8
88.0
80.0
Table 4: Results for ablation studies on hyperparameters ( λ and k ). We skip testing k<6 since the required < think > \n prefix (occupying 2 tokens) makes the prefix too short to form a direct attempt at answering the problem.
πInst
SimpleQA
Omnisci
AIME’24
AIME’25
Qwen3-235A22B
28.3
18.6
89.0
81.5
Qwen3-30A3B
28.0
18.6
89.1
81.3
Qwen2.5-14B
27.8
18.3
89.0
81.1
Table 5: Results for ablation studies on different prefix providers ( πInst ). For brevity, the ‘Instruct’ suffix is omitted for all models. Note that Qwen3 models in the table refer to the 2507 version.
Response Provider
SimpleQA
Omnisci
AIME’24
AIME’25
30A3B-w. ToCoRL
29.1
16.0
74.5
62.0
235A22B-Instruct *
28.3
15.5
68.6
55.2
30A3B-Instruct *
25.3
14.5
69.3
59.2
Table 6: We use ’Response Provider’ to denote the source of the factual responses used for new SFT datasets as described above. All providers are from the Qwen3 series; * indicates manually constructed responses, otherwise are generated normally.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Qwen3-235B-A22B-2507
Qwen3-30B-A3B-2507
Thinking
Instruct
Thinking
Instruct
AIME’25
92.3
70.3
80.5
61.3
SimpleQA
47.0
54.3
19.2
22.1
Appendix
Table 7: Comparison of accuracy on complex math benchmark (AIME’25) and factual problem benchmark (SimpleQA). In the same model series, the Thinking version, which conduct step-by-step reasoning before arriving at final answer, consistently perform better on complex math benchmark and worse on factual problem benchmark.
Figure 4: Detailed demonstration of the behavior change brought by token-conditional generation and ToCoRL. Concrete queries and responses are provided.
Figure 5: Since most experiments are conducted with large reasoning models, we only illustrate the procedure in this setting. Moreover, we only show the case where the length of token prefix k=3 as an example. That is, besides the < think > token, two initial tokens from direct answer (in orange) are used for constructing token prefix.
Inference-time language model alignment steers or selects outputs during generation to better meet safety, preference, and task goals without retraining.