Despite significant advances in Large Reasoning Models (LRMs) driven by reinforcement learning with verifiable rewards (RLVR), this paradigm is fundamentally limited in specialized or novel domains where such supervision is prohibitively expensive or unavailable, posing a key challenge for test-time adaptation. While existing test-time methods offer a potential solution, they are constrained by learning from static query sets, risking overfitting to textual patterns. To address this gap, we introduce Test-Time Variational Synthesis (TTVS), a novel framework that enables LRMs to self-evolve by dynamically augmenting the training stream from unlabeled test queries. TTVS comprises two synergistic modules: (1) Online Variational Synthesis, which transforms static test queries into a dynamic stream of diverse, semantically-equivalent variations, enforcing the model to learn underlying problem logic rather than superficial patterns; (2) Test-time Hybrid Exploration, which balances accuracy-driven exploitation with consistency-driven exploration across synthetic variants. Extensive experiments show TTVS yields superior performance across eight model architectures. Notably, using only unlabeled test-time data, TTVS not only surpasses other test-time adaptation methods but also outperforms state-of-the-art supervised RL-based techniques trained on vast, high-quality labeled data.
Figures & tables
Figure 1: The schematic illustration of the Test-time Variational Synthesis (TTVS). In Stage 1: Online Variational Synthesis. Pseudo labels are generated from test data to synthesize diversified samples, which are then online filtered for quality and diversity. In Stage 2: Test-time Hybrid Exploration. A policy model is optimized on this augmented data through a hybrid intra- and cross-group exploration strategy, culminating in diversified rewards that guide the model’s optimization.
Methods
MATH500
AIME2024
AMC2023
GPQA
Average
RL Post-trained Models w / Labeled Data
DeepSeek-R1-Distill-1.5B (800K)
52.2
2.5
21.7
14.6
22.8
[1pt/2pt] DeepSeek-R1-Distill-7B (800K)
60.1
10.0
26.2
25.7
30.5
[1pt/2pt] OpenReasoner-Zero-7B (129K)
79.2
13.3
47.0
28.4
42.0
[1pt/2pt] SimpleRL-Zero-7B (8.9K)
78.2
26.7
60.2
27.6
48.2
Qwen3 Family
Table 1: Performance comparison of TTVS with state-of-the-art methods on various model architectures. For RL Post-trained Models w / Labeled Data ( ⋅ ), ( ⋅ ) indicates the amount of labeled data used during post-training.
Methods
MATH500
AMC2023
GPQA
Qwen3-4B
64.0
26.5
26.3
[1pt/2pt] ↪ CGE
86.4
62.6
43.9
↪ IGE
88.4
69.8
46.6
[1pt/2pt] ↪ TTVS
90.3
71.1
48.0
Table 2: Performance analysis of different components in TTVS.
Figure 2: Entropy Curve of TTVS components on AMC2023 using Qwen3-4B.
Methods
MATH500
AIME2024
AMC2023
Qwen3-4B
64.0
10.0
26.5
[1pt/2pt] ↪ TTVS ( w/ OPO)
89.6
33.3
69.8
[1pt/2pt] ↪ TTVS ( w/ DAPO)
88.8
36.7
71.6
[1pt/2pt] ↪ TTVS ( w/ GRPO)
90.3
36.7
71.1
Table 3: Performance comparison of different RL algorithms for TTVS using Qwen3-4B.
Figure 3: Comparative performance analysis across the five difficulty levels of MATH-500.
Figure 4: Computational cost during test-time training on Qwen3-4B
Methods
GPU Memory (GB)
Token Length
Throughput (Token/s)
TTRL
401.4
1925.5
15.60
[1pt/2pt] TTVS
398.0
1864.9
14.28
Table 4: Computational cost at inference phase based on Qwen3-4B using 32 rollouts
Filtering thresholds
Qwen3-4B
Qwen2.5-Instruct-3B
τlow=0.0
τhigh=0.875
86.4
72.0
τlow=0.125
τhigh=0.875
90.3
75.0
τlow=0.25
τhigh=0.875
86.2
72.4
τlow=0.125
τhigh=0.625
88.4
73.8
τlow=0.125
τhigh=1.0
88.7
73.2
Table 5: Ablation study for filtering thresholds
Methods
Qwen3-4B
Qwen2.5-Instruct-3B
512
86.1
72.2
1024
90.3
75.0
2048
90.7
74.9
Table 6: Ablation study for Maximum Token Length.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Hyperparameter analysis of the warmup steps for intra-group exploration ( Eintra ) on various reasoning datasets..
Figure 6: Hyperparameter analysis of the warmup steps cross-group exploration ( Ecross ) on various reasoning datasets.
Methods
MATH500
AMC2023
GPQA
Qwen3-4B
64.0
26.5
26.3
[1pt/2pt] TTRL (32 Rollouts)
85.0
61.4
43.0
TTRL (64 Rollouts)
86.9
65.5
45.0
TTVS (32 Rollouts)
90.3
71.1
48.0
Appendix
Table 7: Performance comparison with increased rollouts for TTRL
Methods
Qwen3-4B
Qwen2.5-Instruct-3B
TTVS (16 Rollouts)
88.0
73.8
TTVS (32 Rollouts)
90.3
75.0
TTVS (64 Rollouts)
92.9
76.2
Appendix
Table 8: Ablation study for the number of rollouts for TTVS.
Beijing University of Posts and Telecommunications · Southwestern University of Finance and Economics · China Unicom Online Information Technology Co., Ltd.