Test-Time Training offers a promising way to improve the reasoning ability of large language models (LLMs) by adapting the model using only the test questions. However, existing methods struggle with difficult reasoning problems for two reasons: raw test questions are often too difficult to yield high-quality pseudo-labels, and the limited size of test sets makes continuous online updates prone to instability. To address these limitations, we propose TTCS, a co-evolving test-time training framework. Specifically, TTCS initializes two policies from the same pretrained model: a question synthesizer and a reasoning solver. These policies evolve through iterative optimization: the synthesizer generates progressively challenging question variants conditioned on the test questions, creating a structured curriculum tailored to the solver's current capability, while the solver updates itself using self-consistency rewards computed from multiple sampled responses on both original test and synthetic questions. Crucially, the solver's feedback guides the synthesizer to generate questions aligned with the model's current capability, and the generated question variants in turn stabilize the solver's test-time training. Experiments show that TTCS consistently strengthens the reasoning ability on challenging mathematical benchmarks and transfers to general-domain tasks across different LLM backbones, highlighting a scalable path towards dynamically constructing test-time curricula for self-evolving. Our code and implementation details are available at https://github.com/XMUDeepLIT/TTCS.
Figures & tables
Figure 1: Comparison of TTRL and our TTCS: (a) When applied to difficult test questions such as AIME24, TTRL suffers from noisy rewards caused by incorrect majority voting consensus. (b) TTCS synthesizes tractable variants to ensure valid pseudo-labels, providing reliable supervision for stable self-evolution.
Figure 2: Overview of TTCS, a co-evolving test-time training framework. ( a ) Synthesizer training : the synthesizer first rollout synthetic questions conditioned on test questions with the prompt template, and then is optimized via GRPO with the question quality reward. ( b ) Solver training : the solver performs online self-evolving on a mixture of test and synthetic questions with self-supervised rewards, updated via GRPO using the self-consistency reward.
Method
AIME 2024
AIME 2025
AMC23
MATH-500
Minerva
OlympiadBench
AVG
Qwen2.5-Math-1.5B
Pretrained model Qwen et al. (2025)
7.10
4.20
27.50
33.20
9.60
22.20
17.30
Self-Consistency Wang et al. (2023)
13.30
10.00
50.00
49.80
10.70
31.90
27.62
R-Zero Huang et al. (2025)
10.00
4.58
47.50
66.20
30.88
31.01
31.70
TTRL Zuo et al. (2025)
13.23
9.38
55.00
71.20
34.93
35.61
36.56
TTCS (Ours)
19.79
13.33
62.50
76.80
40.44
36.05
41.49
Table 1: Comparison of our TTCS framework against other baselines on mathematical benchmarks. Best results are highlighted in bold.
Figure 3: Generalization analysis. (a) General-domain generalization: Accuracy trends of TTCS and TTRL on general-domain reasoning benchmarks (MMLU-Pro, SuperGPQA) during test-time training on AIME25. The green dashed line indicates the R-Zero baseline. (b) Mathematical-domain generalization: Performance comparison on out-of-distribution (OOD) mathematical benchmarks.
Method
AIME24
AIME25
Minerva
AVG
Qwen2.5-Math-1.5B
13.23
9.38
34.93
19.18
+Strong Synthesizer
16.35
10.21
38.97
21.84
+ TTCS(Ours)
19.79
13.33
40.44
24.52
Table 2: Performance comparison on the Qwen2.5-Math-1.5B model investigating the impact of synthesizer capability. The Strong synthesizer variant employs a fixed Qwen2.5-14B-Instruct model to generate questions.
Figure 4: Data efficiency analysis of test-time training on AIME24.
Settings
AMC23
Olympiad
Minerva
Score
Δ
Score
Δ
Score
Δ
TTCS (Full)
62.50
–
36.05
–
40.44
–
w/o Synthesizer Training
55.00
↓ 7.50
32.79
↓ 3.26
37.50
↓ 2.94
w/o Online Data Filter
60.00
↓ 2.50
33.68
↓ 2.37
38.60
↓ 1.84
w/o Diversity Penalty
55.00
↓ 7.50
35.31
↓ 0.74
39.34
↓ 1.10
Table 3: Ablation study of TTCS components on the Qwen2.5-Math-1.5B model. Performance drops ( Δ ) are computed relative to the full TTCS setting.
Iteration
Case 1: Function Intersection
Case 2: Roots of Unity
Test Question (AIME24)
Define f(x)=∣∣x∣−21∣ and g(x)=∣∣x∣−41∣ . Find the number of intersections of the graphs of y=4g(f(sin(2πx))) and x=4g(f(cos(3πy))) .
Let ω=1 be a 13th root of unity. Find the remainder when ∏k=012(2−2ωk+ω2k) is divided by 1000.
Iter 1
Find the number of integer solutions (x,y) to the equation 4x2−16y2+14x−24y+9=0 .
The polynomial f(x)=x3+x2−x−1 has three distinct roots, p , q , and r . What is p3q3+r3q3+p3r3 ?
Iter 5
How many points (x,y) lie at intersections between the graphs of x=∣sin(πy)∣ and y=∣sin(πx)∣ ?
Find the sum of all complex numbers w such that the real part of w is greater than 0 and w3+(4−758i)w2−(6+13i)w+10=0 .
Iter 10
Find the number of points where the graphs of y=∣x+1∣+∣x−1∣ and y=∣2x−1∣+∣3x−2∣ intersect.
Let ω=1 be a 13th root of unity. Compute the remainder when ∑n=012ωn is divided by 1000.
Iter 15
Find the sum of the x -coordinates of the points where the graph of f(x)=sin−1(1+2x23x−x3) intersects the line y=4π , for −2≤x≤2 .
Let N be the number of ordered triples (a,b,c) of positive integers such that a2+b2+c2=2025 . Find N modulo 1000.
Table 4: Evolution of synthesized questions across training iterations. Two representative cases from AIME24 are shown, demonstrating how our self-evolving framework progressively improves question synthesis quality.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Config
Synthesizer Training
Solver Training
Batch Size
32
64
Learning Rate
1e-6
1e-6
Weight Decay
0.01
0.01
KL Coefficient
0.01
0.01
Max Steps
5
15
Rollout Group Size
4
8
Appendix
Table 5: Detailed training configurations and hyper-parameters for the synthesizer and solver agents.
Figure 5: The general-domain performance comparison of TTCS and the other baselines when TTCS and TTRL trained on AIME24 dataset.
Figure 6: The general-domain performance comparison of TTCS and the other baselines when TTCS and TTRL trained on AIME25 dataset.
Figure 7: The out-of-distribution performance comparison of TTCS and other baselines on mathematical benchmarks.
Test-time training adapts model parameters or memory during deployment using each input or stream to improve performance under changing tasks and data.
Test-time training (TTT) adapts large language models (LLMs) during inference using only unlabeled test inputs. Existing methods, however, face two major bottlenecks on hard reasoning tasks: (1) lack of learnable samples, as self-generated pseudo-labels on difficult questions are often noisy and yield unstable rewards; and (2) inefficient exploration, as performance gains depend on repeatedly sampling many rollouts without explicit diagnosis of why previous attempts fail. We propose TTSR (Test-Time Self-Reflection), a self-evolving framework based on a reflect-then-synthesize paradigm. A single pretrained model alternates between a Student role and a Teacher role: the Student solves test questions and updates, while the Teacher analyzes failed trajectories and synthesizes targeted variant questions closer to the Student's capability frontier. TTSR further maintains a cross-iteration weakness memory and compiles persistent weaknesses into a lightweight strategy note prepended to subsequent Student inputs, so diagnostic knowledge can guide exploration and gradually fade as weaknesses are resolved. Experiments on challenging mathematical reasoning benchmarks show consistent test-time improvements, strong cross-backbone generalization, and transfer to general-domain reasoning tasks.
Haoyang He, Zihua Rong, Yunjia Zhao +3
Beijing University of Posts and Telecommunications · Southwestern University of Finance and Economics · China Unicom Online Information Technology Co., Ltd.
Large language models (LLMs) are typically deployed with fixed parameters, and their performance is often improved by allocating more computation at inference time. While such test-time scaling can be effective, it cannot correct model misconceptions or adapt the model to the specific structure of an individual query. Test-time optimization addresses this limitation by enabling parameter updates during inference, but existing approaches either rely on external data or optimize generic self-supervised objectives that lack query-specific alignment. In this work, we propose Query-Conditioned Test-Time Self-Training (QueST), a framework that adapts model parameters during inference using supervision derived directly from the input query. Our key insight is that the input query itself encodes latent signals sufficient for constructing structurally related problem--solution pairs. Based on this, QueST generates such query-conditioned pairs and uses them as supervision for parameter-efficient fine-tuning at test time. The adapted model is then used to produce the final answer, enabling query-specific adaptation without any external data. Across seven mathematical reasoning benchmarks and the GPQA-Diamond scientific reasoning benchmark, QueST consistently outperforms strong test-time optimization baselines. These results demonstrate that query-conditioned self-training is an effective and practical paradigm for test-time adaptation in LLMs. Code is available at https://chssong.github.io/Query-Conditioned-TTST/.
Chaehee Song, Minseok Seo, Yeeun Seong +2
School of Electrical Engineering, KAIST · 2Graduate School of Green Growth and Sustainability, KAIST
Test-time adaptation offers a promising avenue for improving reasoning performance in large language models without additional supervision, but existing approaches often apply a uniform optimization objective across all inputs, leading to inefficient or unstable adaptation on heterogeneous reasoning problems. We propose DiSCTT, a difficulty-aware, consensus-guided self-curriculum framework that dynamically allocates test-time optimization strategies based on instance-level epistemic uncertainty estimated from agreement among sampled reasoning trajectories. Inputs with high consensus are consolidated via supervised fine-tuning using majority-agreed solutions as pseudo-labels, while low-consensus inputs are optimized via reinforcement learning with a consensus-regularized objective that encourages diversity under relevance constraints. Across a broad suite of mathematical and general reasoning benchmarks, DiSCTT consistently outperforms strong test-time adaptation baselines, achieving higher accuracy with reduced variance and substantially lower computation and wall-clock training times. These results demonstrate that explicitly accounting for instance difficulty and uncertainty enables more stable, efficient, and effective test-time adaptation for reasoning models.
Mohammad Mahdi Moradi, Sudhir Mudur
Department of Compute Science, Concordia University, Montreal, Canada.