Effective multi-agent collaboration requires finalizing solutions from reasoning that is distributed across the individual attempts of many agents. However, useful intermediate reasoning can remain unexploited when collaboration is limited to voting on conclusions reached among individual agents, or aggregating long, noisy collections of reasoning trajectories in context. In this work, we propose TTT-Consolidate, a framework that uses multi-agent reasoning trajectories as adaptation data to improve a model's subsequent reasoning. TTT-Consolidate jointly consolidates trajectories into parametric representations through test-time training, then generates a new solution by conditioning on the consolidated information. We meta-learn how to consolidate using bi-level optimization at training time, where an inner loop encodes reasoning trajectories into a lightweight LoRA adapter, and an outer loop learns meta-parameters for the inner loop such that the consolidation is helpful for the model to synthesize a final solution. As no large-scale corpus of diverse reasoning trajectories exists for training aggregation methods, we construct the OpenParallelThinking corpus composed of 12,810 math problems paired with a mixture of trajectories from six LLMs. Evaluated on challenging math and STEM benchmarks, TTT-Consolidate outperforms the best aggregation baselines by up to 20 percentage points, even under matched compute and runtime settings, with the gap widening at larger model scales. TTT-Consolidate also recovers from unsuccessful individual trajectories more robustly, an ability that improves as the number of available (even incorrect) trajectories increases. Finally, we demonstrate that TTT-Consolidate benefits from both SFT and RL-based outer loop objectives. We will release TTT-Consolidate and OpenParallelThinking for use in future work on reasoning aggregation.
Figures & tables
Figure 1: Meta-learning TTT-Consolidate for reasoning consolidation. The training procedure involves a nested inner and outer loop. The inner loop optimizes the likelihood of a batch of reasoning trajectories with respect to the parameters of the LoRA reasoning aggregator. In the outer loop, the model uses the merged reasoning information in the aggregator to synthesize the final solution. In both loops, only the LoRA is updated while the base LLM is frozen.
Figure 2: Left: Token length distribution of the OpenParallelThinking . Center: Correct ratio distribution OpenParallelThinking , reflecting a cosine-like difficulty profile shaped by correctness-based filtering. Right: Correctness distribution of the evaluation trajectory groups across three tiers.
Math Reasoning
STEM
Setting
Method
AIME25
AIME26
Apex25
ArXivMath
CMIMC25
HMMT-2602
HMMT-2511
IMO-Bench
GPQA
TheoremQA
Extended Thinking
Qwen3-1.7B
40.0
40.8
0.0
13.9
25.3
21.9
25.8
22.8
37.4
57.5
Qwen3-4B
71.7
68.3
0.0
16.7
52.2
40.5
60.4
37.0
51.5
68.6
Rule-based Agg.
Majority Voting @8 (None)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Majority Voting @8 (Minority)
26.7
23.3
25.0
18.1
30.0
30.3
16.7
21.5
16.2
19.5
Qwen3-1.7B None
Prompt Aggregation (CoT)
5.0
6.7
0.0
0.0
0.0
2.7
2.1
1.5
7.1
14.6
Table 1: Main results (%) on math and STEM reasoning benchmarks. Each row group corresponds to a model scale and trajectory quality setting. None : All 8 trajectories are incorrect. Minority : A minority of trajectories are correct. Bold shows the best result per column within each group. Underline reflects the best performance across all groups.
Figure 3: Aggregating collectively incorrect trajectories ( None ) with TTT-Consolidate . With both 1.7B and 4B models, TTT-Consolidate improves the most compared to other methods.
Figure 4: Scaling the number of weak trajectories under aggregation. As more None -tier weak trajectories are aggregated, baseline aggregations improve slightly or degrade while TTT-Consolidate continues to improve.
Method
FLOPs
Relative Compute
Minority Acc.
MV @8
7.019e14
0.67 ×
22.7
MV @13
1.141e15
1.09 ×
22.2
MV @18
1.579e15
1.50 ×
22.6
TTT-Consolidate @8
1.050e15
1.00 ×
35.4
Table 2: Majority voting vs. TTT-Consolidate on matched compute budget . MV matches TTT-Consolidate ’s compute budget by spending on extra rollouts. MV does not improve with more rollouts and stays well below TTT-Consolidate , even with more compute.
K
MV
In-context Aggregation Methods
TTT-Consolidate
FLOPs
Acc.
FLOPs
Prompt Acc.
SFT Acc.
RL Acc.
FLOPs
Acc.
2
1.755e14
0.0
2.227e14
4.3
2.2
1.9
2.897e14
15.2
4
3.509e14
0.0
4.105e14
3.4
3.9
1.7
5.432e14
23.0
8
7.019e14
0.0
7.890e14
5.5
3.7
2.4
1.050e15
23.6
16
1.404e15
0.0
1.558e15
4.7
6.7
0.8
2.064e15
26.5
Table 3: Accuracy vs. compute on the None tier, averaged across the three benchmarks. K denotes the number of input trajectories, all of which have incorrect final answers. We report accuracy (%) and total FLOPs for trajectory sampling and aggregation. The in-context FLOPs apply to all in-context aggregation methods (prompt-only, SFT, RL). TTT-Consolidate achieves higher accuracy even when the baselines use twice as many trajectories and more compute.
Figure 5: Does trajectory information interact in the consolidation step? Under Single-Trace × 8 (left), each trajectory is adapted and decoded independently, then reconciled by a vote. Under full TTT-Consolidate (middle), all 8 trajectories jointly update one LoRA before any answer is generated. Single-Trace × 8 underperforms full TTT-Consolidate on the None tier (right).
Figure 6: Robustness to majority manipulation. Dumbbells connect each method’s score under Majority (filled) and Majority-reversed (open, correct/incorrect answers swapped); bar length is the resulting drop. Baselines collapse while our method stays short in every panel, indicating consolidation rather than answer-tallying.
Figure 7: TTT-Consolidate enhanced with RLVR on None tier trajectories, shows consistent gains over TTT-Consolidate trained with SFT.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Total compute as K increases
Latency, sufficient parallel hardware
Majority voting
Θ(KL) sampling +Θ(K) voting =Θ(KL)
Θ(L) , independent of K with concurrent generation
Sampling parallelizes; consolidation is sequential, Θ(KL) , followed by Θ(T)
In-context aggregation
Θ(KL) sampling +Θ((KL)2) prefill +Θ(TKL+T2) decode =Θ(K2) w.r.t. K
Sampling parallelizes; aggregation cost grows with context length
Appendix
Table 4: Asymptotic scaling of aggregation methods with the number of sampled trajectories K . TTT-Consolidate matches majority voting’s linear compute in K , avoids the quadratic cost of in-context aggregation, and pays only in sequential latency.
Method
Setting
Pipeline FLOPs
Minority
None
Best-of-N, AceMath-7B-RM
N=10
1.040e15
24.5
0.0
PRM rerank, Qwen2.5-Math-PRM-7B
N=10 , last step
1.040e15
29.1
0.0
Recursive Refinement
N=8,k=4,T=2
1.048e15
17.9
5.4
TTT-Consolidate
K=8
1.050e15
35.4
23.6
Appendix
Table 5: Iso-compute comparison against reward-guided and recursive-refinement baselines. All methods run at TTT-Consolidate @8’s budget ( ≈ 1.05e15 pipeline FLOPs). Best-of-N and PRM reranking select from N=10 sampled solutions. Recursive Refinement runs T=2 rounds, each regenerating N=8 candidates from subsets of k=4 . Reranking scores 0.0 on None because it can only select an answer that is already in the pool, and every input there is wrong. TTT-Consolidate outperforms the strongest baseline on Minority and None .
Runtime
Memory
Accuracy
Method
Mode
Out. budget
Prompt
Consolidate
Decode
Total
Speed
Peak KV
Peak mem
Minority
None
tok
tok
s ↓
s ↓
s ↓
ms/tok ↓
GB ↓
GB ↓
% ↑
% ↑
Prompt agg
CoT
8,192
6,182
–
25.867
25.867
3.158
1.649
15.742
17.2
4.0
SFT agg
CoT
8,192
6,182
–
25.867
25.867
3.158
1.649
15.742
10.2
3.2
Prompt agg
Thinking
16,384
6,178
–
55.038
55.038
3.359
2.588
15.761
25.8
13.0
RL agg
Thinking
16,384
6,178
–
55.034
55.034
3.359
2.588
15.742
22.0
3.8
Appendix
Table 6: Inference runtime and memory usage on a single H100 GPU. Trajectory sampling is shared by all methods and excluded from runtime. Baselines decode from a ∼ 6.2K-token prompt of concatenated trajectories. TTT-Consolidate instead consolidates the trajectories into LoRA weights (1.9 s) and decodes from the 156-token problem alone. TTT-Consolidate has the lowest total latency and the smallest KV cache, while achieving the highest accuracy on both tiers. The cost is ∼ 6 GB higher peak memory for gradients and optimizer states.
Model
Trajc
SFT Agg
TTT-Consolidate
Gain
Qwen3-1.7B
None
3.2
23.6
+20.4
Qwen3-1.7B
Minority
10.2
35.4
+25.2
Qwen3-4B
None
7.7
40.8
+33.1
Qwen3-4B
Minority
15.4
57.6
+42.2
Appendix
Table 7: Comparing different consolidation methods trained with frontier supervision. SFT Aggregation (CoT) and TTT-Consolidate -SFT train on the same set of identical questions, trajectory groups, and frontier-generated reference solutions. TTT-Consolidate -SFT’s positive performance gain improves with model scale. Its improvement therefore comes from the meta-learned test-time training, not simply frontier supervision.
AIME26
Apex25
ArxivMath
HMMT2602
No aggregation at inference
SFT-1.7B
3.3
0
5.6
3.0
SFT-4B
14.2
0
11.1
6.1
With aggregation at inference
SFT-1.7B
0.8
0
0.7
1.5
SFT-4B
3.3
0
2.4
3.0
Appendix
Table 8: Outer-loop-only ablation. We remove the inner loop, reducing training to SFT on a single frontier-teacher CoT trace, ignoring sampled trajectories, evaluated with and without aggregation at inference. The outer loop alone is insufficient. SFT on frontier supervision does not match TTT-Consolidate ’s performance.
Backbone
Setting
AIME26
Apex25
ArxivMath
HMMT2602
Qwen3-1.7B (None)
Only inner
5.0
0
10.8
3.8
Inner + outer
26.7
16.7
23.6
18.2
Qwen3-1.7B (Minority)
Only inner
6.7
4.2
9.0
4.6
Inner + outer
30.0
25.0
34.7
27.3
Qwen3-4B (None)
Only inner
14.2
0
12.2
13.6
Inner + outer
39.1
33.3
37.9
48.5
Appendix
Table 9: Inner-loop-only ablation. We start each problem’s test-time update from an untrained LoRA initialization, without outer-loop meta-learning. Plain test-time training fails to consolidate trajectories, so both loops under the bilevel setup are needed.
Condition
Inner-loop input
Avg. acc.
Δ pre-adapt.
Δ post-adapt.
Isolate
Pre-adaptation
No inner-loop update
7.5%
–
−14.5
Initialization alone
Passive repetition
8 copies of the target question
5.4%
−2.1
−16.6
Repeated exposure
Task activation
Trajectory groups from other problems, same task distribution
4.0%
−3.5
−18.0
Task-distribution activation w/o target-specific info
Single-Trace × 8
One trajectory per LoRA, 8 LoRAs, majority vote
11.7%
+4.2
−10.3
Absence of cross-trajectory interaction
TTT-Consolidate
All 8 target trajectories, jointly
22.0%
+14.5
–
–
Appendix
Table 10: Isolating the source of TTT-Consolidate ’s improvement. Every ablation control falls far below full TTT-Consolidate . Our method’s performance gain requires jointly consolidating the target problem’s trajectories during adaptation.
AIME26
Apex25
ArxivMath
HMMT2602
GPQA
TTT-Consolidate -SFT
Before TTT
16.7
0
2.8
6.1
6.6
After TTT
26.7
16.7
23.6
18.2
22.7
Gain from adaptation
+10.0
+16.7
+20.8
+12.1
+16.1
TTT-Consolidate -RL
Before TTT
16.7
0
4.2
9.1
19.7
After TTT
23.5
16.7
26.0
21.7
40.0
Gain from TTT
+6.8
+16.7
+21.8
+12.6
+20.3
Appendix
Table 11: TTT-Consolidate model with and without test-time adaptation on four None tier benchmarks. Inner-loop adaptation improves both the SFT and RL models on every benchmark. Performance gains do not come from the initialization alone.
Table 12: Matched SFT → GRPO training path for RL Aggregation and TTT-Consolidate . Both methods get the same frontier-supervised initialization and GRPO budget. RL Aggregation stays behind; thus, RL alone does not explain TTT-Consolidate ’s improvement.
Benchmark
Domain
# Questions
Math Reasoning
AIME 2025
Competition Math
30
AIME 2026
Competition Math
30
Apex 2025
Competition Math
12
ArXiv Math
Research Math
72
CMIMC 2025
Competition Math
40
Appendix
Table 13: Summary of the ten reasoning benchmarks. Each question is used to generate 40 candidate trajectories (5 source models × 8 samples), from which three quality-stratified trajectory groups are formed.
Figure 9: Left : Distribution of extracted CoT lengths over reasoning trajectory collection. Right : Source model composition by quality tier, where weaker models dominate the None tier while stronger models contribute more in the Majority tier.
Component
Content
Judge Prompt
You are evaluating whether a model’s final solution to a problem reflects
{synthesis vs. artifact exploitation ∣ consolidation vs. memorization}.
When multiple LLM agents solve the same problem, standard practice compresses each agent's reasoning into a majority vote or layered synthesis, treating agreement as the finish line. We show this is unnecessarily lossy: an LLM aggregator that reads complete reasoning traces recovers correct solutions even when agents unanimously agree, with beneficial corrections consistently outweighing harmful ones -- the aggregation paradox. Majority voting has a ceiling that perturbation diversity does not raise (error correlations are identical); the aggregator's gain comes from trace-level complementarity, assembling correct intermediate steps from minority chains that voting discards. These findings motivate Self-Consistent Mixture of Agents which generates trace diversity through semantic-preserving input perturbations, safeguards the majority via anchored refinement with provable non-degradation guarantees, and always synthesizes -- never gates on consensus. A single model with perturbation-induced trace variation outperforms heterogeneous model pools across structured reasoning, PhD-level science, competition mathematics, and competitive programming. The unit of aggregation should be the reasoning trace, not the answer.
Shreyas Fadnavis, Praitayini Kanakaraj, Felix Wyss
Test-time scaling has become an effective paradigm for improving the reasoning ability of large language models by allocating additional computation during inference. Recent structured approaches have further advanced this paradigm by organizing inference across multiple trajectories, refinement rounds, and verification-based feedback. However, existing structured test-time scaling methods either weakly coordinate parallel reasoning trajectories or rely on noisy historical information without explicitly deciding what should be retained and reused, limiting their ability to balance exploration and exploitation. In this work, we propose TMAS, a framework for scaling test-time compute via multi-agent synergy. TMAS organizes inference as a collaborative process among specialized agents, enabling structured information flow across agents, trajectories, and refinement iterations. To support effective cross-trajectory collaboration, TMAS introduces hierarchical memories: the experience bank reuses low-level reliable intermediate conclusions and local feedback, while the guideline bank records previously explored high-level strategies to steer subsequent rollouts away from redundant reasoning patterns. Furthermore, we design a hybrid reward reinforcement learning scheme tailored to TMAS, which jointly preserves basic reasoning capability, enhances experience utilization, and encourages exploration beyond previously attempted solution strategies. Extensive experiments on challenging reasoning benchmarks show that TMAS achieves stronger iterative scaling than existing test-time scaling baselines, with hybrid reward training further improving scaling effectiveness and stability across iterations. Code and data are available at https://github.com/IQuestLab/tmas.
Multi-agent LLM systems have shown promise for complex reasoning, yet recent evaluations reveal they often underperform single-model baselines. We identify a structural failure mode in sequential fine-tuning of shared-context teams: updating one agent shifts the team's context distribution, and when subsequent updates are evaluated on cached rollouts, this mismatch compounds. We formalize this as the compounding occupancy shift and prove that stale-occupancy evaluation incurs a penalty that scales quadratically with the number of agents. In contrast, intermediate-occupancy evaluation reduces this to linear scaling. We propose TeamTR, a trust-region framework that resamples trajectories after each component update and enforces per-agent divergence control, yielding rigorous per-update and per-stage improvement lower bounds. Experiments show that TeamTR outperforms single-agent and sequential baselines with 7.1% on average, mitigates coordination regressions, and supports plug-and-play component replacement. Code is available at https://github.com/Yydc/TeamTR.
Yi Xie, Siao Liu, Falong Fan +3
Department of Electrical & Computer Engineering, University of Arizona · Future Science and Engineering College, Soochow University · INSAIT, Sofia University ”St. Kliment Ohridski” +1