Effective multi-agent collaboration requires finalizing solutions from reasoning that is distributed across the individual attempts of many agents. However, useful intermediate reasoning can remain unexploited when collaboration is limited to voting on conclusions reached among individual agents, or aggregating long, noisy collections of reasoning trajectories in context. In this work, we propose TTT-Consolidate, a framework that uses multi-agent reasoning trajectories as adaptation data to improve a model's subsequent reasoning. TTT-Consolidate jointly consolidates trajectories into parametric representations through test-time training, then generates a new solution by conditioning on the consolidated information. We meta-learn how to consolidate using bi-level optimization at training time, where an inner loop encodes reasoning trajectories into a lightweight LoRA adapter, and an outer loop learns meta-parameters for the inner loop such that the consolidation is helpful for the model to synthesize a final solution. As no large-scale corpus of diverse reasoning trajectories exists for training aggregation methods, we construct the OpenParallelThinking corpus composed of 12,810 math problems paired with a mixture of trajectories from six LLMs. Evaluated on challenging math and STEM benchmarks, TTT-Consolidate outperforms the best aggregation baselines by up to 20 percentage points, even under matched compute and runtime settings, with the gap widening at larger model scales. TTT-Consolidate also recovers from unsuccessful individual trajectories more robustly, an ability that improves as the number of available (even incorrect) trajectories increases. Finally, we demonstrate that TTT-Consolidate benefits from both SFT and RL-based outer loop objectives. We will release TTT-Consolidate and OpenParallelThinking for use in future work on reasoning aggregation.
Figures & tables
Figure 1: Meta-learning TTT-Consolidate for reasoning consolidation. The training procedure involves a nested inner and outer loop. The inner loop optimizes the likelihood of a batch of reasoning trajectories with respect to the parameters of the LoRA reasoning aggregator. In the outer loop, the model uses the merged reasoning information in the aggregator to synthesize the final solution. In both loops, only the LoRA is updated while the base LLM is frozen.
Figure 2: Left: Token length distribution of the OpenParallelThinking . Center: Correct ratio distribution OpenParallelThinking , reflecting a cosine-like difficulty profile shaped by correctness-based filtering. Right: Correctness distribution of the evaluation trajectory groups across three tiers.
Math Reasoning
STEM
Setting
Method
AIME25
AIME26
Apex25
ArXivMath
CMIMC25
HMMT-2602
HMMT-2511
IMO-Bench
GPQA
TheoremQA
Extended Thinking
Qwen3-1.7B
40.0
40.8
0.0
13.9
25.3
21.9
25.8
22.8
37.4
57.5
Qwen3-4B
71.7
68.3
0.0
16.7
52.2
40.5
60.4
37.0
51.5
68.6
Rule-based Agg.
Majority Voting @8 (None)
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Majority Voting @8 (Minority)
26.7
23.3
25.0
18.1
30.0
30.3
16.7
21.5
16.2
19.5
Qwen3-1.7B None
Prompt Aggregation (CoT)
5.0
6.7
0.0
0.0
0.0
2.7
2.1
1.5
7.1
14.6
Table 1: Main results (%) on math and STEM reasoning benchmarks. Each row group corresponds to a model scale and trajectory quality setting. None : All 8 trajectories are incorrect. Minority : A minority of trajectories are correct. Bold shows the best result per column within each group. Underline reflects the best performance across all groups.
Figure 3: Aggregating collectively incorrect trajectories ( None ) with TTT-Consolidate . With both 1.7B and 4B models, TTT-Consolidate improves the most compared to other methods.
Figure 4: Scaling the number of weak trajectories under aggregation. As more None -tier weak trajectories are aggregated, baseline aggregations improve slightly or degrade while TTT-Consolidate continues to improve.
Method
FLOPs
Relative Compute
Minority Acc.
MV @8
7.019e14
0.67 ×
22.7
MV @13
1.141e15
1.09 ×
22.2
MV @18
1.579e15
1.50 ×
22.6
TTT-Consolidate @8
1.050e15
1.00 ×
35.4
Table 2: Majority voting vs. TTT-Consolidate on matched compute budget . MV matches TTT-Consolidate ’s compute budget by spending on extra rollouts. MV does not improve with more rollouts and stays well below TTT-Consolidate , even with more compute.
K
MV
In-context Aggregation Methods
TTT-Consolidate
FLOPs
Acc.
FLOPs
Prompt Acc.
SFT Acc.
RL Acc.
FLOPs
Acc.
2
1.755e14
0.0
2.227e14
4.3
2.2
1.9
2.897e14
15.2
4
3.509e14
0.0
4.105e14
3.4
3.9
1.7
5.432e14
23.0
8
7.019e14
0.0
7.890e14
5.5
3.7
2.4
1.050e15
23.6
16
1.404e15
0.0
1.558e15
4.7
6.7
0.8
2.064e15
26.5
Table 3: Accuracy vs. compute on the None tier, averaged across the three benchmarks. K denotes the number of input trajectories, all of which have incorrect final answers. We report accuracy (%) and total FLOPs for trajectory sampling and aggregation. The in-context FLOPs apply to all in-context aggregation methods (prompt-only, SFT, RL). TTT-Consolidate achieves higher accuracy even when the baselines use twice as many trajectories and more compute.
Figure 5: Does trajectory information interact in the consolidation step? Under Single-Trace × 8 (left), each trajectory is adapted and decoded independently, then reconciled by a vote. Under full TTT-Consolidate (middle), all 8 trajectories jointly update one LoRA before any answer is generated. Single-Trace × 8 underperforms full TTT-Consolidate on the None tier (right).
Figure 6: Robustness to majority manipulation. Dumbbells connect each method’s score under Majority (filled) and Majority-reversed (open, correct/incorrect answers swapped); bar length is the resulting drop. Baselines collapse while our method stays short in every panel, indicating consolidation rather than answer-tallying.
Figure 7: TTT-Consolidate enhanced with RLVR on None tier trajectories, shows consistent gains over TTT-Consolidate trained with SFT.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Total compute as K increases
Latency, sufficient parallel hardware
Majority voting
Θ(KL) sampling +Θ(K) voting =Θ(KL)
Θ(L) , independent of K with concurrent generation
Sampling parallelizes; consolidation is sequential, Θ(KL) , followed by Θ(T)
In-context aggregation
Θ(KL) sampling +Θ((KL)2) prefill +Θ(TKL+T2) decode =Θ(K2) w.r.t. K
Sampling parallelizes; aggregation cost grows with context length
Appendix
Table 4: Asymptotic scaling of aggregation methods with the number of sampled trajectories K . TTT-Consolidate matches majority voting’s linear compute in K , avoids the quadratic cost of in-context aggregation, and pays only in sequential latency.
Method
Setting
Pipeline FLOPs
Minority
None
Best-of-N, AceMath-7B-RM
N=10
1.040e15
24.5
0.0
PRM rerank, Qwen2.5-Math-PRM-7B
N=10 , last step
1.040e15
29.1
0.0
Recursive Refinement
N=8,k=4,T=2
1.048e15
17.9
5.4
TTT-Consolidate
K=8
1.050e15
35.4
23.6
Appendix
Table 5: Iso-compute comparison against reward-guided and recursive-refinement baselines. All methods run at TTT-Consolidate @8’s budget ( ≈ 1.05e15 pipeline FLOPs). Best-of-N and PRM reranking select from N=10 sampled solutions. Recursive Refinement runs T=2 rounds, each regenerating N=8 candidates from subsets of k=4 . Reranking scores 0.0 on None because it can only select an answer that is already in the pool, and every input there is wrong. TTT-Consolidate outperforms the strongest baseline on Minority and None .
Runtime
Memory
Accuracy
Method
Mode
Out. budget
Prompt
Consolidate
Decode
Total
Speed
Peak KV
Peak mem
Minority
None
tok
tok
s ↓
s ↓
s ↓
ms/tok ↓
GB ↓
GB ↓
% ↑
% ↑
Prompt agg
CoT
8,192
6,182
–
25.867
25.867
3.158
1.649
15.742
17.2
4.0
SFT agg
CoT
8,192
6,182
–
25.867
25.867
3.158
1.649
15.742
10.2
3.2
Prompt agg
Thinking
16,384
6,178
–
55.038
55.038
3.359
2.588
15.761
25.8
13.0
RL agg
Thinking
16,384
6,178
–
55.034
55.034
3.359
2.588
15.742
22.0
3.8
Appendix
Table 6: Inference runtime and memory usage on a single H100 GPU. Trajectory sampling is shared by all methods and excluded from runtime. Baselines decode from a ∼ 6.2K-token prompt of concatenated trajectories. TTT-Consolidate instead consolidates the trajectories into LoRA weights (1.9 s) and decodes from the 156-token problem alone. TTT-Consolidate has the lowest total latency and the smallest KV cache, while achieving the highest accuracy on both tiers. The cost is ∼ 6 GB higher peak memory for gradients and optimizer states.
Model
Trajc
SFT Agg
TTT-Consolidate
Gain
Qwen3-1.7B
None
3.2
23.6
+20.4
Qwen3-1.7B
Minority
10.2
35.4
+25.2
Qwen3-4B
None
7.7
40.8
+33.1
Qwen3-4B
Minority
15.4
57.6
+42.2
Appendix
Table 7: Comparing different consolidation methods trained with frontier supervision. SFT Aggregation (CoT) and TTT-Consolidate -SFT train on the same set of identical questions, trajectory groups, and frontier-generated reference solutions. TTT-Consolidate -SFT’s positive performance gain improves with model scale. Its improvement therefore comes from the meta-learned test-time training, not simply frontier supervision.
AIME26
Apex25
ArxivMath
HMMT2602
No aggregation at inference
SFT-1.7B
3.3
0
5.6
3.0
SFT-4B
14.2
0
11.1
6.1
With aggregation at inference
SFT-1.7B
0.8
0
0.7
1.5
SFT-4B
3.3
0
2.4
3.0
Appendix
Table 8: Outer-loop-only ablation. We remove the inner loop, reducing training to SFT on a single frontier-teacher CoT trace, ignoring sampled trajectories, evaluated with and without aggregation at inference. The outer loop alone is insufficient. SFT on frontier supervision does not match TTT-Consolidate ’s performance.
Backbone
Setting
AIME26
Apex25
ArxivMath
HMMT2602
Qwen3-1.7B (None)
Only inner
5.0
0
10.8
3.8
Inner + outer
26.7
16.7
23.6
18.2
Qwen3-1.7B (Minority)
Only inner
6.7
4.2
9.0
4.6
Inner + outer
30.0
25.0
34.7
27.3
Qwen3-4B (None)
Only inner
14.2
0
12.2
13.6
Inner + outer
39.1
33.3
37.9
48.5
Appendix
Table 9: Inner-loop-only ablation. We start each problem’s test-time update from an untrained LoRA initialization, without outer-loop meta-learning. Plain test-time training fails to consolidate trajectories, so both loops under the bilevel setup are needed.
Condition
Inner-loop input
Avg. acc.
Δ pre-adapt.
Δ post-adapt.
Isolate
Pre-adaptation
No inner-loop update
7.5%
–
−14.5
Initialization alone
Passive repetition
8 copies of the target question
5.4%
−2.1
−16.6
Repeated exposure
Task activation
Trajectory groups from other problems, same task distribution
4.0%
−3.5
−18.0
Task-distribution activation w/o target-specific info
Single-Trace × 8
One trajectory per LoRA, 8 LoRAs, majority vote
11.7%
+4.2
−10.3
Absence of cross-trajectory interaction
TTT-Consolidate
All 8 target trajectories, jointly
22.0%
+14.5
–
–
Appendix
Table 10: Isolating the source of TTT-Consolidate ’s improvement. Every ablation control falls far below full TTT-Consolidate . Our method’s performance gain requires jointly consolidating the target problem’s trajectories during adaptation.
AIME26
Apex25
ArxivMath
HMMT2602
GPQA
TTT-Consolidate -SFT
Before TTT
16.7
0
2.8
6.1
6.6
After TTT
26.7
16.7
23.6
18.2
22.7
Gain from adaptation
+10.0
+16.7
+20.8
+12.1
+16.1
TTT-Consolidate -RL
Before TTT
16.7
0
4.2
9.1
19.7
After TTT
23.5
16.7
26.0
21.7
40.0
Gain from TTT
+6.8
+16.7
+21.8
+12.6
+20.3
Appendix
Table 11: TTT-Consolidate model with and without test-time adaptation on four None tier benchmarks. Inner-loop adaptation improves both the SFT and RL models on every benchmark. Performance gains do not come from the initialization alone.
Table 12: Matched SFT → GRPO training path for RL Aggregation and TTT-Consolidate . Both methods get the same frontier-supervised initialization and GRPO budget. RL Aggregation stays behind; thus, RL alone does not explain TTT-Consolidate ’s improvement.
Benchmark
Domain
# Questions
Math Reasoning
AIME 2025
Competition Math
30
AIME 2026
Competition Math
30
Apex 2025
Competition Math
12
ArXiv Math
Research Math
72
CMIMC 2025
Competition Math
40
Appendix
Table 13: Summary of the ten reasoning benchmarks. Each question is used to generate 40 candidate trajectories (5 source models × 8 samples), from which three quality-stratified trajectory groups are formed.
Figure 9: Left : Distribution of extracted CoT lengths over reasoning trajectory collection. Right : Source model composition by quality tier, where weaker models dominate the None tier while stronger models contribute more in the Majority tier.
Component
Content
Judge Prompt
You are evaluating whether a model’s final solution to a problem reflects
{synthesis vs. artifact exploitation ∣ consolidation vs. memorization}.
Department of Electrical & Computer Engineering, University of Arizona · Future Science and Engineering College, Soochow University · INSAIT, Sofia University ”St. Kliment Ohridski” +1