TraceSpec: Understanding and Improving Speculative Decoding for LLM Agents
Authors: Jihao Zhao, Shuaishuai Zu, Pengfei Li, Biao Qin, Guanghuan Fang, Ben Feng, Baochuan Yang
Organizations: School of Information, Renmin University of China, Beijing, China · Huawei Technologies Co., Ltd., Beijing, China · Huawei Technologies Co., Ltd., Hangzhou, China
LLM agents execute tasks through repeated model calls over evolving histories of reasoning, actions, and tool feedback. Aggregate speculative-decoding metrics merge these distinct decisions, providing limited guidance on where draft supervision is needed. We present TraceSpec, a framework connecting trajectory evaluation with adaptive draft training. It combines dynamic execution with fixed-prefix replay and introduces TraceSpec-Traj, a corpus of 10,000 training states and 2,500 test states spanning browsing and software engineering. Analysis of five speculative baselines reveals that aggregate acceptance can rise with context length while declining within matched late browsing actions. Workload and phase further distinguish these trends, motivating supervision guided by trajectory conditions. TraceSpec trains on intermediate assistant turns and allocates a fixed prediction budget using state-conditioned estimates of prefix-aware acceptance gaps. Across three target sizes and two temperatures, it improves replay throughput by 12.5% to 22.6% over its DSpark base. Qwen3-4B ablations show that adaptive allocation increases accepted length by 3.3% to 10.0% over fixed full-turn supervision, with additional comparisons substantiating the roles of state conditioning and consecutive-token agreement. The resulting evaluation resources, trajectory findings, and training strategy connect the analysis of agent execution to practical draft improvement.
Figures & tables
Figure 1. TraceSpec overview. (a) Trajectory-state construction, coverage, and source-task splits. (b) Dynamic execution and fixed-prefix replay with trajectory diagnosis. (c) Full-turn supervision with state-conditioned allocation and event-normalized sampling under a fixed prediction budget. Three panels connect trajectory data, evaluation, and adaptive draft training. Panel a constructs history and continuation pairs across two workloads, five context strata, and four phases, with source tasks split into training and test partitions. Panel b shows dynamic draft verification by frozen Qwen3, tool feedback updating the history, and five baseline drafters evaluated on identical saved test prefixes. Trajectory diagnostics summarize acceptance, throughput, and episode latency. Panel c expands target-generated training trajectories into assistant segments. Frozen Qwen3 outputs and features supervise a trainable DSpark-based draft. Token overlap contributes to acceptance gaps smoothed by context, phase, workload, and tool state. Base coverage and adaptive priority determine supervision shares, converted to segment sampling probabilities using expected prediction events per visit. Training feedback updates allocation, and the resulting draft returns to evaluation.
Corpus property
Training
Test
Trajectory states
10,000
2,500
Source tasks
5,795
1,380
Web research states
5,000
1,250
Software engineering states
5,000
1,250
Plan
2,098
501
Act
3,151
920
Table 1. Trajectory corpus statistics. Source tasks are disjoint across splits.
Figure 2. Baseline replay acceptance by context length. Qwen3-8B on TraceSpec-Traj, grouped by domain, temperature, and actual input length. All methods receive the same prefixes. Four line charts show accepted length across context intervals for five draft methods in Web research and software engineering at temperatures zero and one.
Figure 3. Baseline acceptance across workloads. Columns show dynamic execution on BrowseComp-Plus and SWE-bench Verified, followed by fixed-prefix replay on TraceSpec-Traj. Groups denote Qwen3-4B, 8B, and 14B; rows denote T=0 and T=1 . Error bars show pointwise 95% bootstrap intervals. Six grouped bar charts in two rows and three columns compare five speculative baselines across three workloads, two temperatures, and three target sizes. Among these baselines, DSpark has the highest overall accepted length in all 18 settings. Accepted length decreases from temperature zero to one for every matched baseline and target.
Workload
Target
DFlash
EAGLE-3
DSpark
Gain (%)
BC
4B
89.17
87.16
82.86
7.1
BC
8B
73.18
77.27
68.14
6.9
BC
14B
147.08
167.77
143.15
2.7
SWE
4B
63.26
63.15
59.96
5.2
SWE
8B
80.12
84.13
75.90
5.3
SWE
14B
95.60
107.77
91.90
3.9
Table 2. Mean episode latency (s), T=0 . BC: BrowseComp-Plus; SWE: SWE-bench Verified. Gain is the latency reduction of DSpark over DFlash.
Figure 4. Online baseline acceptance and throughput by context length. Qwen3-8B at T=0 : upper panels show accepted tokens per round; lower panels show output tokens/s, including prefill. Four line charts compare DFlash, EAGLE-3, and DSpark on two online workloads. Browsing acceptance rises across the five intervals. Software engineering acceptance peaks at 8k to 16k before falling at 16k to 32k. Throughput is lower in the longest than in the shortest interval in both workloads.
Workload
Drafter
L8:16
L16:32
95% CI of ΔL
BC
DFlash
5.661
4.326
[-1.650, -0.941]
BC
EAGLE-3
6.007
4.897
[-1.489, -0.612]
BC
DSpark
6.194
5.322
[-1.205, -0.473]
SWE
DFlash
5.333
4.762
[-0.852, -0.257]
SWE
EAGLE-3
6.149
6.304
[-0.347, 0.716]
SWE
DSpark
6.057
6.106
[-0.238, 0.379]
Table 3. Late-action acceptance: Qwen3-8B, T=0 , step ≥20 . Tasks are paired across 8k to 16k and 16k to 32k contexts; ΔL=L16:32−L8:16 .
Figure 5. Baseline acceptance by domain and phase. Qwen3-8B results on TraceSpec-Traj at T=0 and T=1 , measured in accepted tokens per round. All panels share a common color scale. Four heatmaps compare five baseline drafters across planning, action, reflection, and answer generation. DSpark has the highest accepted length in every displayed phase. At temperature zero, its highest-acceptance phases are Web research actions and software engineering reflections.
Target
Phase
BrowseComp-Plus
SWE-bench Verified
EAGLE-3
DFlash
LongSpark
DLite
DSpark
TraceSpec
EAGLE-3
DFlash
LongSpark
DLite
DSpark
TraceSpec
Qwen3-4B
Plan
3.674
3.581
3.112
2.804
4.031
4.161
4.058
3.972
3.109
2.661
4.407
4.456
Act
4.605
4.072
3.260
3.281
4.617
4.923
5.113
4.491
3.710
2.638
5.116
5.467
Reflect
4.658
4.112
3.064
3.060
4.779
5.123
5.118
4.624
2.956
2.665
5.258
5.402
Answer
3.910
3.463
4.186
3.261
3.980
4.279
3.687
3.356
3.181
2.592
3.901
4.295
Qwen3-8B
Plan
3.946
3.922
3.373
3.035
4.456
4.570
3.271
3.190
2.840
2.614
3.562
3.656
Table 4. Phase-level dynamic acceptance at T=0 (accepted tokens per round, L ). Bold marks the highest value per phase and workload. TraceSpec uses adaptive full-turn supervision.
Target
Phase
TraceSpec-Traj, T=0
TraceSpec-Traj, T=1
EAGLE-3
DFlash
LongSpark
DLite
DSpark
TraceSpec
EAGLE-3
DFlash
LongSpark
DLite
DSpark
TraceSpec
Qwen3-4B
Plan
3.899
3.585
3.611
2.618
4.161
4.856
2.972
2.756
2.863
2.470
3.639
4.218
Act
3.704
3.419
2.788
2.549
3.953
4.909
3.316
2.892
2.824
2.471
3.789
4.695
Reflect
3.777
3.439
3.252
2.651
4.092
4.859
3.303
2.810
3.004
2.499
3.752
4.669
Answer
3.089
2.803
3.013
2.578
3.294
4.126
2.788
2.507
2.881
2.488
3.248
3.993
Qwen3-8B
Plan
3.156
3.345
3.041
2.610
3.808
4.397
2.755
2.925
2.912
2.553
3.620
4.060
Table 5. Replay acceptance on TraceSpec-Traj ( L ). All methods use identical saved prefixes. Column groups separate temperatures; rows and highlighting follow Table 4 .
Training variant
BrowseComp-Plus
SWE-bench Verified
TraceSpec-Traj
T=0
T=1
T=0
T=1
T=0
T=1
TraceSpec
4.803
4.145
5.142
4.859
4.753
4.428
w/o intermediate-turn supervision ( A )
4.307
3.745
4.619
4.169
4.275
3.782
w/o adaptive budget allocation ( B )
4.473
3.904
4.849
4.417
4.602
4.169
w/o state-conditioned estimation ( C1 )
4.420
4.000
4.769
4.449
4.422
4.056
w/o prefix-aware scoring ( C2 )
4.510
4.055
4.855
4.439
4.598
4.220
Table 6. Qwen3-4B component ablations on accepted length L . A and B use fixed allocation; C1 and C2 each modify TraceSpec.
Drafter
BrowseComp-
SWE-bench
TraceSpec-
Plus
Verified
Traj
T=0
T=1
T=0
T=1
T=0
T=1
Verification yield R (%)
DFlash
50.6
40.3
54.9
45.9
42.7
34.8
EAGLE-3
56.9
48.1
61.3
53.0
47.8
40.3
LongSpark
40.5
38.4
43.0
37.0
40.4
36.3
Table 7. Qwen3-4B inference results. Throughput includes prefill and uses native PyTorch methods. Bold marks column maxima within each metric.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Source
Training
Test
DeepResearch-traj ( IPF, 2026 )
5,000
1,250
Nebius SWE-agent ( Nebius, 2024 )
3,806
972
R2E-Gym SFT ( R2E-Gym, 2025a )
1,031
243
R2E-Gym testing agent ( R2E-Gym, 2025b )
163
35
Appendix
Table 8. Source composition of the constructed trajectory corpus.
Target
Phase
BrowseComp-Plus
SWE-bench Verified
EAGLE-3
DFlash
LongSpark
DLite
DSpark
TraceSpec
EAGLE-3
DFlash
LongSpark
DLite
DSpark
TraceSpec
Qwen3-4B
Plan
3.168
2.813
3.072
2.528
3.471
3.614
3.454
3.247
2.800
2.459
3.934
3.987
Act
3.878
3.263
3.074
2.554
4.146
4.228
4.423
3.800
3.069
2.510
4.841
5.246
Reflect
4.032
3.194
2.894
2.532
4.160
4.361
4.592
3.847
2.664
2.540
4.845
5.054
Answer
3.620
3.172
3.736
2.561
4.200
4.289
2.949
2.647
3.034
2.505
3.426
3.713
Qwen3-8B
Plan
3.036
3.021
3.175
2.790
3.639
3.583
2.495
2.556
2.539
2.397
3.016
3.044
Appendix
Table 9. Dynamic acceptance at T=1 . Metric and highlighting follow Table 4 .
Target
Drafter
BrowseComp-Plus
SWE-bench Verified
TraceSpec-Traj
T=0
T=1
T=0
T=1
T=0
T=1
Qwen3-8B
DFlash
48.9
37.1
50.4
37.2
44.1
36.6
EAGLE-3
49.9
38.1
54.0
40.2
42.5
35.9
LongSpark
39.2
36.5
39.6
35.5
39.2
35.9
DLite
36.3
33.1
33.3
31.1
33.2
32.0
DSpark
54.9
45.0
56.9
47.2
50.3
45.9
Appendix
Table 10. Verification yield R (%) for Qwen3-8B and 14B, extending Table 7 . Bold marks column maxima within each target.
Target
Drafter
BrowseComp-Plus
SWE-bench Verified
TraceSpec-Traj
T=0
T=1
T=0
T=1
T=0
T=1
Qwen3-8B
DFlash
42.96
51.35
48.95
49.18
37.02
31.07
EAGLE-3
40.33
41.20
45.98
41.32
32.68
27.81
DLite
34.51
44.86
37.19
35.99
30.69
28.45
DSpark
46.79
54.94
52.63
51.93
40.44
37.06
TraceSpec
48.25
53.92
54.30
55.12
47.14
41.70
Appendix
Table 11. Generation throughput (output tokens/s) for Qwen3-8B and 14B. Timing scope and highlighting follow Table 7 .