cs.CLOct 30, 2026

TraceSpec: Understanding and Improving Speculative Decoding for LLM Agents

Authors: Jihao Zhao, Shuaishuai Zu, Pengfei Li, Biao Qin, Guanghuan Fang, Ben Feng, Baochuan Yang

Organizations: School of Information, Renmin University of China, Beijing, China · Huawei Technologies Co., Ltd., Beijing, China · Huawei Technologies Co., Ltd., Hangzhou, China

Abstract

LLM agents execute tasks through repeated model calls over evolving histories of reasoning, actions, and tool feedback. Aggregate speculative-decoding metrics merge these distinct decisions, providing limited guidance on where draft supervision is needed. We present TraceSpec, a framework connecting trajectory evaluation with adaptive draft training. It combines dynamic execution with fixed-prefix replay and introduces TraceSpec-Traj, a corpus of 10,000 training states and 2,500 test states spanning browsing and software engineering. Analysis of five speculative baselines reveals that aggregate acceptance can rise with context length while declining within matched late browsing actions. Workload and phase further distinguish these trends, motivating supervision guided by trajectory conditions. TraceSpec trains on intermediate assistant turns and allocates a fixed prediction budget using state-conditioned estimates of prefix-aware acceptance gaps. Across three target sizes and two temperatures, it improves replay throughput by 12.5% to 22.6% over its DSpark base. Qwen3-4B ablations show that adaptive allocation increases accepted length by 3.3% to 10.0% over fixed full-turn supervision, with additional comparisons substantiating the roles of state conditioning and consecutive-token agreement. The resulting evaluation resources, trajectory findings, and training strategy connect the analysis of agent execution to practical draft improvement.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. AgentSpec: Speculative Decoding for Batch Inference of LLM Agents

    Aug 25, 2026Xin Wang, Ziming Miao, Yi Zhu +4Speculative DecodingLLM Inference Acceleration

  2. Speculate with Memory: Lossless Acceleration for LLM Agents

    Jul 14, 2026Yu Li, Qinyuan Ye, Prafulla Kumar Choubey +2Next-Action PredictionLLM Agent Memory

  3. LEAP: Learning Efficient Action Proposals For LLM Agents

    Oct 2, 2026Zhen Xu, Qizheng Zhang, Gerry Wan +2LLM AgentsLLM Agent Training