Large language model-based deep research agents have been increasingly popular for addressing long-horizon information-seeking tasks, but they often incur high end-to-end latency due to extensive reasoning and frequent tool use. Speculation frameworks aim to reduce latency by overlapping action execution with reasoning; however, existing approaches typically rely on uniform speculation strategies and strict action matching, which limits inference speedups and robustness. In this work, we revisit the speculate-verify paradigm for deep research agents through the lens of action heterogeneity. We show that Search and Visit actions exhibit fundamentally different reasoning and model capacity requirements: entropy-based analysis reveals that Search decisions have higher uncertainty and benefit significantly from explicit reasoning, whereas Visit decisions have lower entropy and depend primarily on model capacity. Motivated by this dual-process characteristic, we propose DualSpec, a heterogeneous speculation framework equipped with a lightweight, confidence-based semantic verifier. Experiments across multiple models and benchmarks demonstrate that DualSpec achieves up to 3.28× end-to-end speedup while maintaining accuracy comparable to fully reasoning agents.
Figures & tables
Figure 1: Deep research agent workflow. Deep research agents follow a Reason-Action-Observation loop, where the agent alternates between generating reasoning traces and executing actions (Search or Visit) to gather information.
Figure 2: Deep research inference characteristics using different models. ‘‘Miro’’ denotes ‘‘MiroThinker’’ while ‘‘Qwen’’ denotes ‘‘Qwen-3’’. (a) Tool usage ratio on the GAIA benchmark 2 2 2 Unless otherwise specified, all micro-level analyses are conducted on the GAIA benchmark; consistent trends are observed across other benchmarks. . (b) Time breakdown per step on model reasoning and tool execution. Model reasoning accounts for a significant fraction of the total latency.
Figure 3: Average reasoning length for generating Search and Visit actions across models and benchmarks. Search requires significantly longer reasoning than Visit.
Figure 4: Action alignment comparison of two speculative methods relative to the Oracle (large model with reasoning) when drafting Search and Visit. (a) The small reasoning model produces queries more aligned with the Oracle than the large model without reasoning. (b–c) For Visit, the large model skipping reasoning achieves higher accuracy in both URL selection and extraction instruction.
Configuration
MiroThinker
Qwen3
LLM with Reasoning
63.11
29.13
SLM with Reasoning for Search
63.11
29.13
LLM without Reasoning for Search
57.28
23.30
SLM without Reasoning for Search
55.34
22.33
SLM with Reasoning for Visit
56.31
22.33
LLM without Reasoning for Visit
64.08
28.16
Table 1: End-to-end agent performance (pass@1 accuracy) on GAIA under action-level replacements. We selectively replace the generation strategy for Search or Visit using lightweight methods.
Figure 5: Action log probability distributions with and without reasoning. A higher log probability indicates lower uncertainty. Without reasoning (dark blue color), Search actions exhibit lower log probabilities than Visit actions, indicating higher baseline decision uncertainty. When reasoning is incorporated (light blue color with hatching), both action types see increased log probabilities, but the increase is significantly larger for Search actions, reflecting a greater reduction in uncertainty due to reasoning.
Figure 6: Overview of DualSpec.
Figure 7: Comparison of the accuracy (pass@1) and latency of different schemes across model combinations. DualSpec consistently reduces end-to-end latency by 1.33–3.28 × ( ∼ 2 × on average) over the base model while maintaining comparable accuracy. Compared with DSP and SPAgent, DualSpec achieves a better accuracy–latency trade-off across all datasets and model pairs.
Models
Datasets
Speculation
Acc
Lat
Miro 72B+8B
GAIA
Origin
63.1
1041
LLM w/o Reason
59.2
575
SLM w/ Reason
56.3
651
Heterogeneous
63.1
605
Xbench
Origin
66
1007
LLM w/o Reason
65
492
Table 2: Performance under different speculation schemes.
Figure 8: Accuracy (pass@1) as a function of the reasoning intervention rate under a fixed drafting policy.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Verifier score distributions on GAIA and XBench-DeepResearch. We report trajectory-level aggregated verifier scores for (a) MiroThinker mean score, (b) Qwen mean score, (c) MiroThinker p25 score, and (d) Qwen p25 score. Correct trajectories consistently receive higher scores than incorrect ones, suggesting that the verifier provides a useful reliability signal.
LLM agents are known to be slow in rollouts. An agent completes a task one step at a time. At each step, it reasons and then chooses an action to execute. The next step and action cannot start until the previous one has finished. Speculative decoding accelerates the rollouts at the reason phase by drafting and verifying the inference tokens. Recent works have also started to apply similar ideas at the action phase. These works use off-the-shelf models, usually large, to draft action proposals for target model to verify. Large drafters match the target more often but take longer to propose, while small off-the-shelf models are fast but rarely make the same decision as the target. We ask a more general question: what determines the end-to-end speedup of action speculation? To answer it, we develop a latency framework for the speculative round. The framework compares what a round gains with what it costs. The gain depends on how well the drafter predicts the target and on how many steps the task can take before it ends. The cost comes from drafting, from waiting for target verification and from executing tools. Guided by the framework, we introduce LEAP (Learning Efficient Action Proposals) which keeps the drafter small and makes it accurate by training it on the target actions sequences. With a small 0.6B model, LEAP agrees with the target on most decisions and makes agents up to 60% faster in end-to-end wall clock time, with no systematic change in task success. Across various datasets, target models and draft models, the framework accounts for most of the measured speedups. We also show the draft model can be online trained with no prior trace collection and match the performance of offline training, making LEAP practical to deploy in the real world.
Zhen Xu, Qizheng Zhang, Gerry Wan +2
University of Chicago · Stanford University · Together AI
Large language model agents increasingly act through stateful tools, yet model generation and environment execution remain serialized at every step. As decoding accelerates, tool execution becomes a growing bottleneck. Existing action- or observation-only speculation leaves much of this latency exposed: value is concentrated in a few slow calls, some outcomes emerge only through execution, and longer lookahead typically requires an increasingly unlikely chain of action predictions. We present AOSpec, a lossless framework that co-speculates actions and observations across the full agent-environment loop. Expected Value Decoding (EVD) directs observation speculation toward outcomes with the greatest expected latency benefit, optimizing expected time hidden rather than hit rate. For outcomes only execution can reveal, AOSpec launches latency-critical target actions in isolated forks that contain their effects, while Joint Action-State Verification (JASV) verifies both the action and its origin state against committed execution before reuse. JASV recasts long-horizon action dependency from full-chain prediction into target action-state verification, breaking the lookahead--accuracy tradeoff and unlocking long-range overlap without sacrificing serial semantics. Across Terminal-Bench serving settings spanning four harnesses, five actor models, and five serving speeds, AOSpec outperforms every practical baseline, reducing mean end-to-end latency by 11.8-32.5% and p99 latency by up to 42.8%. Its gains increase as decoding accelerates, and its observation model transfers from Terminal-Bench to SWE-bench Verified without retraining.
Speculative execution accelerates LLM agents by using a smaller, cheaper model to predict and pre-launch the next step while the environment is idle. However, existing speculators are stateless and discard all information between tasks, preventing prediction quality from improving with experience. We equip the speculator with three online memory systems that learn from past agent trajectories: a contrastive transition table tracking action-sequence statistics, an episodic memory retrieving contextually similar segments, and a confusion tracker suppressing recurring errors. We evaluate this approach on six benchmarks spanning three speculation types: action prediction, observation prediction, and chained prediction. Memory-augmented speculation yields a 19--39% relative accuracy improvement on action prediction and up to a 2.5× increase on observation prediction tasks with repetitive action spaces. These gains grow continuously as memory accumulates and generalize across speculator models of varying cost. All speculation is lossless because it runs during idle time at zero added wall-clock cost, and the actor's trajectory is identical to non-speculative execution.