Speculative Ad-hoc Querying
Organizations: The University of Texas at Austin · Amazon Web Services · Microsoft Research
Abstract
Large language models (LLMs) invite substantial effort and investment, yet agentic systems can spend much of their time waiting for tools. This paper explores how a small auxiliary LLM can reduce these waits by speculatively preparing a database while the primary LLM reasons. The challenge is that the auxiliary model must anticipate the primary model's actions before the primary model takes them. Our key insight is that database systems give the speculator a structural advantage: it need not predict the exact next query. Instead, it can prepare work shared by many plausible queries. Predicting query shapes enables advance compilation, predicting data access warms caches, and predicting broader data requirements enables precomputation of superset results that subsequent queries can refine. We present Wonder, a system that exploits this advantage for analytical SQL workloads. Wonder constructs reusable superset queries, materializes their components, and rewrites subsequent queries against the resulting temporary tables. Its scheduler prioritizes foreground queries while using reasoning intervals for speculative preparation. The same engine supports humans writing SQL, using incomplete queries to guide speculation and displaying intermediate results as they type. In a replay-based evaluation of agent workflows, Wonder reduces SQL execution time by up to 31.8% and combined primary-agent reasoning and active SQL time by up to 22.6%, with approximately 6% higher recorded main-agent and prediction API cost and 10% more read I/O on the full dataset. A separate 24-participant study finds significantly lower query latency on both analytical tasks and improved time-to-answer on one, with 87.5% of participants reporting increased productivity.
Figures & tables
| Per-task Metric | Scale | Strawman | Speculative Actions | Rule-based MV | SpeQL |
|---|---|---|---|---|---|
| User-perceived time (s) | Full | 1842.78 | 1839.44 (-0.18%) | 1733.02 (-5.96%) | 1426.85 (-22.57%) |
| 1/4 | 829.42 | 826.55 (-0.35%) | 773.83 (-6.70%) | 737.15 (-11.12%) | |
| 1/16 | 617.37 | 615.28 (-0.34%) | 604.48 (-2.09%) | 594.17 (-3.76%) | |
| SQL execution time (s) | Full | 1307.11 | 1303.77 (-0.26%) | 1197.35 (-8.40%) | 891.18 (-31.82%) |
| 1/4 | 293.75 | 290.88 (-0.98%) | 238.15 (-18.93%) | 201.48 (-31.41%) | |
| 1/16 | 81.70 | 79.60 (-2.56%) | 68.81 (-15.78%) | 58.50 (-28.40%) |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| 10/100/1000 GB | Median | Mean | Max |
|---|---|---|---|
| Lines in queries | 39 | 48.4 | 227 |
| Temp. tables | 8/7/7 | 10.5/10.1/8.4 | 52/52/44 |
| Previews | 13/13/11 | 13.5/13.0/11.0 | 21/21/21 |
| Edges | 37/36/30 | 49.6/48.9/39.6 | 285/285/273 |
| Total size (GB) | 2.0/2.8/3.6 | 5.1/7.5/9.6 | 33.0/39.0/75.9 |
| Task | Area | Private goal |
|---|---|---|
| 1 | Computer science | Shortlist promising research directions by comparing publication growth and citation concentration, rather than publication volume alone. |
| 2 | Neuroscience | Identify overlooked topics with relatively few publications but sustained citation attention. |
| 3 | Renewable energy | Determine whether publication growth is spread across many topics or driven by a few areas, such as solar cells or batteries. |
| 4 | Mathematics | Select topics for a graduate reading program that show sustained publication activity and citation attention rather than short-lived surges. |
| 5 | Quantum computing | Compare how broadly or narrowly research communities, defined by countries or institutions, distribute their work across topics. |
| 6 | Robotics | Investigate whether robotics research has become more connected to medicine or manufacturing, and whether the conclusion depends on the measure used. |
LLM Agent Workflow Optimization
LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.
See all 283 papers