cs.HCSep 6, 2026
SaveFrankenReport: Early Exiting in Long-Form Generation Using Expected Value of Computation
Organizations: Johns Hopkins University · Microsoft
Abstract
While deep research systems address interactive information-seeking needs impressively, their real-world deployments face latency and resource-consumption challenges. We present FrankenReport, an interface for long-form knowledge-seeking report generation that supports adaptive early exiting per section: it evaluates intermediate outputs during generation and predicts whether further targeted computation will yield significant quality gains. In a simulation study, FrankenReport outperforms random allocation baselines by a large margin (up to 4x) under low budgets and smoothly recovers full-pipeline quality as the budget grows, showing that future quality gains are predictable from intermediate drafts. Through experiments and user studies, we further show that despite varying preferences across users and topics, FrankenReport adapts to simple, natural user feedback as efficiently as methods requiring much costlier supervision such as generated drafts and explicit rationales.
Figures & tables
| Case | Low Effort | High Effort | Action |
| Jimmy Zavala | Insufficient information available regarding early life and career details… | Born July 12, 1955 . Influenced by Junior Walker and King Curtis , shaping his style… | Run (Knowledge) |
| Sahara Desert | The Sahara Desert is the world’s largest hot desert, covering approx 9.2M sq km across North Africa… | The Sahara is the largest hot desert… spanning approx 3000 miles East to West across 11 countries… | Stop (Marginal) |
| GlycoBlue | GlycoBlue acts as a PCR additive to improve yield and specificity of reactions… | It enhances the visibility of DNA bands during electrophoresis and stabilizes the mixture… | Stop (Unhelpful) |
Table 1: More computational effort does not guarantee better results, as illustrated in the “low effort” and “high effort” columns. Gray texts add little value, Red are factually wrong and Blue are additional accurate facts. FrankenReport predicts the quality gain from additional computational effort and takes actions accordingly, only expending effort on cases with a high estimated quality gain.
Figure 1: Mean wall-clock timing for a single request. The top pair shows the original STORM pipeline and its late Sketch/Verify checkpoints; the bottom pair shows gist-primed planning and its earlier checkpoints ( § 3 ). Within each pair, the thin checkpoint row shares the timeline of the pipeline row above it. Sections execute in parallel after the shared planning work.
Figure 2: System overview of FrankenReport . User queries are decomposed into section-level plans ( gist ), with sections generated in parallel. Conditioned on early drafts, a trained action predictor enables adaptive stopping to optimize resource allocation. Partial outputs are displayed in real-time, allowing users to select preferred drafts for the final report while background refinements continue.
| CORE | FS | FW | LF | WH | ||
| #D | 200 | 100 | 100 | 100 | 500 | |
| #S | 3459 | 1761 | 1686 | 1749 | 8451 | |
| S/D | 17.3 | 17.6 | 16.9 | 17.5 | 16.9 | |
| T/S | Ske | 390.7 | 297.9 | 409.6 | 441.3 | 400.0 |
| Ver | 439.5 | 414.4 | 451.4 | 475.8 | 464.7 | |
| Exp | 999.4 | 882.0 | 1026.7 | 1175.8 | 1110.1 |
Table 2: Per-document (D), section (S) and token (T) statistics across drafts. A/B is “number of A per B”.
| Metric | Range | Ske | Ver | Exp |
| Coherence | (1-5) | |||
| Engagingness | (1-3) | |||
| Organization | (1-5) | |||
| Informativity | ( 0) | |||
| Factuality | (0-1) |
Table 3: Mean standard deviation of quality metrics for , where Sketch, Verify, Expand , across sections.
Figure 3: Total quality gain as the normalized compute budget increases. We compare our learned allocator with random-greedy, random-uniform, shortest-first, and zero-shot GPT-5-mini score routing. Shading is the 95% nonparametric bootstrap interval for our deterministic policy (2,000 section-level resamples).
| Aspect | Target Conditioning | MSE | ||
| Coherence | V Q | .353 | .170 | .670 |
| S Q | .307 | .072 | .724 | |
| V S | .746 | .326 | .371 | |
| E S | .697 | .234 | .415 | |
| E V | .302 | .304 | .339 | |
| E S, V | .071 | .070 | .509 |
Table 4: Predictor performance across dimensions. In each target conditioning label, the left side denotes the improvement target and the right side denotes the predictor input. { S , V , E } stands for {Sketch, Verify, Expand} drafts. Q stands for user query.
Figure 4: Online learning to adapt the orchestration policy to simulated user preference.
Figure 5: Preferences change with topic familiarity.
| Acc | Acc (binary) | |
| Majority | .406 | .716 |
| GPT-5-mini | .649 | .811 |
| +familiarity | .640 | .815 |
| GPT-5.1 | .676 | .820 |
Table 5: Profiling and user preference prediction results. We evaluate draft selection accuracy and “Needs Improvement” prediction (binary).
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Overview of the FrankenReport interface that surfaces key controls and status indicators to the user. The initial draft uses a retrieval-informed gist but omits section-level evidence retrieval.
Figure 7: Draft-selection dropdown in the side panel, used to inspect alternative generations before confirming an exit decision.
Figure 8: Prompt used for coherence scoring.
Figure 9: Prompt used for organization scoring.
Figure 10: Prompt used for engagingness scoring.
Figure 11: Prompt used for user preference summarization.
Figure 12: Prompt used for preference-conditioned option selection.
| Metric | Ske | Ver | Exp |
| Coherence (1–5) | 4.930 | 4.982 | 4.739 |
| Engagingness (1–3) | 2.888 | 2.931 | 2.948 |
| Organization (1–5) | 4.889 | 4.993 | 4.986 |
| Informativity ( 0) | .831 | .871 | 1.020 |
Table 6: Quality metrics with GPT-4.1 as the generation model, averaged across all test queries. Factuality is omitted because it requires search-backed evaluation.
Figure 13: Budget allocation for engagingness and factuality with the same four baselines as Figure 3 . Shading is the 95% section-level bootstrap interval for our deterministic policy.
Figure 14: Sample-efficiency results for factuality scoring (left) and user-preference adaptation under different information weights (right).
Figure 15: Screenshot of the user study interface for “Pitstone Green Museum” topic.
Figure 16: Average inter-annotator agreement as a function of annotator-reported confidence (left) and topic familiarity (right).
More in this topic
LLM Inference Efficiency
Large language model inference efficiency reduces the time, memory, energy, and cost of generating useful outputs.
See all 351 papers