Incremental Open-Ended Deep Research with Structured Harness
Organizations: Xiaohongshu Inc. · Zhejiang University
Abstract
Existing Open-Ended Deep Research (OEDR) systems primarily generate reports from scratch, making them inefficient for scenarios where research reports need to be continuously maintained as new information emerges. We introduce \textbf{Incremental Open-Ended Deep Research (Incremental-OEDR)}, a research setting that treats a report as an evolving research state and incrementally updates it by preserving valid knowledge, revising outdated or incomplete content, and incorporating newly available information. To support this setting, we propose \textbf{Structured Harness}, which represents reports as structured collections of outlines, sections, and supporting evidence, and provides structured retrieval, a persistent structured evidence pool, and structured generation for selective report updating and evidence reuse. We further establish a temporal evaluation framework spanning ten years, with \emph{Single-Step Task} and \emph{Long-Chain Task} to evaluate incremental updates over both individual transitions and long-term update chains. Extensive Experiments on DeepResearch Bench and DeepConsult under both the Open-source Configuration (OC) and Proprietary Configuration (PC) show that Incremental-OEDR maintains competitive report quality while substantially improving report continuity and reducing research costs. As shown in Figure~\ref{fig:profile}, it achieves up to 0.51 higher content-level ROUGE-L F1, 0.63 higher outline-level EM F1, 33% lower token consumption, and 61% fewer search calls than OEDR on DeepResearch Bench. For more details, please refer to our project page: https://ioedr-project.github.io/.
Figures & tables
| Agent Systems | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | Avg. | |
| OEDR | Doubao-DR ∗ | – | – | – | – | – | – | – | – | – | 46.62 | – |
| Kimi-DR ∗ | – | – | – | – | – | – | – | – | – | 46.50 | – | |
| WebWeaver ∗ | – | – | – | – | – | – | – | – | – | 45.20 | – | |
| WebShaper ∗ | – | – | – | – | – | – | – | – | – | 40.28 | – | |
| DualGraph ∗ | – | – | – | – | – | – | – | – | – | 51.50 | – | |
| Gemini-2.5-Pro (DR) † | – | – | – | – | – | – | – | – | – | 49.98 | – |
| Agent Systems | 2016 | 2017 | 2018 | 2019 | 2020 | 2021 | 2022 | 2023 | 2024 | 2025 | Avg. | |
| OEDR | Claude-Research ∗ | – | – | – | – | – | – | – | – | – | 4.60 | – |
| OpenAI-DR ∗ | – | – | – | – | – | – | – | – | – | 5.00 | – | |
| Doubao-Research ∗ | – | – | – | – | – | – | – | – | – | 5.42 | – | |
| Gemini-2.5-Pro-DR ∗ | – | – | – | – | – | – | – | – | – | 6.70 | – | |
| WebShaper ∗ | – | – | – | – | – | – | – | – | – | 3.71 | – | |
| WebWeaver ∗ | – | – | – | – | – | – | – | – | – | 5.65 | – |
| Agent Systems | Report Continuity | Research Cost | |||||||
| Content Level | Outline Level | Tokens(M) | Searches | ||||||
| ROUGE-L F1 | BLEU-4 | ROUGE-L F1 | EM F1 | In | Out | Total | |||
| DeepResearch Bench | |||||||||
| OEDR | MS-Agent(OC) | 0.17 | 0.13 | 0.28 | 0.04 | 3.67 | 0.11 | 3.78 | 45.19 |
| MS-Agent(PC) | 0.15 | 0.09 | 0.30 | 0.05 | 6.85 | 0.15 | 7.00 | 104.88 | |
| SST | IOEDR(OC) | 0.57 | 0.58 | 0.66 | 0.60 | 2.90 | 0.06 | 2.96 | 18.62 |
| Interval | Quality | Continuity | Cost |
| 10 years | 0.984 | 1.951 | 1.446 |
| 5 years | 1.002 | 2.839 | 0.972 |
| 1 year | 1.007 | 3.353 | 0.783 |
| 1 month | 1.004 | 3.360 | 0.694 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Category | Tool | Function | Agents | |
| General | File System | write_file | Write files to the working directory | R, S, P |
| read_file | Read files from the working directory | R, S, P | ||
| list_files | List files in the working directory | R, S, P | ||
| search_file_content | Search for content within files | R | ||
| replace_file_contents | Replace specified file contents | R, P | ||
| replace_file_lines | Replace specified lines in files | R, P |