Lost in Delegation: Studying the Effectiveness of Task Orchestration in Generative Search
Organizations: Ruhr University Bochum and UA Ruhr Research Center for Trustworthy AI
Abstract
LLMs have quickly evolved from models used in isolation to agentic systems that delegate tasks to external tools and other LLMs to solve complex assignments. This agentic augmentation has led to impressive capability gains. However, when and how delegation improves performance remains poorly understood. For example, it is unclear whether delegation helps when tasks involve post-cutoff information that the orchestrating LLM cannot reliably resolve from its own knowledge. In this work, we use purpose-built datasets with pre- and post-cutoff splits to compare various delegation modes. We find that delegation provides no consistent accuracy improvement over non-delegated setups, with effects varying across models and tasks. When tasks involve knowledge unavailable to the orchestrator, delegation can substantially degrade accuracy, while direct tool use with the same LLM performs much better. Our analysis identifies distorted delegated queries, inefficient tool use, and failures to preserve and use information between agents as recurring failure modes. These findings suggest that effective orchestration requires both well-calibrated delegation decisions and effective exchange of information between agents.
Figures & tables
| Setup | Pre-cutoff | Post-cutoff | Change |
|---|---|---|---|
| No Tools | 0.53 | 0.01 | \hlsecondarytab 98% |
| Single Agent | 0.77 | 0.75 | \hlsecondarytab 3% |
| Optional Del. | 0.74 | 0.64 | \hlsecondarytab 14% |
| Forced Del. | 0.77 | 0.66 | \hlsecondarytab 13% |
| Pre-cutoff | Post-cutoff | ||||
|---|---|---|---|---|---|
| Model | Setting | Medium | Author | Medium | Author |
| GPT-OSS | Single agent | 58 | 80 | 28 \hlsecondarytab 52% | 100 \hlprimarytab 25% |
| Forced | 58 | 72 | 21 \hlsecondarytab 63% | 0 \hlsecondarytab 100% | |
| Optional | 51 | 71 | 16 \hlsecondarytab 69% | – | |
| Qwen | Single agent | 82 | 92 | – | – |
| Forced | 69 | 69 | 22 \hlsecondarytab 69% | – | |
| Model | GPT-OSS | Qwen | GPT-5 | Claude |
|---|---|---|---|---|
| Single agent | 0.5 | 0.3 | 1.3 | 1.5 |
| Forced | 2.7 | 0.0 | 0.9 | 0.6 |
| Optional | 4.5 | 0.0 | 3.2 | 0.4 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Provider | Identifier | Cutoff | |
|---|---|---|---|---|
| Closed source | Claude | Anthropic | claude-sonnet-4-5 | Jan 2025 1 1 1 https://platform.claude.com/docs/en/models/sonnet-4-5/overview |
| GPT-5 | OpenAI | gpt-5 | Sep 30, 2024 2 2 2 https://developers.openai.com/api/docs/models/gpt-5 | |
| Open weight | GPT-OSS | OpenAI | openai/gpt-oss-120b | Jun 1, 2024 3 3 3 https://developers.openai.com/api/docs/models/gpt-oss-120b |
| Qwen | Qwen | Qwen/Qwen3-30B-A3B- Thinking-2507 | Jul 31, 2025 4 4 4 https://github.com/QwenLM/Qwen3 |
| Dataset | Source | Years (Pre / Post) | # Prompts (Pre / Post) | Items / Prompt |
|---|---|---|---|---|
| Books | Goodreads Top 100 [ Goodreads, ] | 2014–2019 / 2026 | 120 / 20 | 5 |
| Nobel | Nobel Prize API [ Nobel Prize Outreach, ] | 2014–2019 / 2025–2026 | 34 / 12 | 1–3 |
| GDP | World Bank API [ World Bank, ] | 2019 / 2025 | 30 / 30 | 4 |
| Products | iF Design Award [ iF International Forum Design GmbH, ] | 2021 / 2026 | 30 / 30 | 4 |
| Single Agent | Delegation (Forced / Optional) | |||
|---|---|---|---|---|
| Dataset | Budget Items ( ) | Search / Open URL | Subagent Calls | Search / Open URL per Subagent |
| Books | 5 | 10 / 15 | 10 | 2 / 3 |
| Nobel | 3 | 6 / 9 | 6 | 2 / 3 |
| GDP | 4 | 8 / 12 | 8 | 2 / 3 |
| Products | 4 | 8 / 12 | 8 | 2 / 3 |
| Condition / Role | System instruction |
|---|---|
| No tools | You are a helpful assistant. |
| Single-agent | You are a helpful assistant with access to a web_search and open_url tool. If you do not know the answer to a question, use the web_search tool. |
| Forced-delegation orchestrator | You are a helpful assistant. Your task is to break the problem down into atomic subproblems and combine the answers into a final solution. Do not provide the answers to the subproblems yourself, but use the tool call_subagent() to delegate each subproblem to a subagent which will answer the subproblem for you. The subagent specializes in answering factual questions. Make sure each subproblem is self-contained. |
| Optional-delegation orchestrator | You are a helpful assistant. Your task is to break the problem down into atomic subproblems and combine the answers into a final solution. If you know the answer to a subproblem, provide the answer yourself without calling the subagent. If you do not know the answer, use the tool call_subagent() to delegate each subproblem to a subagent which will answer the subproblem for you. The subagent specializes in answering factual questions. Make sure each subproblem is self-contained. |
| Executor | You are a helpful assistant with access to a web_search and open_url tool. Use them judiciously and only call a tool if you are not sure of the answer. If you know the answer and think the tool will not provide any additional information, do not use it. |
Tool-Augmented Language Model Agents
Tool-augmented language model agents use external tools, APIs, and environments to solve tasks beyond text generation alone.
See all 175 papers