OrgAgent: Organize Your Multi-Agent System like a Company

Yiru Wang; Xinyue Shen; Yaohui Han; Michael Backes; Pin-Yu Chen; Tsung-Yi Ho

cs.MAApr 1, 2026

OrgAgent: Organize Your Multi-Agent System like a Company

Authors: Yiru Wang, Xinyue Shen, Yaohui Han, Michael Backes, Pin-Yu Chen, Tsung-Yi Ho

Organizations: The Chinese University of Hong Kong · CISPA Helmholtz Center for Information Security · IBM Research

Abstract

While large language model-based multi-agent systems have shown strong potential for complex reasoning, how to effectively organize multiple agents remains an open question. In this paper, we introduce OrgAgent, a company-style hierarchical multi-agent framework that separates collaboration into governance, execution, and compliance layers. OrgAgent decomposes multi-agent reasoning into three layers: a governance layer for planning and resource allocation, an execution layer for task solving and review, and a compliance layer for final answer control. By evaluating the framework across reasoning tasks, LLMs, execution modes, and execution policies, we find that multi-agent systems organized in a company-style hierarchy generally outperform other organizational structures. Besides, hierarchical coordination also reduces token consumption relative to flat collaboration in most settings. For example, for GPT-OSS-120B, the hierarchical setting improves performance over flat multi-agent system by 102.73% while reducing token usage by 74.52% on SQuAD 2.0. Further analysis shows that hierarchy helps most when tasks benefit from stable skill assignment, controlled information flow, and layered verification. Overall, our findings highlight organizational structure as an important factor in multi-agent reasoning, shaping not only effectiveness and cost, but also coordination behavior.

Figures & tables

Figure 1: Illustration of our company-style hierarchical MAS framework OrgAgent . Layer A performs governance-level planning, including skill assignment and execution control; Layer B carries out task solving through collaborative drafting and feedback; Layer C finalizes the output through answer consolidation and compliance checking.

Figure 2: Overview of OrgAgent , a company-style hierarchical MAS framework.

Model	Baseline		Flat		Hierarchical (AUTO)		$\Delta$ Improvement (%)	$\Delta$ Token Reduction (%)
Model	Score	Avg token	Score	Avg token	Score	Avg token	$\Delta$ Improvement (%)	$\Delta$ Token Reduction (%)
MuSiQue (F1-score)
GPT-5mini	$51.28\pm 4.2$	2,778	$50.31\pm 2.50$	28,479	$\mathbf{68.98\pm 1.70}$	11,408	+37.11%	59.94%
GPT-OSS-120B	$37.98\pm 2.58$	2,687	$48.40\pm 1.55$	25,209	$\mathbf{57.58\pm 1.98}$	12,046	+18.97%	52.22%
LLaMA-3.1-8B	$11.52\pm 3.09$	2,370	$14.55\pm 0.09$	51,425	$\mathbf{32.59\pm 14.65}$	12,322	+123.99%	76.04%
MuSR (Accuracy)

Table 1: Performance and token cost comparison across baseline, flat, and hierarchical organizations.

Figure 3: Performance comparison of different execution policies across three benchmarks. Rows correspond to MuSiQue, MuSR, and SQuAD 2.0, while columns correspond to GPT-5 mini, GPT-OSS-120B, and Llama-3.1-8B. Bars denote the performance under FLAT, AUTO, STRICT, BALANCE, and NOCAP policies, and the red dashed line indicates the single-agent baseline.

Avg Token
Model	AUTO	STRICT	BALANCE	NOCAP	FLAT
MuSiQue
GPT-5mini	11,545	3,795	12,711	21,766	28,479
GPT-OSS-120B	12,046	3,633	12,428	14,539	25,209
LLaMA-3.1-8B	12,322	3,282	12,399	37,306	48,198
MuSR

Table 2: Average token consumption under different execution policies across benchmarks and models.

Figure 4: Skill distribution on SQuAD 2.0 across GPT-5mini, GPT-OSS-120B, and LLaMA-3.1-8B. The pie charts show the proportion of selected skill profiles under the hierarchical framework.

SQuAD 2.0 Unanswerable Questions Abstention Rate (%)
	GPT- 5mini	GPT- OSS-120B	LLaMA- 3.1-8B
Baseline	0	0	0
Flat	3.02	0	0
AUTO	22.58	36.56	19.39
STRICT	13.98	32.26	19.35
BALANCE	30.11	35.48	21.51

Table 3: AbsRate(%) on Unanswerable Questions in SQuAD 2.0 under different organizational settings and execution policies.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Benchmark	Type	Total
MuSR	Multistep soft reasoning	756
MuSiQue	Compositional multi-hop QA	24,814
SQuAD 2.0	Reading comprehension with unanswerable questions	151,054

Appendix

Table 4: Overview of the benchmarks.

Structure	Layer	Configuration	Max Round
Flat	–	–	3
Hierarchical	Layer A	CEO / CTO / COO	3
	Layer B	DIRECT	1
		LIGHT MAS	3
		FULL MAS	5

Appendix

Table 5: Organizational structures and maximum coordination rounds used in our framework.

Figure 5: Token-performance trade-off on MuSiQue across GPT-5 mini, GPT-OSS-120B, and Llama-3.1-8B.

Figure 6: Token-performance trade-off on MuSR across GPT-5 mini, GPT-OSS-120B, and Llama-3.1-8B.

Figure 7: Token-performance trade-off on SQuAD 2.0 across GPT-5 mini, GPT-OSS-120B, and Llama-3.1-8B.

Figure 8: Skill-selection distributions across six skill types on MuSiQue. The top row shows the Drafter, and the bottom row shows the Specialist, across GPT-5mini, GPT-OSS-120B, and LLaMA-3.1-8B.

Figure 9: Skill-selection distributions across six skill types on MuSR. The top row shows the Drafter, and the bottom row shows the Specialist, across GPT-5mini, GPT-OSS-120B, and LLaMA-3.1-8B.

Multi-Agent Systems

Multi-Agent Systems studies how autonomous agents coordinate, compete, communicate, and learn in shared or interdependent tasks.

See all 651 papers

OpenDGO: Benchmarking Multi-Agent Organizations Across Diverse Application Domains
Oct 12, 2026Xunkai Li, Miao Zhang, Zekai Chen +6Multi-Agent LLM Systems Multi-Agent Systems
From Skills to Talent: Organising Heterogeneous Agents as a Real-World Company
Apr 24, 2026Zhengxu Yu, Yu Fu, Zhiyuan He +5Multi-Agent Orchestration Multi-Agent Systems
1HUAWEI Noah’s Ark Lab · University Liverpool · University College London
Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm
Jul 28, 2026Huan Chen, Xiang Song, Jian Jin +2Multi-Agent LLM Systems Multi-Agent Orchestration
Shunfeng Technology Co., Ltd., China · Shenzhen University, China

Oct 12, 2026J/K move · Enter open · S save

OpenDGO: Benchmarking Multi-Agent Organizations Across Diverse Application Domains

Xunkai Li, Miao Zhang, Zekai Chen, Xuaner Chen+5

Multi-Agent LLM Systems Multi-Agent Systems Multi-Agent Collaboration AI Agent Benchmarks Multi-Agent Orchestration Multi-Agent Communication Topology+1

Open paper AskPDF arXiv

Large language model (LLM)-based multi-agent systems are increasingly applied across diverse domains, yet different applications require different forms of agent organization. Existing studies typically adopt predefined organizations, such as Star, Chain, and Tree, or learn communication graphs to improve performance on specific tasks. However, these methods are largely designed and evaluated within a limited set of application domains, providing little understanding of whether an organization effective in one domain remains suitable for others with different collaboration requirements. We present OpenDGO, a controlled benchmark and analysis framework covering six real-world domains and three collaboration settings: Sequential Dependency, Parallel Decomposition, and Information Integration. Under matched tasks, agents, tools, and resource limits, OpenDGO compares fixed and learned organizations while separately examining communication graphs, role assignments, input assignments, resource budgets, and component failures. Our results show that there is no ``one-organization-for-all'' solution: preferred organizations vary across applications, and even the same communication graph can yield different outcomes under different role and input assignments. These findings highlight the need for application-dependent organization design, aiming to understand which organization is suitable for a given application, under what conditions, and why.

Abstract

Multi-Agent Systems

Explore similar work

OpenDGO: Benchmarking Multi-Agent Organizations Across Diverse Application Domains

From Skills to Talent: Organising Heterogeneous Agents as a Real-World Company

Toward an Organizational Science of Multi-Agent LLM Systems: Decoupling Who, How, and Which Algorithm