ABSTRAL: Automatic Design of Multi-Agent Systems Through Iterative Refinement and Topology Optimization

Weijia Song; Jiashu Yue; Zhe Pang

cs.AIMar 24, 2026

ABSTRAL: Automatic Design of Multi-Agent Systems Through Iterative Refinement and Topology Optimization

Authors: Weijia Song, Jiashu Yue, Zhe Pang

Abstract

How should multi-agent systems be designed, and can that design knowledge be captured in a form that is inspectable, revisable, and transferable? We introduce ABSTRAL, a framework that treats MAS architecture as an evolving natural-language document, an artifact refined through contrastive trace analysis. Three findings emerge. First, we provide a precise measurement of the multi-agent coordination tax: under fixed turn budgets, ensembles achieve only 26% turn efficiency, with 66% of tasks exhausting the limit, yet still improve over single-agent baselines by discovering parallelizable task decompositions. Second, design knowledge encoded in documents transfers: topology reasoning and role templates learned on one domain provide a head start on new domains, with transferred seeds matching coldstart iteration 3 performance in a single iteration. Third, contrastive trace analysis discovers specialist roles absent from any initial design, a capability no prior system demonstrates. On SOPBench (134 bank tasks, deterministic oracle), ABSTRAL reaches 70% validation / 65.96% test pass rate with a GPT-4o backbone. We release the converged documents as inspectable design rationale.

Figures & tables

System	Topology Var.?	Readable Knowledge?	Role Discovery?
DSPy	No	No	No
ADAS	Yes	No	No
AFlow	Yes	No (code)	No
MaAS	Yes	No (neural)	No
MASS	Yes	No	No
GPTSwarm	Partial	No	No

Table 1: Abstral vs. prior automated agent design systems.

Figure 1: The Abstral pipeline. Layer 1 refines

\mathcal{A}_{t}

via BUILD

\to

RUN

\to

ANALYZE

\to

UPDATE; Layer 2 monitors convergence signals (C1–C4); Layer 3 seeds new topology families via GED repulsion.

Class	Failure Signature	Target	Example
EC1	Wrong conclusion, correct info	K	Add domain rule
EC2	Bottleneck or wrong routing	R	Prefer hierarchy for heterogeneous tasks
EC3	One agent, incompatible sub-tasks	T	Create specialist role
EC4	Malformed message / type mismatch	P	Fix interface contract
EC5	Successful pattern, not yet codified	T	Encode reusable heuristic

Table 2: Five evidence classes and their target SKILL.md sections.

Method	Success %	$\Delta$ vs. GPT-4o ∗	Topology	Search cost
Published baselines (FC mode, cited):
GPT-4o-mini ∗	33.58%	$-$ 25.38	single	—
GPT-4o ∗	58.96%	—	single	—
Claude-3.7-Sonnet ∗	65.67%	+6.71	single	—
GPT-4.1 ∗	69.40%	+10.44	single	—
o4-mini-high ∗	76.87%	+17.91	single	—

Table 3: SOPBench bank results. Published baselines ∗ from Li et al. (2025) (FC mode). Our experiments use GPT-4o backbone with identical evaluation protocol.

Figure 2: SOPBench pass rate trajectory (20-task validation batches). Gray band marks consolidation at I6; dashed line = published GPT-4o baseline. Held-out test (94 tasks): 65.96% using the O3/I4 configuration.

Outer	Family	Peak Pass Rate	Roles	Conv. Iter	Discovered Roles
1	Hierarchical	55%	5, 8 edges	$T_{\max}{=}8$	Auth., Trans., Credit, Currency
2	Ensemble	60%	6, 8 edges	$T_{\max}{=}8$	Auth., Account, Credit, Currency
3	Ensemble	70%	6, 8 edges	$T_{\max}{=}8$	Auth., CreditApp., Admin., FX

Table 4: Outer-loop diversity (SOPBench bank). All pairwise GED

>\delta_{\text{struct}}{=}3

Outer	Role	Type	Origin
O1	AuthenticationSpecialist	Specialist	EC3: auth failures when manager handled login
O1	TransactionSpecialist	Executor	EC3: deposit/withdrawal parameter errors
O1	CreditCardSpecialist	Specialist	EC3: credit ops mixed with account ops
O1	CurrencySpecialist	Specialist	EC3: FX rate lookup errors
O2	RequestDispatcher	Router	EC2: no delegation in flat ensemble
O2	ResultAggregator	Aggr.	EC3: multi-agent outputs not combined

Table 5: Specialist roles discovered via EC3 (SOPBench bank). Broad

\to

fine-grained across outer loops.

Outer	EC1	EC2	EC3	EC4	EC5
O1 (8 iter)	10	9	2	17	2
O2 (8 iter)	7	17	6	8	2
O3 (8 iter)	11	25	1	0	2
Total	28	51	9	25	6

Table 6: Evidence class distribution across outer loops (SOPBench bank, 24 iterations total).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Signal	Condition	Weight	Rationale
C1: Skill diff	$<5$ lines changed	2	Document has stabilized
C2: Pass rate plateau	$<\varepsilon$ change for 3 iters	2	Performance has plateaued
C3: EC collapse	EC1+EC2 $<10\%$ of evidence	1	No actionable failures remain
C4: Complexity	$>200$ rules	1	Document is bloated (triggers consolidation first)

Appendix

Table 7: Convergence detector signals.

Field	Content
role_name	[Adjective][Noun] format (e.g., “Unit-Consistency Enforcer”). Generic names rejected.
epistemic_stance	{Skeptic, Verifier, Explorer, Synthesizer, Enforcer}
system_prompt	Failure mode, domain knowledge excerpts, escalation criteria
tool_access	Specific tools (principle of least privilege)
birth_trace	Trace ID of the triggering EC3 event
interface_contract	Required input/output fields with types (Pydantic-enforced)

Appendix

Table 10

Family	Structure	When Preferred
Single	One agent, full tool access	Simple tasks
Pipeline	Linear DAG: A $\to$ B $\to$ C	Sequential reasoning
Ensemble	Fan-out + aggregator	Parallel independent sub-tasks
Debate	Two agents + judge (cyclic)	Adversarial verification
Hierarchical	Manager $\leftrightarrow$ specialists (cyclic)	Heterogeneous sub-tasks
Dynamic routing	Router $\to$ conditional branches	Variable complexity

Appendix

Table 11

O	I	Pass Rate	Family	Agents	Rules	Words
1	1	35%	hierarchical	5	60	2055
1	2	40%	hierarchical	5	70	2343
1	3	50%	hierarchical	5	75	2613
1	4	40%	hierarchical	5	81	2893
1	5	50%	hierarchical	5	83	3135
1	6	55%	hierarchical	5	52 †	1370

Appendix

Table 8: Full SOPBench bank iteration trajectory (20-task validation batches).

Outer	Rules (start $\to$ end)	Words (start $\to$ end)	Consol. ( $\rho$ )	Pre $\to$ Post
O1	60 $\to$ 56	2055 $\to$ 1847	0.63	83 $\to$ 52
O2	45 $\to$ 69	861 $\to$ 1821	0.64	84 $\to$ 54
O3	57 $\to$ 66	1223 $\to$ 1666	0.73	75 $\to$ 55

Appendix

Table 9: SKILL.md document growth across outer loops (SOPBench bank).

Failure Mode	Mechanism	Mitigation
Semantic drift	Contradictory rules accumulate	Consolidation every 5 iter. + C4
Synonym collapse	Identical topologies, different names	Dual-criteria repulsion
Weak trace signal	Miscalibrated sandbox	Abort if EC1+EC2 $<30\%$

Appendix

Table 10: Anticipated failure modes with mitigations.

Metric	Value
Total iterations	24
Cumulative tokens	153,959
Cumulative API calls	480
Avg. tokens per iteration	$\sim$ 6,415
Avg. API calls per iteration	20
Estimated cost	$\sim$ $5

Appendix

Table 11: Resource usage for SOPBench bank experiment (3 outer

\times

8 inner iterations).

Multi-Agent Systems

Multi-Agent Systems studies how autonomous agents coordinate, compete, communicate, and learn in shared or interdependent tasks.

See all 669 papers

Automated Multi-Agent Organization Design: From Architecture Search to Organizational Learning
Oct 20, 2026Shao-Jun Xia, Yuner Zhang, Huixin Zhang +3Multi-Agent Systems Multi-Agent System Optimization
Carnegie Mellon University, Pittsburgh, United States · Northeastern University, Boston, United States · University of Southern California, Los Angeles, United States
MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems
Jul 30, 2026Mao-xun Huang, Jerry Wang, Yi-Cheng Lai +3Multi-Agent LLM Systems Multi-Agent Coordination
Department of Computer Science, Cornell University, United States · Department of Computer Science, University of Illinois Urbana-Champaign, United States · Institute of Information Science, Academia Sinica, Taiwan
TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems
May 10, 2026Chen Xu, Yicheng Hu, Ruizi Wang +4Multi-Agent LLM Systems Multi-Agent System Optimization
1Carnegie Mellon University · University of Science and Technology of China · 3National University of Singapore +1

Oct 20, 2026J/K move · Enter open · S save

Automated Multi-Agent Organization Design: From Architecture Search to Organizational Learning

Shao-Jun Xia, Yuner Zhang, Huixin Zhang, Di Wang+2

Carnegie Mellon University, Pittsburgh, United States · Northeastern University, Boston, United States · University of Southern California, Los Angeles, United States

Multi-Agent Systems Multi-Agent System Optimization Multi-Agent Coordination

Open paper AskPDF arXiv

Agentic system design is increasingly shifting from hand-crafted prompts and workflows toward automated search over agentic programs and interaction structures, with multi-agent systems receiving growing attention. Yet a central question remains unresolved: when does an automatically generated multi-agent structure exhibit a functionally meaningful division of responsibility? We argue that automated multi-agent design should move beyond searching for effective architectures toward identifying which responsibilities are functionally necessary, how they should be allocated, and when they should change. To explore this, we propose Automated Multi-Agent Organization Design (AMOD), which treats organization itself as a first-class design object, explicitly modeling how agent teams are composed, what functional responsibilities the agents assume, how they coordinate, and how collective decisions are made. We formulate AMOD as an execution-driven design loop. For each task, an automated designer instantiates an organization by assigning functional responsibilities and coordination rules to a set of agents. After execution, task outcomes, intermediate artifacts, and interaction traces are examined to identify which responsibilities may be missing, duplicated, or ineffective, and the organization is revised accordingly. We also propose testing discovered responsibilities by removing them, replacing them with generic roles, or changing their assignment under matched resource budgets, with complementary evaluations. AMOD provides a principled foundation for evidence-based and adaptive multi-agent organization design.

Abstract

Multi-Agent Systems

Explore similar work

Automated Multi-Agent Organization Design: From Architecture Search to Organizational Learning

MANTA: Multi-Agent Network Topology Adaptation for Self-Evolving Multi-Agent Systems

TacoMAS: Test-Time Co-Evolution of Topology and Capability in LLM-based Multi-Agent Systems