Selecting and Composing Agent Design Patterns: Industrial Case Evidence and Controlled Reference-System Experiments
Organizations: Agency for Science, Technology and Research (A*STAR), Singapore · Tencent Research Institute, China · Shanghai Dongfang Yiteng Technology Co., Ltd., China · Chengdu Xuansu Technology, China ·
Abstract
Agent pattern catalogs describe architectural alternatives, but engineers also need evidence for selecting and composing them. We extend a cognitive-function--execution-topology framework with an evidence-driven selection procedure and explicit composition contracts. The study combines two industrial case accounts, a contributor's collaboration design, seven practitioner workshops, and executable fault-injection experiments. Three local reference systems use SQLite state, shared artifact files, and a file-backed map-publication surrogate. A 2 x 2 ablation separates function-specific contracts from execution controls across 36 scenario families, with ten input variants per family. Under equal family weighting, composition correctly completes or safely refuses 11 of 12 families in each system, compared with 2, 2, and 1 for the simple baselines. These are constructed challenge-suite results, not production success rates. Recovery without consumer-side verification increases false-success cases in the map surrogate. All composed systems still fail a boundary case in which trusted evidence is wrong. An audit of a frozen six-arm language-model pilot reproduces its scores but confirms an evaluator-dependent ranking reversal. The contribution connects industrial failure, pattern selection, executable treatment, external acceptance, and revision. Selection is demonstrated rather than evaluated against an alternative procedure; catalog admission rests on separate criteria. The results support bounded composition mechanisms, not a claim that the framework improves engineers' design decisions over an equally informative conventional method.
Figures & tables
| Source | Available evidence | Role in this study |
|---|---|---|
| Dongfang Yiteng | Contributor account of prototype failures, state architecture, task dependencies, and recovery design | Industrial motivation for exact parameter binding and durable execution |
| Xuansu GIS | Failure accounts, pipeline and lifecycle descriptions, console and map-output artifacts | Industrial motivation for consumer acceptance and bounded recovery |
| Shared artifacts | Fan’s two design notes and interaction diagram; no supplied deployment measurements | Attributed design input for event and artifact experiments |
| Seven workshops | Dated discussion records, transcripts, questions, disagreements, and proposed revisions | Requirements for measurement and selection |
| RegTech pilot | Six frozen model outputs, one source bundle, 12 checklist items, original scorer | Historical outcome and evaluator audit |
| Reference study | Executed SQLite/file systems, injected faults, per-run traces and artifact readback | Controlled evidence for the implemented mechanisms |
| System | : function-specific contract | : execution control | Example pattern coordinates |
|---|---|---|---|
| Business | Bind the action to a trusted record; check task authority | Fix dependency order; commit effect and completion together; resume from checkpoints | Action/Memory responsibilities; Chain with durable orchestration and Governance admission |
| Artifacts | Validate allowed scope, version, and bytes against a trusted digest | Atomic effect/receipt; deduplicated consumption and completion; collection after all receipts | Collaboration handoff; Memory retention; bounded event execution |
| Maps | Decode consumer response; inspect status, type, body, and image uniformity | Route known observed errors to repair; bound retries; stop on unknown errors | Perception and verification; Reasoning/Reflection with Route and Loop |
| System | ||||
|---|---|---|---|---|
| Business execution | 2 / 0 / 8 | 4 / 2 / 6 | 6 / 0 / 4 | 9 / 2 / 1 |
| Shared artifacts | 2 / 0 / 5 | 2 / 4 / 2 | 7 / 0 / 5 | 7 / 4 / 1 |
| Map publication | 1 / 0 / 8 | 1 / 3 / 1 | 2 / 0 / 10 | 8 / 3 / 1 |
| Arm | Predicted items | Matches | Precision | Recall | F1 |
|---|---|---|---|---|---|
| Single pass | 16 | 6 | .3750 | .5000 | .4286 |
| Critic and repair | 16 | 7 | .4375 | .5833 | .5000 |
| Self-refinement | 16 | 7 | .4375 | .5833 | .5000 |
| Candidate-guided review | 16 | 9 | .5625 | .7500 | .6429 |
| Union queue | 29 | 10 | .3448 | .8333 | .4878 |
| Consensus refinement | 16 | 8 | .5000 | .6667 | .5714 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Business execution | Shared artifacts | Map publication | |
|---|---|---|---|
| 1 | Clean update | Clean two-consumer delivery | Clean publication |
| 2 | Altered proposal identifier | Duplicate delivery | HTTP 200 with XML error |
| 3 | Wrong valid identifier | Crash before effect | Blank rendered image |
| 4 | Reordered steps | Crash after effect | Zero metatile error |
| 5 | Crash before effect | Corrupt artifact bytes | Transient publication failure |
| 6 | Crash after effect | Stale artifact version | Transient consumer failure |
LLM Agent Reliability
Large language model (LLM) agent reliability concerns dependable, safe, and verifiable behavior across tools, multi-step tasks, and changing environments.
See all 843 papers