CHATDIFY: Automating End-to-End Dify Workflow Synthesis from Natural Language via Multi-Agent Collaboration
Organizations: Hong Kong University of Science and Technology, Hong Kong, China · WeBank Co., Ltd, Shenzhen, China
Abstract
The proliferation of Large Language Models has catalyzed powerful application development platforms like Dify, which enable users to build sophisticated LLM-powered workflows through modular node orchestration. However, manually configuring these workflows requires deep platform expertise and remains time-consuming and error-prone, creating a significant barrier for non-expert users. To address this challenge, we propose ChatDify, a novel multi-agent framework that automates end-to-end Dify workflow synthesis from natural language descriptions. ChatDify employs an Actor-Critic architecture comprising three specialized agents: the Planner Agent that decomposes user requests into structured execution plans, the Executor Agent that performs context-aware parameter synthesis and schema-augmented DSL compilation, and the Reviewer Agent that ensures reliability through hybrid verification with closed-loop refinement. We evaluate ChatDify on a curated benchmark of 100 production-inspired tasks across three state-of-the-art LLM backends, achieving 0.96, 0.91, and 0.75 success rates with Qwen-3.7, GLM-5.1, and DeepSeek-v4, respectively. These results demonstrate its robustness and practical applicability for lowering the technical barrier to creating sophisticated LLM applications.
Figures & tables
| Task Category | Count | Percentage |
| Text Generation Tasks | 18 | 18% |
| Information Extraction | 18 | 18% |
| Multimodal Understanding | 11 | 11% |
| Multimodal Generation | 8 | 8% |
| Data Analysis and Processing | 14 | 14% |
| Conversation and Interaction | 10 | 10% |
| Model | Method | SR | Token (k) |
| Qwen-3.7 | ZeroShot | 0.43 | 24 |
| ZeroShot+Dynamic | 0.67 | 52 | |
| ChatDify (ours) | 0.96 | 256 | |
| GLM-5.1 | ZeroShot | 0.13 | 13 |
| ZeroShot+Dynamic | 0.45 | 36 | |
| ChatDify (ours) | 0.91 | 348 |
| Model | Method | SR | Token (k) |
| Qwen-3.7 | Full ChatDify | 0.96 | 256 |
| w/o Static | 0.93 | 270 | |
| w/o Dynamic | 0.73 | 175 | |
| w/o Static & Dynamic | 0.71 | 166 | |
| GLM-5.1 | Full ChatDify | 0.91 | 348 |
| w/o Static | 0.83 | 426 |
| Model | Epochs | SR | Token (k) |
| Qwen-3.7 | 1 | 0.73 | 175 |
| 2 | 0.95 | 244 | |
| 3 | 0.96 | 256 | |
| GLM-5.1 | 1 | 0.71 | 219 |
| 2 | 0.88 | 305 | |
| 3 | 0.91 | 348 |
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
LLM Agent Orchestration
Large language model agent orchestration coordinates models, tools, and workflows to complete complex tasks reliably and efficiently.
See all 494 papers