Large language model-based (LLM-based) multi-agent systems (MAS) are increasingly used to extend agentic problem solving via role specialization and collaboration. MAS workflows can be naturally modeled as directed computation graphs, where nodes execute agents or sub-workflows and edges encode dependencies and message passing. However, implementing complex graph workflows in current frameworks still requires substantial manual effort, offers limited reuse, and makes it difficult to integrate heterogeneous external context sources. To overcome these limitations, we present MASFactory, a graph-centric framework for orchestrating LLM-based MAS. It introduces Vibe Graphing, a human-in-the-loop approach that compiles natural-language intent into an editable workflow specification and then into an executable graph. In addition, the framework provides reusable components, skill support, multimodal message handling, and pluggable context integration, as well as a visualizer for topology preview, runtime tracing, and human-in-the-loop interaction. We evaluate MASFactory on seven public benchmarks, validating both reproduction consistency for representative MAS methods and the effectiveness of Vibe Graphing. Our code (https://github.com/BUPT-GAMMA/MASFactory, licensed under Apache-2.0) and video demonstration (https://youtu.be/ANynzVfY32k) are publicly available.
Figures & tables
Figure 1: Vibe Graphing in MASFactory. MASFactory turns a user’s natural-language intent into an executable multi-agent workflow via a three-stage, human-in-the-loop process, then compiles and executes the resulting workflow at runtime.
Figure 2: Architecture overview of MASFactory framework.
Method
HumanEval
MBPP
BigCodeBench
SRDD
MMLU-Pro
GAIA
GPQA
ChatDev (original)
82.50
71.40
50.70
82.91
–
–
–
ChatDev (MASFactory)
81.30
74.20
53.30
84.23
–
–
–
MetaGPT (original)
67.07
36.03
50.10
78.19
–
–
–
MetaGPT (MASFactory)
89.02
59.14
51.70
72.77
–
–
–
AgentVerse (original)
85.00
74.54
65.92
87.55
64.64
12.12
38.39
AgentVerse (MASFactory)
85.00
75.15
64.12
91.06
64.16
12.73
37.50
Table 1: Main results are reported on a percentage scale (0-100). The symbol “–” indicates not applicable due to the incompatibility between programming-focused MASs and general-purpose reasoning benchmarks.
Metric
ChatDev
AgentVerse
VG
VC-L
VC-M
VG
VC-L
VC-M
Cost ($)
0.26
3.49
3.02
0.59
4.43
6.08
Table 2: Comparison of monetary cost between Vibe Graphing (VG) and Vibe Coding (VC). VC-L and VC-M denote the low and medium reasoning lengths of VC.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Human-in-the-loop interaction during Vibe Graphing in the visualizer. The user provides feedback to refine the role assignment and structure across stages.
Figure 4: Visualizer views used in the Vibe Graphing.