Multi-agent self-improvement enables agents to improve through interaction and feedback. However, sequential dependencies across agents and refinement rounds can propagate errors, increase computational costs, and limit parallelism. Moreover, refining individual features or agents in isolation can overlook their dependencies and restrict collective coordination. To address these challenges, we propose ConvRSI, a framework for parallelizing multi-agent self-improvement through convolution. ConvRSI couples intra-agent convolution, which jointly refines related agent features, with inter-agent graph convolution, which aggregates messages according to the collaboration topology. Both stages operate in latent space, enabling parallel updates without repeatedly decoding and re-encoding intermediate messages. Experiments on MATH-500 show that ConvRSI achieves 5.04% performance improvement while reducing 17.27% inference cost, and 45.11% latency compared with RecursiveMAS. It also supports diverse collaboration patterns by representing their topologies as adjacency matrices. These results demonstrate the potential of convolutional co-evolution for efficient multi-agent self-improvement. Our code and datasets are provided in https://github.com/LYA2333/ConvRSI.
Figures & tables
Figure 1: The ConvRSI framework shown for one slot across N agents. Intra-agent convolution produces a local message from each feature stack. The GCN aggregates the stacked local messages and returns one updated message per agent for the next round. The schematic uses Mt and Mt+1 for the input and output of the graph layer, denoted by Uk and Mk in Eq. ( 7 ).
Figure 2: The ConvRSI training pipeline. Inner training uses independent agent objectives. Outer training can compute prefixes for all rounds through the convolutional recurrence before the collaboration forwards. The collaboration consumes the prefix for each round and computes cross-entropy at the final output. Flames highlight trainable components and snowflakes mark frozen language models.
Style
Adjacency construction
Degree after adding self-loops
Sequential
Ai+1,i=1 , 1≤i<N
D11=1 ; Dii=2 for i>1
Mixture
Ao,j=wj , j∈B
Doo=1+∑j∈Bwj ; Djj=1 for j∈B
Distillation
Asymmetric influence: Aij=Aji on selected pairs
Dii=1+∑jAij
Deliberation
Reciprocal exchange: Aij,Aji>0 on interacting pairs
Dii=1+∑jAij
Table 1: Representative communication graphs for different collaboration styles. Rows of A are receivers and columns are senders. Unspecified entries are zero. In the mixture construction, o is a designated receiving node, B contains the remaining nodes, and wj>0 . These are abstract graph constructions.
Collaboration Pattern
Role
Model Size & Version
Sequential Style (Light)
Planner
Qwen3-1.7B [ Yang et al., 2025a ]
Critic
Llama-3.2-1B-Instruct [ Grattafiori et al., 2024 ]
Solver
Qwen2.5-Math-1.5B-Instruct [ Yang et al., 2025b ]
Mixture Style
Code Specialist
Qwen2.5-Coder-3B-Instruct [ Hui et al., 2024 ]
Science Specialist
BioMistral-7B [ Labrak et al., 2024 ]
Math Specialist
DeepSeek-R1-Distill-Qwen-1.5B [ Yang et al., 2025b ]
Table 3: Agent roles and model assignments for the two collaboration patterns evaluated with ConvRSI . Sequential-Light uses a planner, critic, and solver in sequence. In the mixture, a summarizer combines outputs from code, science, and mathematics specialists.
Method
Solver or agent configuration
MATH-500 accuracy (%)
MBPP+ pass@1 (%)
Modular harness optimization
ModularRSI
Qwen3.5-4B
67.1
32.1
Persistent memory and skills
Hermes+
Sequential-Light solver
64.6
28.6
Hermes+
Qwen3.5-4B
67.2
27.3
Multi-agent workflow optimization and self-training
Table 5: Comparison on mathematical reasoning and code generation, grouped by method family. The second column identifies the solver backbone or agent configuration used in our experiments. Bold marks the highest score in each benchmark column.
Figure 3: Performance versus total benchmark runtime. Along each line, points from left to right denote recursion depths r=1,2,3 , with a filled marker at r=3 . Points higher and further to the left indicate higher scores and shorter runtimes.
Figure 4: Change in total runtime and mean generated tokens per problem from r=1 to r=3 . Each percentage is relative to the same method’s one-round value on the same benchmark.
Recursive self-improvement (RSI) lets a system improve from its own outcomes; in LLM-based multi-agent systems, Agents refine one another within a task, and outcomes improve how they collaborate across tasks. However, existing multi-agent collaboration leaves this loop open: collaboration is pre-defined at the operator level, topology-only learning keeps verbatim exchange that propagates errors, and reward maximization on a system's own outcomes concentrates on a few teams. To address these challenges, we propose CollabFlow, an RSI system of Learned Agent Collaboration: a trainable Collab-Director constructs teams of complete Agents, a frozen executor runs them, and each round's outcomes retrain the director. Within each round, the edges of a collaboration graph carry protocols of Evidence-Conditioned Communication: a receiver adopts a differing answer only when the sender's evidence is stronger by a margin, so the director learns who communicates and how. Across rounds, we further propose Collaborative Trajectory Balance (CTB), a flow-based objective that credits each team once across its construction orders and targets a reward-proportional distribution over teams, so several good teams stay in play. We also bound how far this self-generated target moves between rounds, which shrinks as records accumulate. On twelve datasets, CollabFlow outperforms all baselines and keeps improving across rounds. Code is available at https://anonymous.4open.science/r/CollabFlow-631E.
Xiao Huang, Mingda Zhang, Junming Zhang +5
The Chinese University of Hong Kong, Shenzhen · Fudan University · University of Oxford
Recursive self-improvement (RSI) of a model on non-verifiable tasks, such as open-ended research, faces a supervision bottleneck when its outputs exceed what even human experts can reliably assess, leaving the model itself (optimizee) as the best available optimizer and evaluator. However, a single model instance struggles to critique and improve its own complex reasoning under this homogeneous loop. To address this, we propose Multi-Agent Self-Supervision (MASS), an RSI method that alternates between evolutionary workflow optimization and supervised fine-tuning on self-generated trajectories. Guided by early findings that multi-agent topologies excel at complex reasoning, MASS prompts a single base model to iteratively propose, execute, and self-evaluate multi-agent workflows. Through an evolutionary search constrained by structural guardrails, the model optimizes these computational-graph-like orchestrations, discovering the most effective distinct roles and information routing for a given task. Over two MASS cycles with Qwen3.6-27B, the model achieves 1.2-1.6x higher performance per output tokens on four open-ended public benchmarks. Because the improved model subsequently acts as a better optimizer and evaluator, this alternating framework enables a continuous, recursive bootstrapping of the model's capabilities. Moreover, multi-agent traces are also more training-efficient: a student trained on them outperforms a single-agent student trained on 1.4x more training tokens. These findings suggest that jointly learning orchestration and bounded subagent execution from multi-agent trajectories can provide an effective signal for RSI.
LLM-based multi-agent systems (MAS) have emerged as an effective paradigm for complex and long-horizon tasks. However, in real-world tasks, MAS often exhibit various failures during execution and such failures are difficult to eliminate during design. This motivates experience-driven MAS evolution, where a system improves based on its own execution experience. Yet such evolution is challenging because MAS experience is prolonged and intricate, interleaving multiple agents' execution chains and communication messages, which makes it difficult to identify what should be improved. To address this challenge, we propose Meta-Team, an experience-driven MAS evolution framework based on collaborative self-evolution. Meta-Team preserves the execution context of each agent and coordinates post-task communication, enabling agents to exchange distributed evidence for evolution. Building on this design, Meta-Team conducts multi-scale self-evolution, transforming execution experience into reusable improvements to agent behaviors, inter-agent coordination, and team-level organization. Across six long-horizon agent benchmarks, Meta-Team consistently outperforms single-agent systems, hand-crafted MAS, and prior MAS evolution methods; further analyses demonstrate that Meta-Team enables more reliable and scalable MAS self-evolution.
Zhezheng Hao, Tianfu Wang, Huanshuo Dong +7
Zhejiang University · Hong Kong University of Science and Technology · Tencent