Long-horizon multimodal agents rely on specialized capabilities for perception, retrieval, reasoning, verification, and execution. Existing designs typically activate a fixed capability set or invoke a predefined workflow, incurring substantial computational overhead while failing to accommodate stage-dependent capability demands. In this paper, we study the combinatorial capability allocation problem for long-horizon multimodal agent systems, where the system selects a cost-sensitive subset of specialized capabilities at each interaction stage, which is nontrivial since capability values depend on the selected subset, while previous allocations alter the states encountered by subsequent decisions. We introduce CoCA, an on-policy learning framework that recovers a deployable capability-subset policy from sparse conditional comparisons. On states visited by the student policy, the stronger teacher compares the marginal net values of candidate capabilities, conditioned on the currently selected subset. Then, we adopt a conditional utility model to transform such comparisons into an autoregressive capability-subset policy, avoiding explicit enumeration. We further introduce dual-level on-policy distillation to address distribution mismatch both across environment states and within the partial subsets encountered during set construction. Finally, trajectory-level reinforcement learning refines the distilled policy toward task success, activation cost, and allocation stability. At inference time, allocation is performed solely by the lightweight student policy without teacher queries or online updates. Experiments on long-horizon multimodal environments and controlled capability-demand shifts demonstrate the superiority of our method over the state-of-the-art baseline methods.
Figures & tables
Figure 1: An overview of the proposed CoCA framework. The diagram illustrates the learning pipeline from conditional preference modeling and teacher-guided policy construction to dual-level distillation and trajectory-level optimization.
Models
Baselines
Multi-Agent
Adaptive Control
OSWorld
VisualWebArena
GAIA
MMMU-Pro
Succ. ↑
Cost ↓
Succ. ↑
Cost ↓
Acc. ↑
Cost ↓
Acc. ↑
Cost ↓
Qwen3.6-27B
ReAct
✘
✔
17.2
0.90
13.6
0.88
33.1
0.86
43.2
0.83
Toolformer
✘
✘
20.8
0.83
15.1
0.81
39.4
0.75
44.1
0.85
Puppeteer
✔
✘
24.9
0.72
19.0
0.73
38.7
0.70
47.6
0.71
AutoTool
✘
✘
26.3
0.64
22.4
0.59
44.0
0.62
46.9
0.66
NaviAgent
✘
✔
31.7
0.55
23.1
0.51
46.2
0.53
52.3
0.57
Table 1: The overall evaluation results of CoCA and other baseline methods on four benchmark datasets. The best-performing method is marked by bold .
Methods
OSWorld
VisualWebArena
GAIA
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
ReAct
3.38
0.52
0.53
0.48
0.09
3.21
0.46
0.49
0.47
0.07
2.71
0.54
0.55
0.50
0.15
Toolformer
3.11
0.55
0.58
0.55
0.21
2.79
0.54
0.54
0.51
0.18
2.40
0.58
0.61
0.56
0.22
Puppeteer
2.43
0.62
0.61
0.59
0.29
2.36
0.57
0.62
0.60
0.30
1.88
0.65
0.64
0.62
0.38
AutoTool
2.17
0.64
0.69
0.66
0.42
1.98
0.63
0.65
0.61
0.35
1.69
0.66
0.72
0.69
0.44
NaviAgent
1.71
0.72
0.73
0.70
0.50
1.72
0.68
0.71
0.69
0.49
1.31
0.74
0.76
0.73
0.55
Table 2: Evaluation of allocation quality under Kimi-K2.6. The best-performing method is marked by bold .
Methods
OSWorld
GAIA
Succ. ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
Acc. ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
w/o Set-Dep
39.6
0.97
0.79
0.77
0.75
0.66
56.4
0.74
0.82
0.80
0.78
0.69
w/o Set-On Policy
41.2
0.91
0.84
0.78
0.76
0.68
58.1
0.69
0.86
0.81
0.79
0.71
w/o Stage-On Policy
38.4
0.95
0.81
0.82
0.80
0.67
55.7
0.72
0.84
0.85
0.83
0.70
w/o Traj. RL
40.1
1.58
0.86
0.85
0.83
0.69
57.3
1.19
0.89
0.87
0.85
0.72
w/o Null-Anchor
41.5
0.89
0.83
0.74
0.78
0.58
58.6
0.67
0.85
0.76
0.80
0.61
Table 3: Ablation study on the OSWorld and GAIA benchmarks under Kimi-K2.6.
Figure 2: Hyperparameter sensitivity analysis on OSWorld and GAIA benchmarks under Kimi-K2.6.
Methods
Avg. Latency
LLM Calls
Token Cost (k)
Avg. Act. /stage
Train Cost
ReAct
62.4
14.8
41.2
6.7
0.0
Toolformer
55.1
12.3
35.7
5.9
1.2
Puppeteer
47.8
10.6
31.4
5.1
0.0
AutoTool
43.2
9.4
28.9
4.6
3.5
NaviAgent
38.5
8.1
24.3
3.8
5.8
CoCA (Ours)
31.7
6.5
18.6
2.9
7.4
Table 4: Efficiency on GAIA benchmark under Kimi-K2.6.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Methods
OSWorld
VisualWebArena
GAIA
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
ReAct
3.52
0.49
0.50
0.46
0.08
3.35
0.43
0.46
0.45
0.06
2.85
0.51
0.52
0.48
0.13
Toolformer
3.24
0.52
0.55
0.52
0.19
2.95
0.51
0.51
0.49
0.16
2.53
0.55
0.58
0.53
0.20
Puppeteer
2.55
0.59
0.58
0.56
0.27
2.47
0.54
0.59
0.57
0.28
1.98
0.62
0.61
0.59
0.35
AutoTool
2.31
0.61
0.66
0.63
0.39
2.12
0.60
0.62
0.59
0.33
1.78
0.63
0.69
0.66
0.41
NaviAgent
1.89
0.68
0.69
0.66
0.46
1.88
0.64
0.67
0.65
0.45
1.42
0.70
0.72
0.69
0.51
Appendix
Table 5: Evaluation of allocation quality under Qwen3.6-27B. The best-performing method is marked by bold .
Methods
VisualWebArena
MMMU-Pro
Succ. ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
Acc. ↑
SwC ↓
DSA ↑
PPA ↑
TAR ↑
UGC ↑
w/o Set-Dep
31.5
0.96
0.75
0.74
0.71
0.63
58.7
0.93
0.84
0.76
0.78
0.64
w/o Set-On Policy
33.2
0.90
0.80
0.77
0.74
0.65
60.1
0.86
0.87
0.81
0.82
0.69
w/o Stage-On Policy
32.4
0.93
0.78
0.80
0.77
0.65
61.0
0.90
0.85
0.83
0.82
0.70
w/o Traj. RL
33.1
1.49
0.83
0.82
0.80
0.68
61.6
1.31
0.89
0.86
0.84
0.72
w/o Null-Anchor
33.9
0.87
0.79
0.72
0.75
0.57
59.5
0.85
0.86
0.74
0.79
0.58
Appendix
Table 6: Ablation study on the VisualWebArena and MMMU-Pro benchmarks under Kimi-K2.6.
Recent works train agents by constructing large-scale multimodal environment pools. However, we find that simply increasing the number of multimodal environments does not always benefit. We further analyze the limitations in current multimodal environment distributions through a series of experiments. Based on these findings, we study how to build more effective training environment distributions from two dimensions: diversity and difficulty structure. For diversity, we propose Ability-aware Environment Selection (AES) to obtain diverse environment sets. For difficulty structure, we propose Hierarchical Difficulty Curriculum (HDC), which organizes curriculum learning through two difficulty levels: harness weakening and state-scale progression. Experiments show that AES and HDC effectively improve multimodal agent training.
Kejian Zhu, Zhuoran Jin, Dongqi Huang +4
The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
Multi-teacher on-policy distillation (MOPD) integrates domain specialists into a shared student, but determining how strongly each teacher should supervise remains challenging. Existing discrepancy-based allocation can assign strong supervision even when the student already matches or exceeds the teacher's task performance, since behavioral mismatch does not necessarily imply a remaining capability deficit. We propose Capability-Aware Multi-Teacher On-Policy Distillation (CA-MOPD), which combines specialist merging with capability-guided allocation to account for what the student has already learned. CA-MOPD first merges reinforcement-learning specialists into a strong student initialization, then uses verifier-estimated teacher--student capability gaps to adjust domain-level distillation strength. Supervision decreases as capability gaps close and increases when estimated student capability regresses, while dense token-level guidance remains provided by the teachers. Across six benchmarks spanning mathematics, code generation, and instruction following, CA-MOPD achieves an average score of 32.22, outperforming naive MOPD, Open-MOPD, and the merged initialization by 0.86, 0.83, and 0.74 points, respectively. Controlled comparisons further support the benefits of merged initialization and capability-gap guidance under matched training budgets.
AI agents are expected to play an increasingly important role in future decision-making systems. In this paper, we consider collaborative systems composed of heterogeneous multi-agent systems (MAS), where their members have different capabilities and operating costs. We study how agents can delegate tasks to one another so that certain research tasks can be completed effectively under resource-constrained scenarios. We first develop a multi-agent reinforcement learning-based (MARL) delegation training that enables agents to make sequential delegation decisions while minimizing the total execution cost. We then extend this approach to MARL with prescribed delegation topologies. Furthermore, we introduce two new frameworks for collaboration and delegation in multi-agent systems. The first framework proposes that an agent's policy depends not only on the current state of the underlying Markov decision process but also on the interaction history, including previous joint actions. This history-dependent formulation can improve coordination even in fully observable environments, where conventional MARL methods typically restrict policies to depend only on the current state. The second framework proposes a novel, potentially multi-dimensional monetary mechanism to facilitate the collaboration and delegation for MAS.