cs.MAAug 3, 2026

Collective Reasoning, Single-Writer Execution: Fusion-MoA for Long-Horizon Agents

Authors: Zongyou Yang, Yinghan Hou, Botao Su, Ercheng Zhao

Organizations: Dyson School of Design Engineering, Imperial College London, London, United Kingdom · Department of Electrical and Electronic Engineering, Imperial College London, London, United Kingdom

Abstract

Mixture-of-Agents systems combine several language models into a stronger response, but tool-using agents act over long horizons in which every step changes external state and the history keeps growing. We present Fusion-MoA, a serving architecture that brings multi-model collaboration into this loop through executor-anchored consultation. One Executor model holds the full context and emits every text and tool call; when consultation is admitted, three specialist models review its proposed next action through a compact view under a shared 120-second deadline. The fleet sits behind one OpenAI-compatible endpoint, so an unmodified agent client drives it as a single model. Fusion-MoA serves an in-car assistant and an in-car personal agent for intelligent cockpits and is deployed on MetaX C550 servers. Evaluated as four quantized open-weight models on five NVIDIA L40S GPUs and driven by Claude Code, it passes 78 of the 89 Terminal-Bench 2.1 tasks, 16 more than its Executor alone and 29 more than a self-hosted quantized Kimi K3 on the same client. On the 63 tasks that both it and the API-served LongCat-2.5-Preview pass, it needs 14.5 minutes on average against 17.5. Specialist reviews use 0.4% of the Executor's tokens, and every tool call comes from the Executor.

Figures & tables

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Mar 27, 2026

AgentCollab: A Self-Evaluation-Driven Collaboration Paradigm for Efficient LLM Agents

Autonomous agents powered by large language models (LLMs) perform complex tasks through long-horizon reasoning and tool interaction, where a fundamental trade-off arises between execution efficiency and reasoning robustness. Models at different capability-cost levels offer complementary advantages: lower-cost models enable fast execution but may struggle on difficult reasoning segments, while stronger models provide more robust reasoning at higher computational cost. We present AgentCollab, a self-driven collaborative inference framework that dynamically coordinates models with different reasoning capacities during agent execution. Instead of relying on external routing modules, the framework uses the agent's own self-reflection signal to determine whether the current reasoning trajectory is making meaningful progress, and escalates control to a stronger reasoning tier only when necessary. To further stabilize long-horizon execution, we introduce a difficulty-aware cumulative escalation strategy that allocates additional reasoning budget based on recent failure signals. In our experiments, we instantiate this framework using a two-level small-large model setting. Experiments on diverse multi-step agent benchmarks show that AgentCollab consistently improves the accuracy-efficiency Pareto frontier of LLM agents. The code is available at https://github.com/SparkEngineAI/AgentFlashInfer.
Jun 2, 2026

MOSAIC: Efficient Mixture-of-Agent Scheduling via Adaptive Aggregation and Inference Concurrency

Mixture-of-Agents (MoA) systems improve reasoning accuracy by routing each query to multiple expert LLMs and aggregating their outputs. Efficiently executing this workload on limited GPU resources has bottlenecks. Skill-based routing creates skewed expert demand, and combining instruction-tuned LLMs with long-reasoning models results in extreme variability in generation lengths. Consequently, traditional scheduling strategies suffer from significant GPU idling and throughput collapse due to load imbalances. We present MOSAIC, a scheduling framework for accelerating batch-oriented MoA workloads. MOSAIC uses two-stage workload-aware scheduling: an expert placement stage, where an integer linear program (ILP) decides expert placement, followed by a query-sequencing stage that schedules requests based on predicted output lengths. Additionally, MOSAIC uses confidence-aware adaptive aggregation, leveraging inter-expert agreement to bypass the heavy aggregator LLM for consensus queries. On a 4-GPU system, MOSAIC achieves 2.7-3.4x end-to-end speedups over round-robin scheduling and reduces the mean time-to-answer by 3.9-7.8x, while matching accuracy within +/-1pp.
Aug 27, 2026

One Model, Many Minds: Unlocking Multi-Agent Synergy in a Single Agent via Mixture of Roles

Specializing Large Language Models (LLMs) toward distinct abilities underpins successes ranging from personalized assistants to multi-agent systems (MAS). Single-agent paradigms rely on pre-defined personas or steering vectors to induce specialization, yet they impose a single fixed specialization that fails to adapt to diverse queries. Conversely, MAS achieves dynamic multi-perspective problem solving by orchestrating agents with distinct text-based roles, but fusing these specializations requires multi-turn interactions that inflate context length and inference cost. To address these limitations, we propose Mixture of Roles (MoRe), which adaptively composes multiple specializations into a single steering vector for single-turn inference. Specifically, MoRe learns a diversified codeboox of steering vectors, each of which encodes a latent role. A query-aware router dynamically fuses the codebook into a steering vector that encompasses multiple roles. By steering the backbone LLM with the composed vector, MoRe enables multi-perspective specialization in a single-agent, single-turn inference process. The proposed MoRe can be efficiently trained via a three-stage SFT curriculum and GRPO post-training, while the backbone LLM remains frozen. Experiments across reasoning and personality benchmarks show that MoRe outperforms single-agent baselines by 2.2% on average, and achieves performance on par with MAS while reducing token cost by 20x.