Multi-LLM Collaboration

Follow this topic Ask about this topic

Momentum

11 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 48

All topics

Sibyl: An Efficient Small-large Model Collaboration Framework for Long-horizon Tasks
Oct 4, 2026Zhewei Fang, Yuxin Zhang, Zhenwei Shao +8LLM Agent Training Long-Horizon LLM Agents
Hangzhou Dianzi University · Fudan University · The Hong Kong University of Science and Technology +3
Learning to Ask: Information Acquisition for SLM-LLM Collaboration, under a budget
Oct 1, 2026Yongjun Kim, Xiaoxiao Li, Jaeho LeeRL for Language Model Reasoning Efficient Language Model Reasoning
Pohang University of Science and Technology (POSTECH) · Department of Electrical and Computer Engineering, University of British Columbia · Vector Institute
Multi-LLM Collaborative Alignment via Stackelberg Games
Sep 30, 2026Christina Hahn, Shangbin Feng, Dean Light +3RL for Language Models LLM Alignment
University of Washington · Amazon · University of British Columbia
Which Models Work Well Together? Measuring Heterogeneity for LLM Team Selection
Sep 29, 2026Liangyu Teng, Hengsong Liu, Juncen Guo +4Language Model Ensembles Ensemble Diversity
Fudan Institute on Networking Systems of AI
Draft-KV: Learning Useful Latent Communication Between Language Models
Sep 28, 2026Linquan Wu, Shichang Meng, Tianxiang Jiang +7Knowledge Augmentation for Language Models Multi-LLM Collaboration
City University of Hong Kong · University of Science and Technology of China · University of Electronic Science and Technology of China +3
One Model Is Not a Crowd: Multi-LLM and Aspect-Conditioned Diverse Comment Generation
Sep 27, 2026Nafis Irtiza Tripto, Delvin Ce Zhang, Mahjabin Nahar +1LLM Evaluation Controllable Text Generation
College of Information Sciences and Technology, Pennsylvania State University, PA, USA · University of Sheffield, Sheffield, UK
Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs
Sep 26, 2026Vincent-Daniel Yun, Woosang Lim, Haneul Yoo +3Multi-Agent LLM Systems LLM Inference Acceleration
University of Southern California, USA · Seoul National University, Republic of Korea · New York University, USA
COMED: The Missing Middle Between Routing and Collaboration in Multi-LLM Inference
Sep 22, 2026Norah Alballa, Wenxuan Zhang, Salma Kharrat +4LLM Inference Efficiency LLM Routing
KAUST · LUMS
The Universe of Universes: Benefit Yield Functions, Implosion Thresholds, and Infrastructure-Aware Optimization in Multi-LLM Systems
Sep 14, 2026Danielle Franklin, Vasu Raj JainLanguage Model Ensembles Multi-LLM Collaboration
Humanity + AI, Inc.
From Collaboration to Capability: Internalizing Routed LLM Experts into Compact Reasoners
Sep 14, 2026Frank Nie, Shuyao Wang, Ethan B. LiuRL for Language Model Reasoning Multi-Agent Collaboration
Shandong University · Zhejiang University
UnitBoost: Managing Compound LLM Systems with a Merge Operator, Not a Model
Sep 9, 2026Xing Zhang, Guanghui Wang, Yanwei Cui +2Multi-Agent System Optimization LLM Agent Orchestration
AWS Generative AI Innovation Center
The Collaboration Tax: How Much LLM Multi-Agent Systems Pay to Coordinate
Aug 23, 2026Weixiang Sun, Zehong Wang, Hong Huang +3Multi-Agent Coordination Multi-LLM Collaboration
University of Notre Dame · Meta Superintelligence Labs · Simon Fraser University
FutureBridge: Token Selection Beyond Local Preference in Collaborative Decoding
Aug 7, 2026Quanquan Li, Hongbo Zhang, Yihe Chi +9Language Model Decoding Multi-LLM Collaboration
1East China Normal University · 2Peking University · University of Science and Technology of China +3
MIDAS: Multi-LLM Iterative Data-Adaptive Summarization
Aug 5, 2026Karen Lee, Dhanashree Balaram, Seojun Shon +1Text Summarization Automatic Prompt Optimization
Volkswagen Group Innovation, California
Harnessing the Wisdom of LLM Crowds through Complementarity-Driven Iterative Collaboration
Jul 31, 2026Yanbin Fang, Xuan Wei, Wei ChenLLM Routing Multi-LLM Collaboration
Business School Chinese University of Hong Kong, Shanghai, China · Antai College of Economics and Management Shanghai Jiao Tong University, Shanghai, China · School of Business University of Connecticut, Stamford, CT, US
Divergence Decoding: Training-Free Capability Fusion
Jul 28, 2026Yimi Wang, Hao Li, Shuo Yang +6LLM Routing Scientific Reasoning in Language Models
Peking University · International Digital Economy Academy (IDEA) · The University of Hong Kong
PyroDash: Cost-Efficient Token-Level Small-Large Language Model Collaborative Inference
Jul 22, 2026Niqi Lyu, Pengtao Shi, Wei Qiu +4RL for Language Model Reasoning Cost-Aware Inference
Pyromind Dynamics Inc.
Oracle Gap and Signal Fidelity: A Fixed-Pool Diagnostic for Test-Time Collaboration
Jul 20, 2026Jie HuLLM Evaluation LLM Answer Verification
Research Institute of China Telecom
When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaboration
Jun 27, 2026Luís Brito, Carlos BaqueroLLM Agent Security Multi-LLM Collaboration
Escola Superior de Tecnologia e Gestão (ESTG), Politécnico de Viana do Castelo (IPVC), Portugal · Faculdade de Engenharia (FEUP), Universidade do Porto (UP), Portugal
Sakana Fugu Technical Report
Jun 19, 2026Yujin Tang, Edoardo Cetin, Jinglue Xu +11LLM Agent Orchestration LLM Agent Training
Fugu Team, Sakana AI 1
Heterogeneous LLM Debate Under Adversarial Peers: Honest Gains, Replacement Costs, and Resilience
Jun 18, 2026Prashanti Nilayam, Kiran Kumar Ramanna, Prashil Tumbade +1Multi-Agent LLM Systems Adversarial Attacks on LLMs
ServiceNow
tap: A File-Based Protocol for Heterogeneous LLM Agent Collaboration
Jun 12, 2026Minseo KimMulti-Agent Communication Multi-LLM Collaboration
HUA Labs, Seoul, Republic of Korea
Be My Tutor: On-Policy Co-Distillation for Mutual LLM Improvement via Peer Feedback
Jun 12, 2026Woohyeon Byeon, Jiwon Jeon, Jeonghye Kim +1Language Model Distillation On-Policy Distillation
KAIST
Scaling Participation in Modular AI Systems
Jun 5, 2026Shangbin Feng, Yike Wang, Weijia Shi +3Human-Centered AI Multi-LLM Collaboration
University of Washington, Seattle, WA, USA. · Stanford University, Stanford, CA, USA.
Post-Training Recipe, More Than Model Family, Shapes Multi-Agent LLM Conversational Behavior
May 30, 2026Luyang Zhang, Jialu Wang, Fei Xue +1Language Model Post-Training Multi-LLM Collaboration
Carnegie Mellon University · University of California, Santa Cruz · Independent Researcher
Can LLM Teams Play What? Where? When?
May 28, 2026Anastasia Kotelnikova, Viktor Byzov, Maria Dolzhenkova +1Multi-Agent LLM Systems LLM Agent Evaluation
Vyatka State University Kirov, Russia · European University at St. Petersburg St. Petersburg, Russia
Beyond Consensus: Trace-Level Synthesis in Mixture of Agents
May 27, 2026Shreyas Fadnavis, Praitayini Kanakaraj, Felix WyssLanguage Model Ensembles Multi-Agent Reasoning
Bioscope AI
DEI: Diversity in Evolutionary Inference for Quality-Diversity Search
May 26, 2026John Donaghy, Shikhar RastogiDistributed Optimization Large Language Model-Guided Optimization
Gensyn
HyLaT: Efficient Multi-Agent Communication via Hybrid Latent-Text Protocol
May 25, 2026Xinyi Mou, Siyuan Wang, Zejun Li +2Agent Communication Protocols LLM Agent Training
1Fudan University · 2The Chinese University of Hong Kong · 3King’s College London +2
Ontology-constrained multi-LLM scoring of hypothesis support in the predictive processing literature
May 23, 2026Hamed Nejat, Alexander Maier, Jesse Spencer-Smith +1Predictive Coding Multi-LLM Collaboration
Department of Psychology and Vanderbilt Brain Institute, Vanderbilt University · Data Science Institute, Vanderbilt University

Oct 4, 2026, cs.AIJ/K move · Enter open · S save

Sibyl: An Efficient Small-large Model Collaboration Framework for Long-horizon Tasks

Zhewei Fang, Yuxin Zhang, Zhenwei Shao, Mengze Li+7

Hangzhou Dianzi University · Fudan University · The Hong Kong University of Science and Technology+3

LLM Agent Training Long-Horizon LLM Agents Multi-LLM Collaboration RL for LLM Agents

Open paper AskPDF arXiv

Small language models (SLMs) offer a promising foundation for on-device agents through low-latency, resource-efficient inference, yet limited reasoning and planning capabilities constrain their performance on long-horizon tasks requiring multi-step interaction with the environment. Step-level collaboration between SLMs and larger cloud-hosted models can bridge this gap, but identifying states that warrant cloud assistance remains challenging: the contribution of each cloud call is entangled with subsequent actions and can be assessed only from the final task outcome. Compounding this challenge, the SLM must balance two competing objectives: maximizing task success and minimizing cloud calls. To address this, we propose Sibyl, an algorithm that trains SLM agents to selectively consult cloud models at the step level and internalize their guidance for subsequent decisions, achieving strong task performance with minimal cloud reliance. Sibyl follows a three-stage training pipeline that (1) builds a robust base policy through consultation-free self-evolving reinforcement learning (RL); (2) cold-starts consultation behavior via decisive-disagreement state mining; and (3) jointly optimizes consultation decisions and guidance internalization through consultation-aware RL. Experiments on ALFWorld and WebShop demonstrate that Sibyl, using only a 0.6B-parameter model, outperforms state-of-the-art baselines, including agent training and routing methods, by 95.2% and 80.4% in success rate while averaging only 0.8 and 3.9 cloud calls per trajectory, respectively.

Momentum

Related topics

Latest papers 48