A Survey on Collaborative Mechanisms Between Large and Small Language Models
Authors: Yi Chen, Haiquan Wang, JiaHao Zhao, HaoHao Han
Organizations: Chengdu Institute of Computer Applications, Chinese Academy of Sciences, Chengdu, China · University of Chinese Academy of Sciences, Beijing, China
Large Language Models (LLMs) deliver powerful AI capabilities but face deployment challenges due to high resource costs and latency, whereas Small Language Models (SLMs) offer efficiency and deployability at the cost of reduced performance. Collaboration between LLMs and SLMs emerges as a crucial paradigm to synergistically balance these trade-offs, enabling advanced AI applications, especially on resource-constrained edge devices. This survey provides a comprehensive overview of LLM-SLM collaboration, detailing various interaction mechanisms (pipeline, routing, auxiliary, distillation, fusion), key enabling technologies, and diverse application scenarios driven by on-device needs like low latency, privacy, personalization, and offline operation. While highlighting the significant potential for creating more efficient, adaptable, and accessible AI, we also discuss persistent challenges including system overhead, inter-model consistency, robust task allocation, evaluation complexity, and security/privacy concerns. Future directions point towards more intelligent adaptive frameworks, deeper model fusion, and expansion into multimodal and embodied AI, positioning LLM-SLM collaboration as a key driver for the next generation of practical and ubiquitous artificial intelligence.
Figures & tables
Figure 1: CoGenesis framework structure diagram. 1. Context-aware instructional examples. 2. Context-aware Language Models (LLMs) excel in context awareness but pose privacy risks. 3. On-device specialized Small Language Models (SLMs) prioritize privacy but have lower performance. 4. Collaborative LLMs and SLMs enhance privacy protection and improve performance.
Figure 2: CITER framework structure diagram. Utilizes a router for collaborative inference between SLM and LLM. The router is trained using routing preferences collected through three scenarios. Scenario 1: SLM generates the correct token, routing preference assigned to SLM. Scenario 2: SLM generates an incorrect token, while LLM generates the correct token, routing preference assigned to LLM. Scenario 3: Neither SLM nor LLM generates the correct token; collaborative reasoning is performed to obtain the complete response for assigning routing preference.
Figure 3: Iterative training framework of Collab-RAG. The SLM updates its parameters based on the generation quality of the LLM reader. This process iterates multiple times, progressively improving the SLM’s decomposition capability.
Small language models are compact neural language systems designed to deliver useful text and reasoning capabilities with limited compute, memory, and energy.
Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration, which can be combined into hybrid topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence.
Liangqi Yuan, Wenzhi Fang, Shiqiang Wang +2
Purdue University, IN, USA. · University of Exeter, UK. · Princeton University, USA.
Personal intelligence requires strong capabilities under strict privacy and efficiency constraints. Unlike cloud-first intelligence, which relies on powerful but costly and privacy-vulnerable large language models (LLMs), we focus on local-first intelligence, emphasizing on-device small language models (SLMs), which enable efficient, private inference but have limited capacity. To reconcile the capability gap with privacy and efficiency, we propose a dynamic collaboration framework, where an SLM learns to proactively decide how to consult an LLM during multi-step reasoning, while the LLM provides adaptive feedback instead of acting as a passive tool, modeling them as heterogeneous agents under information asymmetry. We further systematically investigate how collaboration strategies are shaped by SLM and LLM capabilities as well as efficiency and privacy constraints. Evaluation results reveal a distinct scaling effect: stronger SLMs become more self-reliant, while stronger LLMs enable fewer and more informative interactions. In addition, the learned dynamic collaboration strategies significantly outperform various baselines and transfer robustly to unseen LLMs as well as to real-world personal tasks without additional training.
Hang Zeng, Xiangyu Liu, Yong Hu +5
Shanghai Jiao Tong University, Shanghai, China · WeChat Tencent, Beijing, China · State University of New York at Buffalo, New York, United States
Large Language Models (LLMs) have transformed many areas of artificial intelligence (AI), but their substantial resource requirements limit their deployment on mobile and edge devices. Although LLMs have demonstrated impressive capabilities in various applications, the next challenge is deploying these models on resource-limited edge devices. Recent innovations in software optimizations and specialized hardware have made it feasible to deploy LLMs on edge devices. Edge deployment offers benefits including local data processing, cost savings, reduced latency, and personalized responses. This paper reviews the latest techniques that make the deployment of LLMs on edge devices feasible.