Sibyl: An Efficient Small-large Model Collaboration Framework for Long-horizon Tasks
Authors: Zhewei Fang, Yuxin Zhang, Zhenwei Shao, Mengze Li, Zheng Lin, Long Chen, Zhou Yu, Zhe Chen, +3 more
Organizations: Hangzhou Dianzi University · Fudan University · The Hong Kong University of Science and Technology · University of Luxembourg · Simon Fraser University · Alibaba Group
Small language models (SLMs) offer a promising foundation for on-device agents through low-latency, resource-efficient inference, yet limited reasoning and planning capabilities constrain their performance on long-horizon tasks requiring multi-step interaction with the environment. Step-level collaboration between SLMs and larger cloud-hosted models can bridge this gap, but identifying states that warrant cloud assistance remains challenging: the contribution of each cloud call is entangled with subsequent actions and can be assessed only from the final task outcome. Compounding this challenge, the SLM must balance two competing objectives: maximizing task success and minimizing cloud calls. To address this, we propose Sibyl, an algorithm that trains SLM agents to selectively consult cloud models at the step level and internalize their guidance for subsequent decisions, achieving strong task performance with minimal cloud reliance. Sibyl follows a three-stage training pipeline that (1) builds a robust base policy through consultation-free self-evolving reinforcement learning (RL); (2) cold-starts consultation behavior via decisive-disagreement state mining; and (3) jointly optimizes consultation decisions and guidance internalization through consultation-aware RL. Experiments on ALFWorld and WebShop demonstrate that Sibyl, using only a 0.6B-parameter model, outperforms state-of-the-art baselines, including agent training and routing methods, by 95.2% and 80.4% in success rate while averaging only 0.8 and 3.9 cloud calls per trajectory, respectively.
Figures & tables
Figure 1: Sibyl enables step-level device–cloud collaboration on long-horizon tasks, combining high success with low latency and cost. Right: success-rate comparisons with representative baselines; see § 5 for further results.
Figure 2: Three-stage training pipeline of Sibyl, which trains an SLM to act locally, consult the cloud at critical steps, and internalize its guidance to achieve high task success with few cloud calls.
Method
ALFWorld
WebShop
Pick
Look
Clean
Heat
Cool
Pick2
All
Score
Succ.
Closed-Source Model
♢ Qwen3.7-Max
88.6±2.9
100.0±0.0
80.3±7.7
66.7±3.6
64.0±4.0
95.8±4.2
82.4±0.8
52.7±0.2
46.4±0.6
♢ Gemini 3.1 Pro
100.0±0.0
100.0±0.0
96.3±3.7
100.0±0.0
100.0±0.0
95.8±4.1
98.6±1.4
65.2±0.2
60.0±0.4
Qwen3-0.6B assisted by Qwen3.7-Max
♡ GRPO
12.4±1.7
25.6±19.4
6.2±5.7
4.2±3.6
6.7±2.3
4.2±4.2
9.1±2.3
67.1±0.1
16.0±0.4
Table 1: Results on ALFWorld and WebShop, averaged over three runs. ♡ , ♢ , ♣ , and ♠ : device-only, cloud-only, cloud-assisted training, and collaborative inference, respectively. The Qwen3-1.7B results for GRPO and SEED are from ( Wu et al., 2026 ) .
Method
ALFWorld
WebShop
Succ. ↑
Call ratio ↓
Calls / Ep. ↓
Cost ↓
Score ↑
Succ. ↑
Call ratio ↓
Calls / Ep. ↓
Cost ↓
Closed-Source Model
Qwen3.7-Max
82.4
100.0
15.2
$8.6
52.7
46.4
100.0
10.8
$48.5
Gemini-3.1 Pro
98.6
100.0
9.1
$5.8
65.2
60.0
100.0
8.4
$58.6
Qwen3-0.6B assisted by Qwen3.7-Max
RouteLLM
65.7
64.5
12.5
$5.3
24.6
22.8
85.5
5.1
$18.3
Table 2: Task performance and test-time cloud cost on ALFWorld and WebShop. Call ratio is the percentage of executed steps delegated to the cloud, Calls/Ep. is the mean number of cloud calls per episode, and Cost is the total API cost over the full test set, calculated from cloud input/output token usage and API prices.
Figure 5
Training API Cost (Stage 1/2/3)
Training GPU Hours
Inference API Cost (full test set)
ALFWorld
10.7/1.7/$92.7
149.7
$0.4
WebShop
13.0/6.5/$126.0
205.7
$16.1
Table 3: Training and inference costs of Sibyl on ALFWorld and WebShop.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Task type
Success (%)
Pick
91.7
Look
100.0
Clean
100.0
Heat
95.7
Cool
95.2
Pick2
94.1
Appendix
Table 5: Unseen-split success by task type.
Task
Seen success (%)
Seen calls/ep.
Unseen success (%)
Unseen calls/ep.
Pick
100.0
0.1
91.7
0.5
Look
92.3
0.5
100.0
0.7
Clean
100.0
0.5
100.0
0.7
Cool
92.0
1.2
95.2
0.7
Heat
87.5
1.5
95.7
0.8
Pick2
87.5
1.3
94.1
0.6
Appendix
Table 6: Task-wise success and average cloud calls per episode on the Seen and Unseen splits.
Case
Edge-only branch
Cloud’s critical action
Edge continuation and outcome
ALFWorld: clean butterknife
30 steps, no cloud calls, and failure. The on-device SLMs takes an ordinary knife , cleans and returns it, and never reaches the target butterknife .
The cloud steers the search to diningtable 3 at step 20, where butterknife 1 is visible.
The on-device SLMs takes the butterknife, cleans it at the sink, and moves it to sidetable 1 . The collaborative run succeeds in 25 steps with 3 cloud calls.
ALFWorld: two credit cards
30 steps, no cloud calls, and failure while searching for the second card after placing the first.
The cloud first requests inventory , then chooses diningtable 1 to locate creditcard 1 , and finally armchair 1 as the destination.
The on-device SLMs takes the card and executes the final move locally. The collaborative run succeeds in 14 steps with 3 cloud calls.
WebShop: random-color bath mitt
From the same product page, the forced edge action is click[buy now] ; the purchase has empty options and scores 0.75 .
The cloud selects the required option click[random color 1pc] .
The identical edge continuation then executes click[buy now] , producing the correct option and score 1.00 .
WebShop: ottoman size
From the same product page, the forced edge action is click[buy now] ; the purchase has empty options and scores 0.80 .
The cloud selects the required size click[100x40x40cm] .
The identical edge continuation executes click[buy now] , producing the correct size and score 1.00 .
Appendix
Table 7: Representative edge–cloud rescue cases. The cloud provides the bottleneck action shown in the third column; subsequent actions are executed by the on-device SLM. The two benchmarks follow their respective evaluation protocols.
Large language model (LLM) agents excel at solving complex long-horizon tasks through autonomous interaction with environments. However, their real-world deployment faces a fundamental device--cloud dilemma: on-device models are efficient but often brittle, while cloud models are stronger but costly in computation. State-of-the-art LLM device--cloud routers usually make coarse task-level decisions, which cannot adapt to the changing difficulty of multi-step agent interactions. To address this issue, we present Hera, a step-level device--cloud LLM agent coordinator for long-horizon tasks achieving a strong performance--cost Pareto frontier. Hera adopts a novel two-stage training paradigm: (1) imitation learning for cold-start, followed by (2) reinforcement learning that jointly optimizes task success and cloud usage efficiency. The first stage casts step-level routing as a supervised classification problem: the device agent is replayed on cloud trajectories, with each state labeled by the agreement between device and cloud actions. In the second stage, we perform cost-aware reinforcement learning by grouping identical states across trajectories and updating Hera with labels favoring higher expected return and fewer future cloud calls. We evaluate Hera on ALFWorld, WebShop, and AppWorld, where it consistently outperforms prior methods, achieving 92.5% of the cloud-only success rate with cloud use in only 46.3% of steps.
Yuxin Zhang, Mengxue Hu, Zheng Lin +8
1Fudan University · 2Alibaba Group · 3The University of Hong Kong +4
Small language models (SLMs) are attractive as local agent controllers because they reduce remote inference, latency, and deployment footprint, yet structured tool errors can cause an agent step to fail. Existing routers typically select a model once per query. However, agents expose sequential decision points whose difficulty dynamically changes based on intermediate observations. We propose STEPGATE, an uncertainty-aware handoff framework that scores each local SLM action and selectively escalates challenging steps to a stronger model. On a 52-task held-out single-step BFCL-derived test split, the Qwen2.5-1.5B/7B pair attains 82.7% task success with 30.8% escalation, versus 67.3% local-only and 75.4% random escalation (which uses 33.8% escalation). In a separate multi-turn evaluation, STEPGATE achieves 69.0% trajectory success and 84.0% action success using only 30.0% cloud actions, compared with 48.0%/70.5% local-only, 60.0%/78.2% random escalation, and 57.0%/77.1% query-level routing (strong-only achieves 82.0% trajectory success at 100% cloud actions). These results suggest that step-level escalation recovers a large share of the performance gap to the stronger Qwen2.5-7B backend at a matched cloud-action rate while transmitting fewer tokens remotely. However, our evaluation is limited to one model family, a single stronger backend, and scripted tasks. Furthermore, the test sets are small, multi-turn comparisons rely on paired intervals and statistical tests, and our risk tiers serve as research annotations rather than formal safety guarantees.
Abolfazl Younesi
Department of Computer Engineering Sharif University of Technology Tehran, Iran
Large language models (LLMs) offer strong capabilities but raise cost and privacy concerns, whereas small language models (SLMs) facilitate efficient and private local inference yet suffer from limited capacity. To synergize the complementary strengths, we introduce a dynamic collaboration framework, where an SLM learns to proactively decide how to request an LLM during multi-step reasoning, while the LLM provides adaptive feedback instead of acting as a passive tool. We further systematically investigate how collaboration strategies are shaped by SLM and LLM capabilities as well as efficiency and privacy constraints. Evaluation results reveal a distinct scaling effect: stronger SLMs become more self-reliant, while stronger LLMs enable fewer and more informative interactions. In addition, the learned dynamic collaboration strategies significantly outperform static pipelines and standalone inference, and transfer robustly to unseen LLMs.
Hang Zeng, Xiangyu Liu, Yong Hu +5
Shanghai Jiao Tong University, Shanghai, China · WeChat Tencent, Beijing, China · State University of New York at Buffalo, New York, United States.