Solving long-horizon agentic tasks requires language agents to interact with their environments over multiple steps, and reliable performance often comes from the expensive capable models available through cloud providers. Meanwhile, alternative efficient models with inferior performance but a much lower cost are emerging, including lower-tier cloud models and smaller open-weights models deployable on local hardware. This raises a natural question that remains less explored than traditional query-level routing: can capable and efficient models collaborate within a single long-horizon agentic task to substantially reduce cost while preserving task accuracy? In this paper, we systematically investigate key design choices for such collaboration through model handoffs, where one model takes over from another, focusing on handoff direction and context transfer. We find that efficient-to-capable handoffs generally underperform capable-only execution with limited or even no cost reduction, whereas capable-to-efficient handoffs at suitable stages of task progress largely preserve or even improve accuracy while substantially reducing cost. These findings suggest that a one-time, simple, well-timed handoff strategy is good enough for cost-effective agent collaboration, motivating AgentBaton, a lightweight detector that learns when to hand off from recurring patterns in task progress. Across 11 model pairs and two representative benchmarks, AgentBaton reduces API costs by 65.9% on average while largely preserving task accuracy, with accuracy gains of up to 8.8 percentage points over capable-only execution. Experiments across various reasoning-effort settings further show that AgentBaton extends the empirical accuracy-cost Pareto frontier.
Figures & tables
Figure 1: Accuracy–cost trade-offs across reasoning-effort settings on DeepSWE v1.1. Solid lines and circles denote single-model baselines. For AgentBaton , diamonds denote handoffs to Qwen 3.8 27B and squares denote handoffs to GPT-5.6 Luna; marker colors identify the source model, as listed in the legend. The green dashed line with a light green band connects the empirical Pareto frontier among evaluated AgentBaton configurations, whose effort labels are highlighted. For example, the purple labeled MAX denotes GLM-5.3 at max effort handing off to Qwen 3.8 27B at xhigh effort; the red labeled HIGH denotes Kimi K3 at high effort handing off to GPT-5.6 Luna at max effort.
Figure 2: Overview of one-time model handoff and context transfer. The receiving model inherits the workspace at handoff and continues the task. Execution history is provided either as an inspectable logfile or directly in context.
Terminal-Bench 2.1
DeepSWE v1.1
30%
75%
30%
75%
Handoff
In-context
Logfile
In-context
Logfile
In-context
Logfile
In-context
Logfile
Efficient → Capable
Qwen → GLM
83.2 / 1.32
83.2 / 1.22
68.5 / 1.19
79.8 / 1.26
51.3 / 4.04
62.8 / 2.93
38.9 / 2.61
53.1 / 1.87
Qwen → Kimi
79.8 / 2.91
84.3 / 1.58
70.8 / 3.31
79.8 / 1.30
47.8 / 7.77
62.0 / 4.45
37.2 / 5.76
50.4 / 2.52
Qwen → GPT
83.2 / 1.19
89.9 / 0.87
82.0 / 2.15
85.4 / 0.85
73.5 / 6.20
70.8 / 5.00
58.4 / 8.84
67.3 / 5.74
Table 1: Context transfer across handoff directions at 30% and 75% of the source model’s baseline trajectory. Each cell reports accuracy (%) / mean recorded API cost per task (USD) in the handoff runs. Bold marks strictly higher accuracy or lower cost between transfer methods at the same checkpoint. GPT, GLM, Kimi, and Qwen denote GPT-5.6 Sol, GLM-5.3, Kimi K3, and Qwen3.8-27B.
Figure 3: Accuracy and mean API cost across handoff progress on DeepSWE v1.1. Colors identify the capable model: GLM-5.3, Kimi K3, and GPT-5.6 Sol; the efficient model is Qwen 3.8 27B. Solid curves show handoffs using logfile transfer in (a–b); in (c–d), GLM-5.3 and Kimi K3 use in-context transfer, while GPT-5.6 Sol uses logfile transfer. Horizontal dashed lines ( ) show the corresponding capable-only baselines; denotes Qwen-only execution. Progress is relative to the source model’s completed baseline trajectory; 0% denotes receiving-model-only execution.
Figure 4: Unsuccessful and successful capable-to-efficient handoffs. Each row shows an earlier checkpoint, the capable model’s intervening progress, and the next checkpoint. Left and right boxes report separate Qwen 3.8 27B continuations. Percentages refer to the capable-only trajectory.
Capable model
Handoffs
Acc. (%)
Cost ($)
GLM-5.3
6/113
54.9
2.54
Kimi K3
0/113
64.6
4.89
GPT-5.6 Sol
113/113
62.8
3.75
Table 2: Prompt-based handoff to Qwen 3.8 27B on DeepSWE v1.1. Handoffs reports the number of tasks with a recorded handoff; cost is mean API cost/task.
Figure 5: Online detection in AgentBaton with separate, growing message histories.
DeepSWE v1.1
Terminal-Bench 2.1
Capable model
Efficient model
Handoff (%)
Acc. (%) ↑
Cost ()\downarrow$
Handoff (%)
Acc. (%) ↑
Cost ()\downarrow$
—
Qwen 3.8 27B
—
29.2
0.00
—
69.7
0.00
—
GPT-5.6 Luna
—
60.2
0.40
—
82.2
0.07
GPT-5.6 Sol
—
—
68.1
5.32
—
87.6
0.97
Qwen 3.8 27B
26.6
60.2 -11.6%
1.02 -80.8%
58.4
82.0 -6.4%
0.43 -55.7%
GPT-5.6 Luna
67.3 -1.2%
1.39 -73.9%
82.0 -6.4%
0.47 -51.5%
Table 3: AgentBaton results. We use logfile transfer when the source model is GPT-5.6 Sol or GPT-5.6 Luna, and in-context transfer for all other source models. Subscripts give relative changes from capable-only: green is better, red worse. Handoff is the mean detected progress (%), shared across receiving models for a given source. Dashes denote single-model runs.
DeepSWE v1.1
Terminal Bench 2.1
Method
Acc. (%)
Cost ($)
Acc. (%)
Cost ($)
Capable: GLM 5.3; Efficient: Qwen 3.8 27B
MTRouter
58.4
3.67
80.9
1.35
Fixed progress
59.3
0.52
78.7
0.38
Fixed turn
54.9
0.48
76.4
0.18
AgentBaton
58.4
0.49
80.9
0.27
Table 4: AgentBaton versus routing and fixed handoff baselines with Qwen 3.8 27B. Fixed thresholds use the mean progress or turn under AgentBaton for each setting. Accuracy is in percent; cost is mean API cost per task.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Accuracy and mean API cost across handoff progress on Terminal-Bench 2.1. Colors identify the capable model: GLM-5.3, Kimi K3, and GPT-5.6 Sol; the efficient model is Qwen 3.8 27B. Solid curves show handoffs using logfile transfer in (a–b); in (c–d), GLM-5.3 and Kimi K3 use in-context transfer, while GPT-5.6 Sol uses logfile transfer. Horizontal dashed lines ( ) show the corresponding capable-only baselines; denotes Qwen-only execution. Progress is relative to the source model’s completed baseline trajectory; 0% denotes receiving-model-only execution.
DeepSWE v1.1
Terminal-Bench 2.1
Capable model
Efficient model
Transfer
Acc. (%)
Cost ($)
Acc. (%)
Cost ($)
GPT-5.6 Sol
Qwen 3.8 27B
In-context
54.9
1.03
85.4
0.43
Logfile
60.2
1.02
82.0
0.43
GPT-5.6 Luna
In-context
61.9
1.27
84.3
0.47
Logfile
67.3
1.39
82.0
0.47
Claude Opus 5
Qwen 3.8 27B
In-context
61.9
2.61
87.6
1.13
Appendix
Table 5: Complete AgentBaton results for both context-transfer methods. Each row reports accuracy and mean API cost per task from the same configuration.
Single model
AgentBaton → Qwen
AgentBaton → Luna
Source model
Effort
Acc.
Cost
Acc.
Cost
Acc.
Cost
GPT-5.6 Sol
low
35.4
0.65
47.8
0.20
65.5
0.60
medium
51.3
1.26
54.9
0.30
68.1
0.80
high
62.8
1.88
57.5
0.50
62.8
0.90
xhigh
71.7
2.79
59.3
0.70
69.0
1.10
max
71.7
4.84
61.9
1.00
70.8
1.40
Appendix
Table 6: Complete numerical results for the reasoning-effort sweep on DeepSWE v1.1 in Figure 1 . Effort refers to the source model. Accuracy is in percent and cost is mean API cost in USD per task. Dashes indicate configurations not evaluated in this sweep.
Autonomous agents powered by large language models (LLMs) perform complex tasks through long-horizon reasoning and tool interaction, where a fundamental trade-off arises between execution efficiency and reasoning robustness. Models at different capability-cost levels offer complementary advantages: lower-cost models enable fast execution but may struggle on difficult reasoning segments, while stronger models provide more robust reasoning at higher computational cost. We present AgentCollab, a self-driven collaborative inference framework that dynamically coordinates models with different reasoning capacities during agent execution. Instead of relying on external routing modules, the framework uses the agent's own self-reflection signal to determine whether the current reasoning trajectory is making meaningful progress, and escalates control to a stronger reasoning tier only when necessary. To further stabilize long-horizon execution, we introduce a difficulty-aware cumulative escalation strategy that allocates additional reasoning budget based on recent failure signals. In our experiments, we instantiate this framework using a two-level small-large model setting. Experiments on diverse multi-step agent benchmarks show that AgentCollab consistently improves the accuracy-efficiency Pareto frontier of LLM agents.
Enterprise agentic systems that route every trajectory step to a frontier model waste 60-80% of their inference budget on subtasks that smaller models handle equally well. Existing routing solutions optimize single-turn query assignment but ignore a property unique to agentic workflows: subtask complexity varies widely within a single trajectory. A planning step may require frontier-class reasoning while a subsequent formatting step needs only a 7B model. We formalize step-level model routing as a sequential assignment problem over agent trajectories and propose AgentRouter, a lightweight classifier (12M parameters, <5ms overhead per step on an A100 GPU) that maps each trajectory step to one of four model tiers using five features extractable at routing time. Trained on 50,000 annotated agent trajectory steps spanning planning, coding, research, and data analysis tasks, AgentRouter achieves 72% cost reduction relative to frontier-only baselines, retaining 97.3% of frontier-only quality (less than 3% degradation in end-to-end task completion); per-step routing accuracy reaches 91% on minimal-complexity steps and 85% on efficient-tier steps, with 76-82% on the harder mid-range and frontier tiers. On the same benchmarks, RouteLLM and FrugalGPT (applied per-step) achieve only 31% and 44% cost reduction respectively, because their single-turn training signal misses trajectory-level quality dependencies.
Rudrendu Kumar Paul, Sourav Nandy
Boston University, Boston, MA, USA · University of Texas at Austin, Austin, TX, USA
Practical deployment of large language model (LLM) agents requires strong task performance at affordable inference cost. For long-horizon agentic tasks, this performance-cost trade-off can be improved through within-task large-small model collaboration, as smaller models can handle some stages even when they cannot solve the full task. In this paper, we introduce RSI-router, a routing framework that constructs subtask-level model assignments and model-specific skills through recursive self-improvement over accumulated experience. Each iteration consists of four stages: Subtask Mining derives subtask definitions and identification rules from training trajectories; Routing Strategy Evolution proposes and evaluates diverse model assignments; Model-Specific Skill Evolution compares routed and large-model-only trajectories to diagnose failures and develop reusable execution skills; and Pareto-Optimal Router Selection updates the Pareto population using historical and newly generated routers while retaining dominated routers as experience for subsequent evolution. Routing between DeepSeek-V4.1-Flash and Qwen3.5-9B, RSI-router consistently surpasses the DeepSeek-only baseline at roughly half the inference cost (48.3%) across five agentic benchmarks. In particular, on ALFWorld, ScienceWorld, and WebShop, it cuts inference cost by 74.7-82.2% while simultaneously improving performance; on Terminal-Bench 2.0, it achieves a 16.7% relative performance gain at 18.0% lower cost. Moreover, RSI-router establishes a stronger performance--cost Pareto frontier than 9 routing methods.
Hao Li, Hangfan Zhang, Zhiyao Cui +5
Shanghai Artificial Intelligence Laboratory · Northwestern Polytechnical University