Deploying small language models (SLMs) on offline, resource-constrained edge devices such as remote sensing satellites presents a fundamental challenge: their limited reasoning capacity hinders reliable execution of multi-step agent tasks requiring complex tool orchestration. Existing plan-solve paradigms rely on prompt-based enforcement, which our experiments show SLMs almost entirely disregard: weak models fail to invoke the plan. We propose PTC-Decoder (Plan-Tool Constrained Decoder), a training-free, plug-and-play decoder framework that combines (1) a Plan-to-Act paradigm, which elevates planning to an atomic tool and forces its invocation at the first inference step, and (2) TC-Decoder, a deterministic finite automaton that imposes token-level hard constraints on tool names while preserving freedom over parameter generation, thereby retaining SLM reasoning capability. Evaluated on 200 real remote-sensing satellite tasks across 7 SLMs, PTC-Decoder yields a statistically significant mean overall score gain of +1.21 (p<0.01), 95% CI [+1.13, +1.29]), with consistent improvements across models and other datasets. An ablation study that removes TC-Decoder causes substantial performance degradation across all quality metrics without reducing computational cost, confirming TC-Decoder as the primary driver. PTC-Decoder thus offers a lightweight yet effective solution for improving step-level reliability, with final-answer accuracy remaining an open challenge. In essence, we enforce plan adherence by constraining the permissible output vocabulary during inference, without requiring retraining.
Figures & tables
Figure 1: Illustration of the in - orbit task processing flow for a satellite with an onboard agent.
Baseline
Prompt-Plan
Model
P.R.
Suc.
P.R.
Suc.
Qwen3-1.7B
.10
1.00
.88
.71
Qwen3.5-2B
.00
.79
.13
.86
Qwen3.5-0.8B
.07
.92
.20
.99
Qwen3-0.6B
.03
.99
.47
1.00
Gemma-4-2B(Q4)
.00
1.00
.66
1.00
Table 1: Prompt-Plan enforcement versus baseline. The former adds MUST call plan first into system prompt instructing the model to call plan . P.R.: plan call rate; Suc.: success rate, higher the value, fewer times the agent crashes. Weaker models completely ignore the prompt (P.R. = 0), and even capable models show degraded performance.
Figure 2: Example of the Expected Complex Task Workflow in the Remote Sensing Satellite Agent System.
Figure 3: Our complete PTC-Decoder architecture
Baseline
PTC-Decoder (Ours)
Plan w/o TC-Decoder
Model
H.
Ov.
R.A.
F.R.
Rb.
H.
Ov.
R.A.
F.R.
Rb.
H.
Ov.
R.A.
F.R.
Rb.
Qwen3-1.7B
2.05
1.98
0.87
2.12
2.71
3.10
2.94
2.32
2.97
3.06
2.45
2.67
1.34
2.94
3.03
Qwen3.5-2B
1.70
1.28
0.41
1.19
2.25
3.35
3.12
2.81
3.27
2.73
1.50
1.34
0.30
1.41
2.22
Qwen3.5-0.8B
1.05
1.15
0.42
0.96
2.04
2.55
2.29
1.73
2.31
2.67
1.55
1.51
0.53
1.64
2.17
Qwen3-0.6B
0.95
1.27
0.74
1.15
2.02
2.40
2.23
1.60
2.23
2.44
1.10
1.27
0.23
1.05
2.63
Gemma-4-2B(Q4)
1.90
1.71
0.95
1.69
2.55
3.25
3.05
2.42
3.31
2.60
1.55
1.36
0.22
1.45
2.64
Table 2: LLM-judge scores across experimental conditions, with a Human double-check (H.) for each group (20 randomly sampled tasks per group, rated on the same [0,5] by two annotators). The best LLM-judge value for each metric is bolded .
Baseline
PTC-Decoder (Ours)
Plan w/o TC-Decoder
Model
R.
P.
F1
Tokens
R.
P.
F1
Tokens
R.
P.
F1
Tokens
Qwen3-1.7B
.187
.413
.249
3437+181
.305
.295
.292
7018+399
.252
.647
.355
7018+370
Qwen3.5-2B
.123
.137
.123
3424+185
.388
.284
.318
7229+646
.147
.214
.162
7000+509
Qwen3.5-0.8B
.104
.180
.125
3424+137
.229
.237
.227
7000+425
.171
.325
.212
7000+421
Qwen3-0.6B
.148
.167
.155
3437+112
.250
.249
.246
7018+356
.000
.000
.000
7018+262
Gemma-4-2B(Q4)
.246
.328
.279
3130+112
.379
.361
.359
6435+868
.097
.128
.109
6435+422
Table 3: Recall, Precision, F1 (per-task average), and average tokens (input + output). The best value for each metric is bolded .
Figure 4: Recall, Precision, and F1 across three external datasets (means over 7 SLMs). PTC-Decoder consistently improves all metrics. Plan w/o TC-Decoder shows inconsistent improvement, remaining close to baseline. Error bars omitted for clarity; within-group F1 SD ranges: ToolAlpaca 0.18–0.38, Seal-Tools 0.17–0.44, API-Bank 0.11–0.37. ∗p<0.05 , †p=0.06 (Wilcoxon signed-rank test, n=7 , PTC-Decoder vs. Baseline F1: ToolAlpaca p=0.03 , Seal-Tools p=0.03 , API-Bank p=0.06 ).
Small language models (SLMs) have emerged as efficient alternatives to large language models for task-specific applications. However, they are often employed in high-volume, low-latency settings, where efficiency is crucial. We propose TASC, Task-Adaptive Sequence Compression, a framework for SLM acceleration comprising two use-cases: When performing SLM fine-tuning, we propose TASC-ft, which iteratively enriches the tokenizer vocabulary with high-frequency output n-grams and then fine-tunes the model to utilize the expanded vocabulary. Next, we propose an inference-time method, termed TASC-spec. TASC-spec is a lightweight, training-free speculative decoding method that constructs an n-gram draft model from the task's output corpus, mixing task and context n-gram information.TASC-spec avoids any additional training, while bypassing draft-target vocabulary alignment constraints. We demonstrate the effectiveness of both methods across multiple low output-variability generation tasks. Our methods show consistent improvements in inference efficiency while maintaining task performance.
Small language models (SLMs) are attractive as local agent controllers because they reduce remote inference, latency, and deployment footprint, yet structured tool errors can cause an agent step to fail. Existing routers typically select a model once per query. However, agents expose sequential decision points whose difficulty dynamically changes based on intermediate observations. We propose STEPGATE, an uncertainty-aware handoff framework that scores each local SLM action and selectively escalates challenging steps to a stronger model. On a 52-task held-out single-step BFCL-derived test split, the Qwen2.5-1.5B/7B pair attains 82.7% task success with 30.8% escalation, versus 67.3% local-only and 75.4% random escalation (which uses 33.8% escalation). In a separate multi-turn evaluation, STEPGATE achieves 69.0% trajectory success and 84.0% action success using only 30.0% cloud actions, compared with 48.0%/70.5% local-only, 60.0%/78.2% random escalation, and 57.0%/77.1% query-level routing (strong-only achieves 82.0% trajectory success at 100% cloud actions). These results suggest that step-level escalation recovers a large share of the performance gap to the stronger Qwen2.5-7B backend at a matched cloud-action rate while transmitting fewer tokens remotely. However, our evaluation is limited to one model family, a single stronger backend, and scripted tasks. Furthermore, the test sets are small, multi-turn comparisons rely on paired intervals and statistical tests, and our risk tiers serve as research annotations rather than formal safety guarantees.
Abolfazl Younesi
Department of Computer Engineering Sharif University of Technology Tehran, Iran
Recent advances in large language models (LLMs) provide robots with contextual reasoning abilities to comprehend human instructions. Yet, current LLM-enabled robots typically depend on cloud-based models or high-performance computing infrastructure, which limit their deployment on robots under unreliable internet environments or with constrained computational resources, such as UAVs and small ground vehicles. Thus, deploying fine-tuned small language models (SLMs) that support onboard deployment offers a promising alternative. This paper introduces Ro-SLM, a framework that enables reliable SLM-driven robot operation by distilling LLMs' knowledge and reasoning. Ro-SLM starts from dataset synthesis by leveraging LLMs to generate diverse task instructions, produce corresponding ground truth code with minimal human assistance, and augment instructions into real-world application scenarios. Ro-SLM is then fine-tuned with the dataset, in which LLM serves as a reward function to guide the training. Extensive experiments on UAV operation tasks demonstrate that Ro-SLM improves the performance of SLM from being incapable of supporting robotic task planning and code generation to achieving performance that approaches LLM.
Wenhao Wang, Yanyan Li, Long Jiao +1
Department of Computer & Information Science, University of Massachusetts Dartmouth · Department of Computer Science & Engineering, California State University San Marcos