Small Language Models

Also known as SLM

Researchers build small language models by training compact models from scratch, distilling larger teachers, or adapting open models with supervised fine-tuning, LoRA/QLoRA, preference optimization, and quantization. To extend their capabilities, systems combine them with retrieval, tools, structured reasoning, verification, or selective handoff to larger models, particularly for edge, private, and latency-sensitive use. Evaluations span reasoning and coding benchmarks such as GSM8K and SWE-bench, structured tasks such as Spider and BIRD text-to-SQL, and domains including medicine, law, and low-resource languages. Specialized small models can be efficient and competitive, but results depend on the task: grounding retrieved context, reliable multi-step reasoning, safety, and generalization remain challenges, alongside trade-offs among accuracy, latency, memory, energy, and cost.

Latest papers

276 papers42 papers in the last four weeks, up 147% on the four weeks before. 0.2% of all new papers.

Evidence-Grounded SLM Behavioral Prediction Architecture for Future Assistance Use
Oct 23, 2026Adeyemi Adeseye, Aisvarya Adeseye, Hannu Tenhunen +1Human Behavior Prediction Human Behavior Modeling
Department of Computing, University of Turku, Turku, Finland · Electronic System Design, Royal Institute of Technology, Stockholm, Sweden
Training Small Language Models for Transferable Tool Selection
Oct 22, 2026Zhaoyi Zhou, Akshayaa Magesh, Jeongyeol Kwon +4Tool Selection for Language Model Agents Small Language Models
Meta · Carnegie Mellon University
GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation
Oct 21, 2026Quang Anh Kiet Tran, Nam Anh Tran, Manh Hung Trinh +3Small Language Models Language Modeling
UIT, VNU-HCM · TwoHearts Technologies Co., Ltd., Ho Chi Minh City, Vietnam · IU, VNU-HCM +3
OmniSimulator: Aligning Small Language Models for Authentic Heterogeneous Behavior Modeling
Oct 21, 2026Jiawei Chen, Ruoxi Xu, Boxi Cao +12User Simulation Human Behavior Simulation
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Kuaishou Technology
SLoSa: Small, Low-Power, and Safe Foundation Models for Robotic Task Planning
Oct 20, 2026Weizhe Xu, Jialiang Fan, Essien Taylor +3Robot Task Planning Large Language Model-Based Robot Planning
University of Notre Dame · Northwestern University
Context-Aware Prompt Injection Detection via Small Language Models in RAG Pipelines: nokast-secureRAG
Oct 20, 2026Abhishek RaiPrompt Injection Detection RAG Security
Independent Researcher
Bud ContextCompact 2.0: Context Compaction for Agent Success and Accuracy with Small Language Models and Constrained Context Windows
Oct 18, 2026Jithin VG, Ditto PSLLM Context Management LLM Context Compression
Bud Ecosystem
Quality-Controlled Reply and Summary Data for Post-Training Small Language Models in AI Call-Screening Agents
Oct 18, 2026Jiaqi Gan, Tsang Ng, Ankit Raj +5Small Language Models Spoken Dialogue Systems
Efficiency Measured in English: Tokeniser Coverage Bounds Small Language Models on Indic Scripts
Oct 17, 2026Anna PramodTokenization in Language Models Small Language Models
School of Computer Science, University of Leeds, United Kingdom
From Billion-Scale to Million-Scale Language Models: Operational Capabilities for Embedded Post-Exploitation
Oct 16, 2026Gabriel Hortea, Simon Pietro Romano, Juan Tapiador +1Small Language Models
Università degli Studi di Napoli Federico II, Naples, Italy · Universidad Carlos III de Madrid, Madrid, Spain
Embeddings Are Overpriced? A Frozen 16 MB Vocabulary for Small and Ternary Language Models
Oct 15, 2026Sai Kiran BathulaEmbedding Compression Small Language Models
QLNI, New South Wales, Australia
CARE-SD: Explainable Reasoning Architecture for Local SLM Self-Disclosure Analysis in Textual Data
Oct 15, 2026Aisvarya Adeseye, Adeyemi Adeseye, Mohammad Tahir +1Small Language Models Explainable Artificial Intelligence
Department of Computing, University of Turku, Turku, Finland
Maximising Transparency in Context Compaction Improves Retention of Facts and Commitments for Lightweight Coding Agents
Oct 14, 2026Felix WoolfordContext Compaction for LLM Agents Context Compression
Center for Human Nature, Artificial Intelligence, and Neuroscience (CHAIN), Hokkaido University
Progress-Aware Distillation for Mitigating Stagnation in Small Language Model Agents
Oct 14, 2026Yuanpu Cao, Saket Sathe, Hanyu Wang +3LLM Agent Training Language Model Distillation
The Pennsylvania State University · Microsoft CoreAI
ixi-sLM: On-Device sLM for AI Call Agent Service
Oct 14, 2026Han-Sang Lee, Kyuho Lee, Juneyoung Park +10On-Device Language Model Inference Small Language Models
LG Uplus · OptAI Inc.
Verification Turns Capability Gaps into Routing Decisions: Small Language Models for Clinical Documentation
Oct 13, 2026Nina Fatehi, Meysam Ghaffari, Animesh Agarwal +2Medical Report Generation Medical LLMs
Optum
Reliable Text-to-SQL in Resource-Constrained Settings with Small LLMs via Iterative Retry Workflows
Oct 12, 2026Bilel Abderrahmane Benziane, Xavier ScheplerText-to-SQL Small Language Models
Data Lab, Generix Group, Paris, France
Context-Aware Precision Scheduling for Small Language Models
Oct 11, 2026Mukund Athmaram, Manoj Rao, Liang MaEfficient Language Model Inference On-Device Language Model Inference
Advanced Micro Devices Inc. (AMD) · Cornell University
Iterate on an LLM, Serve on an SLM
Oct 10, 2026Keegan WangLLM Agent Training Small Language Models
Carnegie Mellon University · Phoebe
Grammar Concept Annotation at Scale: Deployed Fine-Tuned Small Language Models Outperform Prompted Frontier Models
Oct 7, 2026Marjan Celikik, Ana Peleteiro Ramallo, Javier MoralesSecond Language Acquisition Small Language Models
Preply
Skill-SLM: Agent Skill-driven Small Language Models for Reliable Robot Operation
Oct 7, 2026Wenhao Wang, Yanyan Li, Jiawei YuanSmall Language Models Robot Task Planning
Department of Computer & Information Science, University of Massachusetts Dartmouth · Department of Computer Science & Engineering, California State University San Marcos
Using Small Language Models to Reverse-Engineer Machine Learning Pipelines Structures
Oct 7, 2026Nicolas Lacroix, Frederic Precioso, Mireille Blay-Fornarino +1Small Language Models Software Engineering
Université Côte d’Azur, Inria, CNRS, I3S, Sophia Antipolis, France · McSCert, McMaster University, Hamilton, ON, Canada
Multi-Objective Aligned Small Language Model Framework for SUD Patient Dialogue Generation
Oct 6, 2026Thushara Manjari Naduvilakandy, Hyeju Jang, Mohammad Al HasanSmall Language Models User Simulation
Department of Computer Science, Indiana University Indianapolis, Indianapolis, IN, USA
Multi-Label Topic Assignment via LLM Distillation: A Comparative Analysis of Generative vs. Discriminative Student Models
Oct 6, 2026Sourabh Kasliwal, Shubhranshu SinghMulti-Label Classification Language Model Distillation
eBay Inc., Bellevue, WA · eBay Inc., Portland, OR
Not Every Call Needs a Frontier Model: Per-Call-Site Evaluation of Small Language Models in a Deployed Agentic Home-Automation System
Oct 6, 2026Panagiotis Kasnesis, Christos Chatzigeorgiou, Lazaros Toumanidis +1Small Language Models LLM Agent Evaluation
University of West Attica, Athens, Greece · Waldiez PC · ThinGenious PC
Tiny-Scale Chinese BERT Pretraining: A Controlled Comparison of MLM, WWM, and MacBERT Strategies
Oct 6, 2026Yiping BaiLanguage Model Pretraining Language Modeling
Guangdong Haiqixing Marine Technology Co., Ltd.
Do I Need the Cloud? Uncertainty-Aware Step-Level Handoff for Small Language Model Agents
Oct 6, 2026Abolfazl YounesiSmall Language Models LLM Agents
Department of Computer Engineering Sharif University of Technology Tehran, Iran
Small Language Models for Smart Data Model Classification at the Edge: A Cost-Aware Hybrid Approach
Oct 5, 2026Cristian Martella, Angelo Martella, Antonella Longo +1Small Language Models Semantic Mapping
Dept. of Engineering for Innovation, University of Salento, Via Per Monteroni, Lecce, 73100, Italy
Do Small Language Models Learn to Negotiate? A Controlled Scaling Study of RL-Trained Sellers
Oct 5, 2026Pedro Tabacof, Sagar JoglekarRL for Language Models Multi-Agent Negotiation
Fin AI Research
Small Agents with Semantic Search: Efficient Multilingual Code Localization
Oct 4, 2026Maxence Lasbordes, Aarush Sinha, Raphael Sourty +2AI Coding Agents Semantic Search
LightOn · Inria Paris · University of Copenhagen

Oct 23, 2026J/K move · Enter open · S save

Evidence-Grounded SLM Behavioral Prediction Architecture for Future Assistance Use

Adeyemi Adeseye, Aisvarya Adeseye, Hannu Tenhunen, Jouni Isoaho

Department of Computing, University of Turku, Turku, Finland · Electronic System Design, Royal Institute of Technology, Stockholm, Sweden

Human Behavior Prediction Human Behavior Modeling Small Language Models Selective Prediction

Open paper AskPDF arXiv

Users often exhibit changes in interaction behavior before explicitly seeking assistance, enabling systems to anticipate future assistance use. This is especially important in high-stakes environments, where delayed assistance can have operational, financial, or safety consequences. However, conventional Large Language Models (LLMs) are often unsuitable for continuous on-device behavioral prediction because of their computational and memory demands. Behavioral patterns are also highly individual and device-specific, motivating local processing that preserves privacy and data sovereignty. Small Language Models (SLMs) offer a promising alternative, but their ability to reason reliably over low-level sequential interaction events remains underexplored. This study introduces Evidence-Grounded Local Behavioral Prediction (ELBP), a four-layer architecture combining deterministic interaction processing, dual-view behavioral representation, SLM-based prediction, and evidence verification with abstention. Three instruction-tuned models of 3B parameters or fewer were evaluated using raw, structured, dual-view, and complete ELBP configurations. The best-performing model, Qwen2.5-3B with ELBP, achieved a full-set F1-score of 0.80 and PR-AUC of 0.84, representing a 15.9% relative improvement over raw-event prompting and 17.6% over XGBoost, while full-set ECE decreased by 43.2% relative to raw-event prompting. With evidence verification and abstention, selective F1 increased to 0.84 at 91% coverage and evidence support reached 96%. These results show that structured behavioral representation and evidence-verified selective prediction can improve the reliability of behavioral forecasting in SLMs.