Small Language Models

Also known as SLM

Ask about this topic

Overview

Researchers build small language models by training compact models from scratch, distilling larger teachers, or adapting open models with supervised fine-tuning, LoRA/QLoRA, preference optimization, and quantization. To extend their capabilities, systems combine them with retrieval, tools, structured reasoning, verification, or selective handoff to larger models, particularly for edge, private, and latency-sensitive use. Evaluations span reasoning and coding benchmarks such as GSM8K and SWE-bench, structured tasks such as Spider and BIRD text-to-SQL, and domains including medicine, law, and low-resource languages. Specialized small models can be efficient and competitive, but results depend on the task: grounding retrieved context, reliable multi-step reasoning, safety, and generalization remain challenges, alongside trade-offs among accuracy, latency, memory, energy, and cost.

Latest papers

CardsList

277 papers42 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

  1. MemFlow: Intent-Driven Memory Orchestration for Small Language Model Agents

    May 5, 2026Jiayi Chen, Yingcong Li, Guiling WangLLM Agent Workflow OptimizationPersistent Memory for Language Models

    New Jersey Institute of Technology, Newark, USA

  2. Terminus-4B: Can a Smaller Model Replace Frontier LLMs at Agentic Execution Tasks?

    May 4, 2026Spandan Garg, Vikram Nitin, Yufan HuangAI Coding AgentsSmall Language Models

    Microsoft USA

  3. Benchmarking Local Language Models for Social Robots using Edge Devices

    May 4, 2026Dorian Lamouille, Matevž B. Zorec, Farnaz Baksh +1LLM EvaluationOn-Device Language Model Inference

    University of Tartu Tartu, Estonia · Institute of Computer Science University of Tartu Tartu, Estonia · Institute of Technology University of Tartu Tartu, Estonia

  4. When Correct Isn't Usable: Improving Structured Output Reliability in Small Language Models

    May 4, 2026Cosimo Galeone, Minsu Park, Giuseppe Ettorre +1LLM EvaluationLLM Input Optimization

    Alomana, Grottaglie, Italy

  5. RadLite: Multi-Task LoRA Fine-Tuning of Small Language Models for CPU-Deployable Radiology AI

    May 1, 2026Pankaj Gupta, Kartik BoseLLM Fine-TuningSmall Language Models

    Postgraduate Institute of Medical Education and Research, Chandigarh, India ∗Corresponding author: Additional Professor, Department of Radiodiagnosis and Imaging, Postgraduate Institute of Medical Education and Research, Chandigarh, India · Postgraduate Institute of Medical Education and Research, Chandigarh, India

  6. RSAT: Structured Attribution Makes Small Language Models Faithful Table Reasoners

    Apr 30, 2026Jugal Gajjar, Kamalasankari SubramaniakuppusamyLLM ReasoningLLM Grounding

    Department of Computer Science, The George Washington University, USA

  7. Beyond Semantics: Measuring Fine-Grained Emotion Preservation in Small Language Model-Based Machine Translation

    Apr 30, 2026Dawid Wisniewski, Igor CzudySmall Language ModelsMachine Translation

    Pozna´n University of Technology

  8. Select to Think: Unlocking SLM Potential with Local Sufficiency

    Apr 29, 2026Wenxuan Ye, Yangyang Zhang, Xueli An +2Small Language ModelsEfficient Language Model Reasoning

    1Technical University of Munich · 3Ludwig Maximilian University of Munich · 2Huawei Technologies +2

  9. Quantifying Prior Dominance in RAG Systems

    Apr 29, 2026Barak OrLLM GroundingKnowledge Conflicts in Language Models

    ArtificialGate Ltd. · Google & Reichman Tech School · Technion – Israel Institute of Technology +1

  10. Domain-Adapted Small Language Models for Reliable Clinical Triage

    Apr 29, 2026Manar Aljohani, Brandon Ho, Kenneth McKinley +2Clinical TriageLLM Fine-Tuning

    Department of Computer Science, Virginia Tech, Blacksburg, VA, US · Children’s National Hospital, Washington DC, USA

  11. Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs

    Apr 27, 2026Sagnik Chatterjee, Atharva Patil, Sricharan RameshCoT ReasoningSmall Language Models

  12. Less Is More: Engineering Challenges of On-Device Small Language Model Integration in a Mobile Application

    Apr 27, 2026William OliveiraOn-Device Language Model InferenceSmall Language Models

    Independent Researcher

  13. Nautile-370M: Spectral Memory Meets Attention in a Small Reasoning Model

    Apr 27, 2026Maixent ChenebauxSelf-AttentionSmall Language Models

    Trickstr

  14. LegalDrill: Diagnosis-Driven Synthesis for Legal Reasoning in Small Language Models

    Apr 26, 2026Tianchun Li, Haochen Liu, Vishwa Pardeshi +5Synthetic Data GenerationSmall Language Models

    Purdue University, West Lafayette, IN, USA · Fidelity Investments, Boston, MA, USA

  15. Evolve: A Persistent Knowledge Lifecycle for Small Language Models

    Apr 25, 2026Dikran HovagimianRetrieval-Augmented GenerationMemory-Augmented Language Models

    Independent Researcher

  16. Meta-Tool: Efficient Few-Shot Tool Adaptation for Small Language Models

    Apr 22, 2026Sachin KumarSmall Language ModelsFew-Shot Prompting

    LexisNexis, USA

  17. Micro Language Models Enable Instant Responses

    Apr 21, 2026Wen Cheng, Tuochao Chen, Karim Helwani +3On-Device Language Model InferenceSmall Language Models

    Paul G. Allen School of Computer Science & Engineering, University of Washington · Meta AI

  18. Rethinking Scale: Deployment Trade-offs of Small Language Models under Agent Paradigms

    Apr 21, 2026Xinlin Wang, Mats BrorssonMulti-Agent LLM SystemsTool-Augmented Language Model Agents

    Proximus Luxembourg S.A. · rue du Puits Romain · L-8070 Bertrange, Luxembourg +3

  19. Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes

    Apr 20, 2026Justin Bauer, Thomas Walshe, Derek Pham +4RL for Language Model ReasoningSmall Language Models

    Snorkel AI · University of Wisconsin-Madison · University of Oxford

  20. Generalization Boundaries of Fine-Tuned Small Language Models for Graph Structural Inference

    Apr 20, 2026Michal PodstawskiSupervised Fine-TuningSmall Language Models

    NASK National Research Institute Warsaw, Poland

  21. Local-First Intelligence via Proactive SLM Collaborating with Feedback-Providing LLM

    Apr 20, 2026Hang Zeng, Xiangyu Liu, Yong Hu +5LLM Inference EfficiencySmall Language Models

    Shanghai Jiao Tong University, Shanghai, China · WeChat Tencent, Beijing, China · State University of New York at Buffalo, New York, United States

  22. Bridging the Reasoning Gap in Vietnamese with Small Language Models via Test-Time Scaling

    Apr 20, 2026Bui The Trung, Do Minh Duc, Nguyen Van Vinh +1Supervised Fine-TuningMathematical Reasoning Benchmarks

    Computer Science and Engineering VNU Vietnam Japan University Hanoi, Vietnam · Information Science JAIST Ishikawa, Japan · Faculty of Information VNU University of Engineering and Technology Hanoi, Vietnam +1

  23. Compiling Deterministic Structure into SLM Harnesses

    Apr 19, 2026Zan Kai Chong, Hiroyuki Ohsaki, Bryan NgLarge Language Model-Guided OptimizationSmall Language Models

    School of Science and Technology Kwansei Gakuin University Japan · School of Engineering & Computer Science Victoria University of Wellington New Zealand

  24. The Cognitive Penalty: Ablating System 1 and System 2 Reasoning in Edge-Native SLMs for Decentralized Consensus

    Apr 18, 2026Syed Muhammad Aqdas RizviSmall Language ModelsAdversarial Robustness of Language Models

    Independent Researcher Karachi, Pakistan

  25. PEFT of SLM for Telecommunications Customer Support: A Comparative Study of LoRA Configurations with Energy Consumption Analysis

    Apr 17, 2026Lucas Tamic, Ilan Jaffeux-Cheniout, Xavier MarjouCustomer Support AutomationSmall Language Models

    Orange, France

  26. TinyMU: A Compact Audio-Language Model for Music Understanding

    Apr 17, 2026Xiquan Li, Aurian Quelennec, Slim EssidMusic UnderstandingSmall Language Models

    LTCI, T´el´ecom Paris, Institut Polytechnique de Paris · Shanghai Jiao Tong University

  27. Improving Reasoning Capabilities in Small Models through Mixture-of-Layers Distillation with Stepwise Attention on Key Information

    Apr 17, 2026Yao Chen, Jiawei Sheng, Wenyuan Zhang +1CoT DistillationSmall Language Models

    Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences

  28. HintMR: Eliciting Stronger Mathematical Reasoning in Small Language Models

    Apr 14, 2026Jawad Hossain, Xiangyu Guo, Jiawei Zhou +1LLM Mathematical ReasoningSmall Language Models

    University at Albany · University at Buffalo · Stony Brook University

  29. EdgeCIM: A Hardware-Software Co-Design for CIM-Based Acceleration of Small Language Models

    Apr 13, 2026Jinane Bazzi, Mariam Rakka, Fadi Kurdahi +2On-Device Language Model InferenceEfficient Language Model Inference

    University of California Irvine, Irvine, CA, USA. · Rain Neuromorphics Inc., San Francisco, CA, USA

  30. Ro-SLM: Onboard Small Language Models for Robot Task Planning and Operation Code Generation

    Apr 13, 2026Wenhao Wang, Yanyan Li, Long Jiao +1Code Generation for RoboticsSmall Language Models

    Department of Computer & Information Science, University of Massachusetts Dartmouth · Department of Computer Science & Engineering, California State University San Marcos