Small Language Models

Also known as SLM

Overview

Researchers build small language models by training compact models from scratch, distilling larger teachers, or adapting open models with supervised fine-tuning, LoRA/QLoRA, preference optimization, and quantization. To extend their capabilities, systems combine them with retrieval, tools, structured reasoning, verification, or selective handoff to larger models, particularly for edge, private, and latency-sensitive use. Evaluations span reasoning and coding benchmarks such as GSM8K and SWE-bench, structured tasks such as Spider and BIRD text-to-SQL, and domains including medicine, law, and low-resource languages. Specialized small models can be efficient and competitive, but results depend on the task: grounding retrieved context, reliable multi-step reasoning, safety, and generalization remain challenges, alongside trade-offs among accuracy, latency, memory, energy, and cost.

Latest papers

CardsList

296 papers27 papers in the last four weeks, down 36% on the four weeks before. 0.2% of all new papers.

  1. Where A Small Language Model Helps in Invoice Categorisation, Understood Through Embedding Geometry

    Aug 18, 2026Emma Ceccherini, Daniel Lawson, Anjulika SalhanSmall Language ModelsText Classification

  2. Benchmarking the Benchmarks: Evaluating Automated Safety Benchmarks for Small Language Models

    Aug 17, 2026Nyamtulla Shaik, Fengjun Li, Bo LuoLanguage Model Safety EvaluationSmall Language Models

  3. Structured Driving-State Narratives for Small Language Model-Based GNSS Spoofing Detection

    Aug 17, 2026Abyad Enan, Sagar Dasgupta, Mizanur Rahman +1CybersecurityAdversarial Attacks

  4. Enhancing Virtual Agents through SLMs and Edge-Computing: An Exploratory Evaluation of Think and Memory Processes

    Aug 13, 2026Aimilios Hadjiliasi, Louis NisiotisCognitive Architectures for AI AgentsOn-Device Language Model Inference

  5. Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed

    Aug 12, 2026Haokun Lin, Kaijie Zhu, Haobo Xu +4LLM QuantizationLLM Pruning

  6. SKILLER: Language-Level Reinforcement Learning for Reusable Skill Extraction in Small Language Models

    Aug 11, 2026Chenhao Dang, Siyuan Xiong, Conghui He +1RL for Language ModelsSmall Language Models

  7. Locally Deployable Small Language Models for Emergency Department Decision Support: A Systematic Benchmark of Fine-Tuning Strategies

    Aug 10, 2026Qingfeng Zhang, Yuanxiong Guo, Yanmin GongLLM Fine-TuningClinical Decision Support

  8. Protoreasoning in Tiny Transformers

    Aug 5, 2026Eduardo Valle, Fergal ReidCoT ReasoningOOD Generalization

  9. Energy- and Memory-Efficient PEFT Methods for Personalized On-Device SLMs on Consumer GPUs

    Aug 5, 2026Kuanysh Akhmetzhanov, Jurn-Gyu ParkFine-TuningOn-Device Language Model Inference

  10. Antares: Foundation Models for Agentic Vulnerability Localization

    Aug 3, 2026Supriti Vijay, Aman Priyanshu, Didier Chapoteau +8Software Vulnerability DetectionSmall Language Models

  11. Gaokerena: A Small Persian Medical Language Model Family

    Aug 2, 2026Mehrdad Ghassabi, Hamidreza Baradaran Kashani, Pedram Rostami +5RL for Language Model ReasoningSmall Language Models

  12. MedLLM: An Open Medical Language Model at the Sub-Billion Scale

    Jul 29, 2026Maxx Richard Rahman, Asim Ahmed, Mihan Mohagheghzadeh +1LLM Fine-TuningSmall Language Models

  13. Towards Robust Reinforcement Learning for Small-Scale Language Model Agents

    Jul 27, 2026Md Rezwanul Haque, Md. Milon Islam, Fakhri KarrayRL for Language ModelsLLM Alignment

  14. Do Small Models Use the Law You Give Them? Context-Injected Fine-Tuning for Legal QA in Bangladesh

    Jul 26, 2026Moniruzzaman Mahadi, Abrar Mohammed Tanzim Alam, Sayma Siddika Monalisa +3Multilingual Language Model EvaluationLLM Grounding

  15. Offline Vision-Language Navigation with Geometric Goal Localization for Outdoor Environments

    Jul 24, 2026Ali Salmasi, Xianjia Yu, Tomi WesterlundSmall Language ModelsRobotics

  16. Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study

    Jul 23, 2026Ramesh B. ParamkushamLanguage Model Safety EvaluationDomain Adaptation

  17. Enhancing SLMs for Sustainable Code Optimization in Radio-Astronomy

    Jul 23, 2026Elisa Chiarotto, Jingbo Li, P. Chris Broekema +1Large Language Model-Guided OptimizationCode Generation

  18. Profiling Lightweight Large Language Models

    Jul 23, 2026Tomohiro Harada, Enrique Alba, Gabriel LuqueLLM Inference EfficiencyLLM Evaluation

  19. TINY_SCHILLER: A Drop-In German Drama Corpus for Small Language Models

    Jul 22, 2026Mark SchuteraSmall Language Models