Small Language Models

Also known as SLM

Overview

Researchers build small language models by training compact models from scratch, distilling larger teachers, or adapting open models with supervised fine-tuning, LoRA/QLoRA, preference optimization, and quantization. To extend their capabilities, systems combine them with retrieval, tools, structured reasoning, verification, or selective handoff to larger models, particularly for edge, private, and latency-sensitive use. Evaluations span reasoning and coding benchmarks such as GSM8K and SWE-bench, structured tasks such as Spider and BIRD text-to-SQL, and domains including medicine, law, and low-resource languages. Specialized small models can be efficient and competitive, but results depend on the task: grounding retrieved context, reliable multi-step reasoning, safety, and generalization remain challenges, alongside trade-offs among accuracy, latency, memory, energy, and cost.

Latest papers

CardsList

296 papers30 papers in the last four weeks, down 14% on the four weeks before. 0.2% of all new papers.

  1. ARCHER: From Rationale Imitation to Executable Reasoning in Small Language Models

    Nov 13, 2026Dhruv Dixit, Khushi Patel, Congcong Xu +1LLM ReasoningStructured Reasoning in Language Models

  2. Parameter-Efficient Conciseness Adaptation: Differentiable Length Control for Concise Clinical Reasoning in Smaller Language Models

    Nov 12, 2026Tanzeel Abbas, Ahmad Kamran Malik, Nafees Qamar +1clinical reasoning in language modelsMedical LLMs

  3. Mobile AI Factory: Building Specialized Language Models for Your Mobile Device

    Nov 10, 2026Evan Li, Krishna Deshpande, Wen Cheng +1On-Device Language Model Fine-TuningSmall Language Models

  4. Adapting Small Language Models to Romanized Nepali: Tokenizer Fertility, Reference Likelihood, and Generation Metrics

    Nov 9, 2026Pujan Pant, Sagar Newpane, Hari Sitaula +1Language ModelingSmall Language Models

  5. When Self-Consistency Hurts: The Hidden Cost of Majority Voting in Small Language Models

    Nov 8, 2026Wanrong Yang, Lingfang Li, Jun Zhang +1Self-Consistency in Language ModelsAdaptive Reasoning in Language Models

  6. How Far Can Lightweight LLMs Go in Intent-to-SLO Translation?

    Nov 5, 2026Sabbir Ahmed, Hazem Abbas, Salimur ChoudhuryIntent-Based NetworkingSmall Language Models

  7. Different Corpora, Different Skills? A Paired Study of Small Language Models

    Nov 5, 2026Edoardo Vaira, Georgios Chavales, Pierre NuguesLanguage Model PretrainingSmall Language Models

  8. Template Gap Reconstruction for Shell Output Simulation with Small Language Models

    Nov 4, 2026Jarrod Ragsdale, Rajendra BoppanaLanguage Model DecodingSmall Language Models

  9. AssameseLM-E2B: An Open-Weight Assamese Language Model for Efficient Consumer-Grade Deployment

    Nov 4, 2026Pragyan PhukanLanguage ModelingLow-Resource Language Processing

  10. MedPsy: State-of-the-Art Small Medical Language Models for Efficient Edge Deployment

    Nov 1, 2026Davide Vitabile, Alexandro Buffa, Akshay Nambiar +1Medical LLMsSmall Language Models

  11. Rubric-Based Assessment of Student Reports Using Small Language Models

    Nov 1, 2026Sandesh Kumar Kafle, Jan Vykopal, Valdemar ŠvábenskýAutomated GradingRubric-Based Evaluation

  12. Compression Technique Interactions in Small Language Models: A Factorial Ablation Study

    Oct 30, 2026Bhushit Nema, Siddharth Krishnamurthy, Anjali KadiyalaSmall Language ModelsLLM Quantization

  13. Latent-Lens: Visual Perception in Small Language Models Through Latent Communication

    Oct 30, 2026Mo Shahdloo, Faegheh Sardari, Mathew SalvarisSmall Language ModelsVision-Language Models

  14. Homebrewed SLMs: Autoregressive and Masked Diffusion Language Models Built at Home for Portuguese

    Oct 29, 2026Pedro Carvalho BromLanguage ModelingSmall Language Models

  15. Grammar Control for Language Learning with Sequential Monte Carlo and Small Language Models

    Oct 29, 2026Kate McCurdy, Jacob Hoover Vigly, Thomas Hikaru Clark +6Controllable Text GenerationSmall Language Models

  16. Blind Spots: Where Small Language Models Fall Behind Frontier Models on Terminal Tasks

    Oct 28, 2026Ashima Suvarna, Ruta Desai, Ahmed ElgoharySmall Language ModelsLLM Evaluation

  17. Evidence-Grounded SLM Behavioral Prediction Architecture for Future Assistance Use

    Oct 23, 2026Adeyemi Adeseye, Aisvarya Adeseye, Hannu Tenhunen +1Human Behavior PredictionHuman Behavior Modeling

  18. Training Small Language Models for Transferable Tool Selection

    Oct 22, 2026Zhaoyi Zhou, Akshayaa Magesh, Jeongyeol Kwon +4Tool Selection for Language Model AgentsSmall Language Models

  19. GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation

    Oct 21, 2026Quang Anh Kiet Tran, Nam Anh Tran, Manh Hung Trinh +3Small Language ModelsLanguage Modeling

  20. OmniSimulator: Aligning Small Language Models for Authentic Heterogeneous Behavior Modeling

    Oct 21, 2026Jiawei Chen, Ruoxi Xu, Boxi Cao +12User SimulationHuman Behavior Simulation

  21. SLoSa: Small, Low-Power, and Safe Foundation Models for Robotic Task Planning

    Oct 20, 2026Weizhe Xu, Jialiang Fan, Essien Taylor +3Robot Task PlanningLarge Language Model-Based Robot Planning

  22. From Billion-Scale to Million-Scale Language Models: Operational Capabilities for Embedded Post-Exploitation

    Oct 16, 2026Gabriel Hortea, Simon Pietro Romano, Juan Tapiador +1Small Language Models