Small Language Models

Also known as SLM

Researchers build small language models by training compact models from scratch, distilling larger teachers, or adapting open models with supervised fine-tuning, LoRA/QLoRA, preference optimization, and quantization. To extend their capabilities, systems combine them with retrieval, tools, structured reasoning, verification, or selective handoff to larger models, particularly for edge, private, and latency-sensitive use. Evaluations span reasoning and coding benchmarks such as GSM8K and SWE-bench, structured tasks such as Spider and BIRD text-to-SQL, and domains including medicine, law, and low-resource languages. Specialized small models can be efficient and competitive, but results depend on the task: grounding retrieved context, reliable multi-step reasoning, safety, and generalization remain challenges, alongside trade-offs among accuracy, latency, memory, energy, and cost.

Latest papers

277 papers42 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

DenseSteer: Steering Small Language Models towards Dense Math Reasoning
May 28, 2026Yang Ouyang, Shuhang Lin, Jung-Eun KimSmall Language Models Activation Steering
1North Carolina State University · 2Rutgers University
Apertus LLM Family Expansion via Distillation and Quantization
May 27, 2026Andrei Panferov, Davit Melikidze, Martin Jaggi +1LLM Quantization LLM Compression
ISTA · EPFL · ETH Zürich +1
CosmicFish-HRM: Adaptive Reasoning via Hierarchical Recurrent Mechanisms in Compact Language Models
May 27, 2026Venkat Akhil LakkapragadaSmall Language Models LLM Reasoning
Mistyoz AI Hyderabad, India
Can Hallucinations Be Useful? Solving Multi-Hop Questions With SLMs By Chaining System-I/II Reasoning
May 26, 2026Saptarshi Sengupta, Suhang WangMulti-Hop QA LLM Hallucination Mitigation
The Pennsylvania State University
Second Guess: Detecting Uncertainty Through Abstention and Answer Stability in Small Language Models
May 25, 2026Ashwath Vaithinathan Aravindan, Mayank KejriwalConfidence Estimation in Language Models Multiple-Choice Question Answering
University of Southern California Los Angeles, 90007, California, United States of America · Information Sciences Institute 4676 Admiralty Way #1001, Los Angeles, 90292, California, United States of America
A Unified Structured Query Understanding Framework for Industrial Semantic Search
May 22, 2026Ping Liu, Qianqi Shen, Jianqiang Shen +15Semantic Search Schema-Constrained Generation
LinkedIn Corporation Mountain View, CA, USA
HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval
May 22, 2026Vipul Gupta, Shikhar Mohan, Lakshya Kumar +4Online Advertising Small Language Models
Microsoft AI India
PACE: Two-Timescale Self-Evolution for Small Language Model Agents
May 21, 2026Chen Ling, Pei Chen, Albert Guan +4Small Language Models LLM Agent Workflow Optimization
Amazon
The Constraint Tax: Measuring Validity-Correctness Tradeoffs in Structured Outputs for Small Language Models
May 20, 2026Jaideep RayConstrained Decoding Small Language Models
The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models
May 20, 2026Ming LiuNumerical Reasoning in Language Models CoT Faithfulness
Amazon
Internalizing Tool Knowledge in Small Language Models via QLoRA Fine-Tuning
May 18, 2026Yuval Shemla, Ayal Yakobe, Tanmay Agarwal +2Small Language Models LLM Tool Use
Columbia School of General Studies, Columbia University, NY, USA · Columbia Engineering, Columbia University, NY, USA · IBM Research, NY, USA
Exploring Lightweight Large Language Models for Court View Generation
May 16, 2026Zhitian Hou, Tianyong Hao, Nanli Zeng +2LLM Evaluation Small Language Models
School of Computer Science and Engineering, Sun Yat-sen University · School of Computer Science, South China Normal University · 3China Mobile Internet Co., Ltd.
CompactQE: Interpretable Translation Quality Estimation via Small Open-Weight LLMs
May 15, 2026Kamil Guttmann, Zofia Fraś, Artur Nowakowski +1Small Language Models MT Evaluation
Laniqo, Pozna´n, Poland · Faculty of Mathematics and Computer Science, Adam Mickiewicz University, Pozna´n, Poland
Small, Private Language Models as Teammates for Educational Assessment Design
May 14, 2026Chris Davis Jaldi, Anmol Saini, Shan Zhang +3LLM Evaluation LLM-as-a-Judge
Wright State University, USA · University of Florida, USA · TIB – Leibniz Information Centre for Science and Technology, Germany
VectraYX-Nano: A 42M-Parameter Spanish Cybersecurity Language Model with Curriculum Learning and Native Tool Use
May 13, 2026Juan S. SantillanaSmall Language Models LLMs for Cybersecurity
Globant
Code-Guided Reasoning for Small Language Models: Evaluating Executable MCQA Scaffolds
May 12, 2026Prateek Biswas, Dhaval Patel, Vedant Khandelwal +2Multiple-Choice Question Answering Small Language Models
1IBM · 2Artificial Intelligence Institute at University of South Carolina
It's Not the Size: Harness Design Determines Operational Stability in Small Language Models
May 12, 2026Yong-eun ChoSmall Language Models
KailosLab, Seoul, Republic of Korea
SemChunk-C: Semantic Segmentation for C Code
May 12, 2026Boris Nazarov, Darya Frolova, Shaked Leibzirer +1Small Language Models Code Language Models
StoicLLM: Preference Optimization for Philosophical Alignment in Small Language Models
May 12, 2026Ishmam Khan, Sindhuja Thogarrati, Shuo ZhangLLM Alignment LLM Fine-Tuning
Tufts University · Bose Corporation
SOMA: Efficient Multi-turn LLM Serving via Small Language Model
May 11, 2026Xueqi Cheng, Qiong Wu, Zhengyi Zhou +3LLM Serving Small Language Models
Florida State University · AT&T Chief Data Office · Louisiana State University +1
Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights
May 11, 2026Saba Pourhanifeh, AbdulAziz AbdulGhaffar, Ashraf MatrawyDomain Adaptation Cybersecurity
Department of Systems and Computer Engineering Carleton University · School of Information Technology Carleton University
Can Language Models Analyze Data? Evaluating Large Language Models for Question Answering over Datasets
May 11, 2026Andreas Xenofontos, Pavlos FafaliosTable QA LLM Evaluation
School of Production Engineering and Management, Technical University of Crete, Greece · Institute of Computer Science, Foundation for Research and Technology - Hellas, Greece.
Towards Compact Sign Language Translation: Frame Rate and Model Size Trade-offs
May 10, 2026Kuanwei Chen, Mengfeng TsaiSign Language Translation Small Language Models
Computer Science and Information Engineering, National Central University, Zhongli, Taiwan
Chain-based Distillation for Effective Initialization of Variable-Sized Small Language Models
May 8, 2026Boyu Shi, YiCheng Jiang, Chang Liu +3Small Language Models Cross-Architecture Knowledge Distillation
School of Computer Science and Engineering, Southeast University, Nanjing, China
Fine-Tuning Small Language Models for Solution-Oriented Windows Event Log Analysis
May 7, 2026Siraaj Akhtar, Saad Khan, Simon ParkinsonLLM Fine-Tuning Small Language Models
SMolLM: Small Language Models Learn Small Molecular Grammar
May 7, 2026Akhil Jindal, Harang JuSmall Language Models Mechanistic Interpretability
Moku
A Few Good Clauses: Comparing LLMs vs Domain-Trained Small Language Models on Structured Contract Extraction
May 7, 2026Nicole Lincoln, Nick Whitehouse, Jaron Mar +1Legal NLP Mixture-of-Experts Language Models
Onit AI Labs, Onit Inc., New Zealand
Self-Prompting Small Language Models for Privacy-Sensitive Clinical Information Extraction
May 5, 2026Yao-Shun Chuang, Tushti Mody, Uday Pratap Singh +6Named Entity Recognition Healthcare
McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States · School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States · School of Dentistry, The University of Texas Health Science Center at Houston, Houston, TX 77054, United States +1
MiniMind-O Technical Report: An Open Small-Scale Speech-Native Omni Model
May 5, 2026Jingyao GongSpeech Foundation Models Unified Multimodal Models
Independent Researcher
FINER-SQL: Boosting Small Language Models for Text-to-SQL
May 5, 2026Thanh Dat Hoang, Thanh Trung Huynh, Matthias Weidlich +4Reward Modeling Small Language Models
Griffith University (Australia) · VinUniversity (Vietnam) · Humboldt-Universitat zu Berlin (Germany) +1

May 28, 2026J/K move · Enter open · S save

DenseSteer: Steering Small Language Models towards Dense Math Reasoning

Yang Ouyang, Shuhang Lin, Jung-Eun Kim

1North Carolina State University · 2Rutgers University

Small Language Models Activation Steering LLM Mathematical Reasoning

Open paper AskPDF arXiv

Large language models (LLMs) demonstrate strong chain-of-thought (CoT) reasoning abilities, while smaller models (<= 3B parameters) significantly underperform on multi-step reasoning tasks. Based on empirical analyses of the Qwen-2.5 model family on math reasoning benchmarks, we find that more proficient reasoning is associated with fewer reasoning steps but higher information density per step, a property we term Dense Reasoning. Motivated by this observation, we propose DenseSteer, a training-free inference-time steering framework that enhances small-model reasoning by modulating internal representations toward dense reasoning patterns. Experiments show that our method yields consistent accuracy improvements without increasing token-level Negative Log-Likelihood, highlighting dense reasoning as an effective structural approach to mathematical problem solving.