D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models

Shunsuke Ubukata

cs.CLFeb 25, 2026

D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models

Authors: Shunsuke Ubukata

Organizations: Toyo University

Abstract

Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) often induces "overthinking" in Small Language Models (SLMs), leading to performance degradation and excessive token consumption. In this study, we propose Disciplined Chain-of-Thought (D-CoT), a novel framework that enforces a structured reasoning process using control tags -- such as <TEMP_LOW> for fact-checking and <TEMP_HIGH> for multi-perspective exploration -- as auxiliary scaffolding during training. By optimizing the CoT trajectory, D-CoT suppresses reasoning drift and simultaneously achieves token reduction and performance improvement. We demonstrate the efficacy of our approach on Qwen3-8B: with only 5,000 training samples, D-CoT significantly boosts accuracy on GPQA-diamond by 9.9% and MMLU-Pro (0-shot) by 9.1%, while drastically reducing computational costs. Furthermore, we confirm that the model internalizes this disciplined thought structure, maintaining high performance even without explicit control tags during inference.

Figures & tables

Control Tag	Role
<TEMP_LOW>	Fact-checking, listing prerequisites, and constraints.
<TEMP_MID>	Normal response, algorithmic processing, and calculation.
<TEMP_HIGH>	Creative solutions, multi-perspective viewpoints.

Table 1: Definition of Thinking Modes by Control Tags

Figure 1: Comparison of reasoning traces on MMLU-Pro #238 (promissory note discount problem, Qwen3-8B). (a) Conventional distilled CoT exhibits overthinking: the model reaches the correct answer but continues recalculating, ultimately exhausting the token budget and producing an incorrect prediction. (b) D-CoT with explicit control tags structures reasoning into fact organization ( <TEMP_LOW> ), multi-perspective exploration ( <TEMP_HIGH> ), and algorithmic computation ( <TEMP_MID> ), converging efficiently. (c) After D-CoT training, the model internalizes this disciplined structure and maintains organized reasoning without explicit tags, achieving the best accuracy on MMLU-Pro (64.73%).

Panel of Figure 2 — Figure 2: Cosine similarity distributions between D-CoT training samples and benchmark test sets. Left: MMLU-Pro (102 samples removed above the 0.55 threshold). Right: GPQA-Diamond (0 samples removed). The low similarity confirms that training domains are well-separated from evaluation benchmarks, and that observed performance gains stem from reasoning structure acquisition rather than knowledge leakage.

Method	Temp.	Prompt	MMLU-Pro Acc.	MMLU-Pro Tokens	GPQA Acc.	GPQA Null Rate	GPQA Null-Corr.	GPQA Tokens
Base	Locked	Base	55.66%	1,742	43.03%	30.91%	50.76%	5,875
Base	Locked	Custom	58.24%	1,595	45.05%	24.44%	51.16%	5,539
Base	Dynamic	Custom	57.64%	1,600	45.05%	24.44%	51.16%	5,539
D-CoT (LoRA)	Locked	Base	64.73%	1,496	52.82%	4.24%	53.88%	2,772
D-CoT (LoRA)	Locked	Custom	62.92%	1,199	51.41%	1.82%	53.27%	2,073
D-CoT (LoRA)	Dynamic	Custom	63.43%	1,202	52.93%	3.54%	54.47%	2,153

Table 2: Comparison of Accuracy, Null Rate, and Average Output Tokens on Qwen3-8B

Figure 3: D-CoT: Accuracy vs. Average Output Tokens (Qwen3-8B, 0-shot). (A) MMLU-Pro (12k questions). (B) GPQA-Diamond (5-seed average). D-CoT conditions (red) cluster in the upper-left (high accuracy, low tokens), demonstrating a Pareto improvement over the Baseline conditions (blue). The inter-group separation far exceeds variation from temperature or prompt settings.

Efficient Language Model Reasoning

Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.

See all 370 papers

Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs
Apr 27, 2026Sagnik Chatterjee, Atharva Patil, Sricharan RameshCoT Reasoning Small Language Models
SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning
May 29, 2026Jian Yao, Xiongcai Luo, Ran Cheng +1RL for Language Model Reasoning Efficient Language Model Inference
Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University · Bytedance · The Hong Kong Polytechnic University-Daya Bay Technology and Innovation Research Institute, Huizhou,Guangdong Province, China.
CIRF: Tokenizing Chain-of-Thoughts into Reusable Functional Units for Efficient Latent Reasoning in Large Language Models
May 27, 2026Yukyung Lee, Yumeng Shen, Jinhyeong Park +2CoT Reasoning Efficient Language Model Reasoning
Boston University · Hankuk University of Foreign Studies

Apr 27, 2026J/K move · Enter open · S save

Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs

Sagnik Chatterjee, Atharva Patil, Sricharan Ramesh

CoT Reasoning Small Language Models Efficient Language Model Reasoning Process Supervision Efficient Language Model Inference LLM Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) solve many reasoning tasks via chain-of-thought (CoT) prompting, but smaller models (about 7 to 8B parameters) still struggle with multi-step reasoning under tight compute and token budgets. Existing test time reasoning methods such as self consistency (sampling multiple rationales and voting), Tree-of-Thoughts (search over intermediate thoughts), and critique revise loops improve performance, but often at high token cost and without fine-grained step-level control. This project1 aims to address that gap: can Small Language Models (SLMs) reason reliably using the same or fewer tokens? This question is both scientific and practical. Scientifically, it probes whether process supervision and simple test-time controls (such as token budgets and rejection of redundant steps) can substitute for model scale or large sampling counts. Practically, many deployments (on-device, low-latency, or cost-constrained settings) cannot afford huge models or dozens of sampled rationales per query. A method that improves SLM reasoning at fixed cost would therefore be directly useful.

Abstract

Efficient Language Model Reasoning

Explore similar work

Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs

SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning

CIRF: Tokenizing Chain-of-Thoughts into Reusable Functional Units for Efficient Latent Reasoning in Large Language Models