cs.AINov 13, 2026

ARCHER: From Rationale Imitation to Executable Reasoning in Small Language Models

Authors: Dhruv Dixit, Khushi Patel, Congcong Xu, Min Song

Organizations: Department of Electrical and Computer Engineering, Stevens Institute of Technology, Hoboken, NJ - 07030, United States · Department of Computer Science, Stevens Institute of Technology, Hoboken, NJ - 07030, United States

Abstract

Small language models are attractive for local, low-cost, and privacy-sensitive deployment, but they remain unreliable on mathematical reasoning tasks that require decomposition, exact computation, tool use, and verification. We introduce ARCHER: Action-trace Reinforcement for Calculator-grounded Hierarchical Executable Reasoning, a two-stage framework for training small language models (SLMs) to produce executable and auditable reasoning trajectories. In the first stage, ARCHER compiles free-form chain-of-thought rationales into typed action traces containing reasoning steps, calculator calls, deterministic runtime observations, verification actions, and final answers. This shifts the supervision target from imitating explanatory text to learning a tool-grounded reasoning policy. In the second stage, ARCHER refines this policy through asymmetric hierarchical reinforcement learning, using grouped and conditionally gated trajectory rewards for answer correctness, tool grounding, process discipline, and trace alignment. Across five standard sub-10B backbones, ARCHER improves over the strongest tool-augmented baseline by 5.51 percentage points on macro-average across benchmark accuracies, including gains of 8.95 points on MATH-500 and 5.33 and 5.34 points on AIME-2024 and AIME-2025. Evaluations on two additional reasoning-specialized 7B backbones further show that executable grounding remains beneficial after reasoning-focused reinforcement learning or distillation. These results demonstrate that executable supervision and reward-guided behavioral refinement provide a general and effective training paradigm for compact mathematical reasoners.

Figures & tables

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Teaching Language Models to Think in Code

    May 8, 2026Hyeon Hwang, Jiwoo Lee, Jaewoo KangNumerical Reasoning in Language ModelsLLM Mathematical Reasoning

  2. DenseSteer: Steering Small Language Models towards Dense Math Reasoning

    May 28, 2026Yang Ouyang, Shuhang Lin, Jung-Eun KimSmall Language ModelsActivation Steering

  3. The Quality-Utility Paradox: Why High-Reward Data Impairs Small Model Mathematical Reasoning

    Jun 15, 2026Haolong Qian, Xianliang Yang, Yinuo ma +6CoT DistillationLanguage Model Distillation