cs.CLNov 12, 2026

From Large Language Models to Efficient System One Models via Adaptive Computation

Authors: Minh-Anh Nguyen

Organizations: Nanyang Technological University, Singapore

Abstract

Transferring LLMs into System One decision models typically replaces the language modeling head with a decision head while retaining the full backbone, making every decision pay for all layers. Yet it remains unclear how much depth such decisions require. We find that decision information is concentrated in intermediate layers, with early layers building essential features and the final six layers removable without loss, while independently trained smaller models largely mimic the full model rather than developing specialized decision capabilities. These findings suggest that efficiency requires not simply smaller backbones, but shared computation across different depths. Building on this, we propose \method{}, which jointly trains a full model with prefix sub-models that share its final layers and decision head through self-distillation. A lightweight gate reads the hidden state at each exit to determine whether additional computation is needed, allowing one model to operate at multiple layers. Across seven decision benchmarks, our approach matches or outperforms independently trained models at every depth, while achieving better accuracy-efficiency trade-offs through adaptive computation.

Figures & tables

Explore similar work

CardsList
  1. Releasing the Inner Jev: A Simple Recipe for Open Decision Models

    Oct 12, 2026Jonathan Geuter, Andrew Cheng, David Alvarez-MelisSystem One Decision ModelsLLM Decision-Making

  2. LEV: Thinking Fast and Deep with Looped Decision Models

    Oct 14, 2026Dingjie Song, Zhiling Yan, Yuxuan Zhang +1Efficient Language Model InferenceAdaptive Computation

  3. System1-Arena: Which Decisions Can Be Left to System 1?

    Oct 15, 2026Tianyu Chen, Xinyang Liu, Ruyi An +2LLM Decision-MakingSystem One Decision Models