ATLAS-RTC: Closing the Loop on LLM Agent Output with Token-Level Runtime Control

Christopher Cruz

cs.LGMar 29, 2026

ATLAS-RTC: Closing the Loop on LLM Agent Output with Token-Level Runtime Control

Authors: Christopher Cruz

Organizations: Purdue University

Abstract

We present ATLAS-RTC, a runtime control system for autoregressive language models that enforces structured output during decoding. ATLAS-RTC monitors generation at each step, detects drift from output contracts using lightweight signals, and applies targeted interventions such as biasing, masking, and rollback. Unlike post-hoc validation or static constrained decoding, it operates in a closed loop, enabling correction before errors materialize. Across structured generation and tool-calling tasks, ATLAS-RTC improves first-attempt success rates by 20 to 37.8 percentage points, with up to 88% latency reduction in failure-dominated settings. Results show that many failures arise from decoding artifacts rather than task misunderstanding, motivating runtime control as a distinct layer in LLM systems.

Figures & tables

Task	Baseline (ms)	Controlled (ms)	$\Delta$
Structured (hard)	5498	663	-87.9%
Database Query	2749	1636	-40.5%
Send Email	1495	3430	+129.4%

Table 1: Latency comparison across tasks.

Language Model Decoding

Language model decoding studies how models choose and generate outputs, balancing quality, reliability, diversity, and inference cost.

See all 168 papers

AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning
Aug 30, 2026Hanjun Luo, Qiushi Liu, Jingya Zhang +8Adaptive Inference LLM Inference Acceleration
New York University · New York University Abu Dhabi · University of Washington Seattle +4
CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions
Apr 21, 2026Vanessa Figueiredo, Wilter FranceschiAI Agent Reliability Language Model-Based Control
Orbital Sea Regina, SK, Canada
Don't Overthink It: Inter-Rollout Action Agreement as a Free Adaptive-Compute Signal for LLM Agents
Apr 9, 2026Khushal SethiInference-Time Compute Allocation LLM Inference Efficiency
Stanford University

Aug 30, 2026J/K move · Enter open · S save

AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning

Hanjun Luo, Qiushi Liu, Jingya Zhang, Haihong Pang+7

New York University · New York University Abu Dhabi · University of Washington Seattle+4

Adaptive Inference LLM Inference Acceleration LLM Reasoning

Open paper AskPDF arXiv

Large language models (LLMs) achieve strong reasoning performance, which depends critically on inference-time decisions. Yet these decisions are commonly handled by static, one-size-fits-all policies, limiting adaptation to diverse tasks and reasoning stages. Recent adaptive methods partially address this limitation, but they primarily adapt either decoding stochasticity (how the model explores) or reasoning compute (how long the model reasons) in isolation, leaving their interaction within a single reasoning trajectory unmodeled. To address this challenge, we shift toward a within-trajectory joint control view, and instantiate it in AutoCRAT, a decoder-side controller for frozen backbones. Using only signals available during decoding, AutoCRAT jointly adjusts sampling stochasticity and reasoning budget during generation. AutoCRAT operates over a discrete action space and updates control decisions only at semantic boundaries, improving stability while remaining responsive to the evolving reasoning process. Comprehensive evaluation across 6 benchmarks demonstrates that AutoCRAT (I) uses 13.8-52.7% fewer inference tokens on average than recommended static configurations, (II) surpasses recommended static and adaptive baselines by 1.5-4.5% in relative accuracy, and (III) enjoys strong cross-backbone transferability.

Abstract

Language Model Decoding

Explore similar work

AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning

CogniConsole: Externalizing Inference-Time Control as a Formal Abstraction for Reliable LLM Interactions

Don't Overthink It: Inter-Rollout Action Agreement as a Free Adaptive-Compute Signal for LLM Agents