Bypass Observation: A Conceptual Design of a Non-Intrusive Layer-Wise Semantic Extraction Architecture

Haibin Tong; Jiang Yu

cs.AISep 12, 2026

Bypass Observation: A Conceptual Design of a Non-Intrusive Layer-Wise Semantic Extraction Architecture

Authors: Haibin Tong, Jiang Yu

Organizations: Henan University Kaifeng, China · National University of Defense Technology Changsha, China

Abstract

Large language models reason in high-dimensional hidden-state spaces, while users observe only final outputs. We introduce Bypass Observation, a non-intrusive layer-wise readout architecture that attaches read-only observation heads to selected Transformer layers without feeding their outputs back into the backbone. We consider three variants: a shared LM head across layers, layer-specific heads, and a layer- or step-adaptive head. For full-vocabulary readout, we derive a closed-form overhead approximation governed primarily by V/(12d), with representative estimates ranging from about 30% to 240%, and discuss cost reductions via sparse observation, low-rank factorization, reduced vocabularies, top-k readout, and selective positions. We argue that Bypass Observation can make model computation more observable while remaining only a partial, potentially misleading projection of hidden states. We further distinguish bypass chain-of-thought from conventional chain-of-thought: conventional reasoning tokens enter the autoregressive computation, whereas bypass readouts remain causally external at inference time, although they can still provide training signals in reinforcement learning. Finally, we discuss applications to looped and recurrent-depth Transformers, where iteration-wise readout may expose convergence, oscillation, and potential halting signals. The proposal is conceptual and analytical; systematic empirical validation remains future work.

Figures & tables

Figure 1: Three architectural variants of Bypass Observation. (a) A shared LM head is reused across multiple reasoning layers, providing a parameter-efficient common readout. (b) Each reasoning layer is equipped with its own LM head, allowing layer-specific calibration at increased parameter cost. (c) A

k

-adaptive LM head conditions its readout on the layer index or inference step, enabling one observer to adapt across depth or recurrent computation. Solid black arrows denote the main forward computation, while dashed blue arrows denote the read-only observation (bypass) pathway; the displayed “observed text” should be interpreted as a schematic readout that may correspond to top-

k

vocabulary predictions or, with an additional semantic translator, a textual summary. In all variants, observation outputs are not fed back into the backbone.

Model	$d$	$V$	$h$	$d_{\mathrm{head}}$	$h_{\mathrm{kv}}$	Attention	$d_{\mathrm{ff}}$	QKVO/ $d^{2}$	FFN/ $d^{2}$	Total/ $d^{2}$	$\rho$
GPT-3 175B	12288	50257	96	128	96	MHA	49152 (4.0 $d$ )	4.00	8.00	12.00	$\approx+34\%$
Llama-2 7B	4096	32000	32	128	32	MHA	11008 (2.69 $d$ )	4.00	8.06	12.06	$\approx+65\%$
Llama-2 70B	8192	32000	64	128	8	GQA	28672 (3.5 $d$ )	2.25	10.50	12.75	$\approx+31\%$
Llama-3 8B	4096	128256	32	128	8	GQA	14336 (3.5 $d$ )	2.50	10.50	13.00	$\approx+241\%$
Qwen2.5 7B	3584	152064	28	128	4	GQA	18944 (5.29 $d$ )	2.29	15.86	18.14	$\approx+234\%$

Table 1: Analytical estimates of the per-layer overhead of full-vocabulary bypass decoding. Values are computed from representative model configurations under the accounting assumptions described in Section 3.1.

Dimension	Conventional CoT	Bypass chain-of-thought
Enters the computation loop	Yes; generated text becomes part of subsequent context	No; read-only open-loop observation
Causal role of text	Part of the computation itself	Projection of hidden computation with no direct causal contribution
Generation mechanism	Autoregressive, serial, context-dependent	Layer-wise readout parallel to the main generation stream
Observation granularity	Typically one textual trajectory	Potentially token $\times$ layer observations
Text form	Free-form intermediate reasoning	Progressively changing predictions or semantic summaries
Can provide an RL training signal	Yes	Yes; even with a frozen observer, scalar rewards can update the backbone through policy gradients

Table 2: Causal differences between conventional chain-of-thought and bypass chain-of-thought.

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 781 papers

A transformer architecture alteration to incentivise externalised reasoning
Mar 22, 2026Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan +3Efficient Language Model Inference Large Reasoning Models
MARS · Geodesic Research, equal contribution · University of Amsterdam
ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention
Mar 23, 2026Xinyan Wang, Xiaogeng Liu, Ming Pei +1Early Stopping for Language Model Reasoning Overthinking in Language Models
University of Wisconsin–Madison · Johns Hopkins University
Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary
Jul 20, 2026Jan KirinLanguage Model Introspection Language Model Self-Assessment

Mar 22, 2026J/K move · Enter open · S save

A transformer architecture alteration to incentivise externalised reasoning

Elizabeth Pavlova, Mariia Koroliuk, Karthik Viswanathan, Cameron Tice+2

MARS · Geodesic Research, equal contribution · University of Amsterdam

Efficient Language Model Inference Large Reasoning Models Efficient Transformer Inference

Open paper AskPDF arXiv

We propose a new architectural change, and post-training pipeline, for making LLMs more verbose reasoners by teaching a model to truncate forward passes early. We augment an existing transformer architecture with an early-exit mechanism at intermediate layers and train the model to exit at shallower layers when the next token can be predicted without deep computation. After a calibration stage, we incentivise the model to exit as early as possible while maintaining task performance using reinforcement learning. We provide preliminary results to this effect for small reasoning models, showing that they learn to adaptively reduce computations across tokens. We predict that, applied at the right scale, our approach can minimise the amount of excess computation that reasoning models have at their disposal to perform non-myopic planning using their internal activations, reserving this only for difficult-to-predict tokens.

Abstract

LLM Interpretability

Explore similar work

A transformer architecture alteration to incentivise externalised reasoning

ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention

Operational Proto-Introspection in Looped Language Models: Process-Quality Taps, Executable Branching, and the Readout-Control Boundary