You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model

Ziyang Luo; Zhongyao Chu; Xinjie He; Youting Wang; Xukui Qin; Runxiong Wu; Yan-Syuan Chen

cs.CLAug 14, 2026

You Only Pass Once: Answering and Abstaining Together in a Single Forward Pass of a Frozen Language Model

Authors: Ziyang Luo, Zhongyao Chu, Xinjie He, Youting Wang, Xukui Qin, Runxiong Wu, Yan-Syuan Chen

Abstract

A frozen language model on reasoning tasks has two coupled weaknesses: it under-uses evidence its own residual stream already encodes, and it fails to detect when the input is insufficient to answer, so it confabulates. This paper consolidates two research lines that address these on the same residual stream: a conditional steering probe writes the stream at mid-stack layers and recovers reasoning accuracy from a frozen backbone, and a zero-shot sufficiency direction reads the stream and abstains when information is insufficient. Deployed in one forward pass they interfere: the steering write shifts the state the direction reads, costing up to 8 AUROC points of cross-domain transfer on small models; a separate clean pass doubles inference cost. We keep the direction fixed and train a small network to reconstruct the pre-steering residual from the steered one -- mean-squared error on (steered, clean) pairs, no sufficiency labels -- and read the direction on the reconstruction. The resulting system, YOPO (You Only Pass Once), answers, steers, and abstains in one forward pass of a frozen Qwen2.5 backbone (1.5B/3B/7B). End to end, three-way accuracy more than doubles the frozen baseline (0.375->0.798 on 1.5B alphaNLI) and one pass beats the two-pass reference at every scale (0.798/0.830/0.893 vs 0.753/0.790/0.863) and on ten backbones across six model families. We chart the capacity-transfer frontier quantifying the principle that abstention should not be trained in; a source-side audit catches our own alphaNLI construction leaking a surface artifact, so architectural claims are anchored on native-label replications (SQuAD2, RepLiQA, MuSiQue); and on the standard four-domain suite we contribute, to our knowledge, the first answer-or-abstain benchmark, where our gate tops every in-domain dataset and the label-free direction is the only gate family to survive domain transfer.

Figures & tables

Panel of Figure 1 — Figure 1: The conditional steering probe (the writer): at each injection layer a per-layer sub-probe reads the residual at spread-5 prompt positions, emits an input-conditioned unit direction (MLP) and a gated magnitude, and writes $\mathbf{r}\!+\!a\lVert\mathbf{r}\rVert\mathbf{u}(\mathbf{r})$ back through forward hooks. The backbone stays frozen; the edit is hot-swappable and fully reversible.

model	read	in-domain	transfer (HellaSwag)
1.5B	clean pass (2 passes)	$0.962$	$0.918$
1.5B	steered pass (1 pass)	$0.913$	$0.836$
3B	clean pass (2 passes)	$0.967$	$0.922$
3B	steered pass (1 pass)	$0.942$	$0.891$
7B	clean pass (2 passes)	$0.985$	$0.968$
7B	steered pass (1 pass)	$0.995$	$0.955$

Table 1: Contamination (AUROC, full vs. replace;

\alpha

NLI in-domain, HellaSwag zero-shot transfer): the fixed zero-shot direction

\mathbf{d}

read on the steered vs. clean pass. The steering write costs the gate

8.2

transfer points at 1.5B and

3.1

at 3B; at 7B the transfer cost is

1.3

points and the in-domain effect inverts (the steered residual is a better substrate for the read than the clean one). The conflict is concentrated at small scale.

Panel of Figure 2 — Figure 2: YOPO (You Only Pass Once): one-pass fusion with a label-free correction. The steering probe (blue, the writer) writes the residual stream at layers $\{12,16,20\}$ ; the same steered pass is read at a mid layer, passed through the reconstruction map $M$ (ours, orange—trained only to reconstruct the clean residual, never on sufficiency labels), and scored against the FIXED zero-shot direction $\mathbf{d}$ (the sufficiency gate, purple, the reader). Below threshold $\tau$ (a percentile cut) the system abstains; otherwise it emits the steering-amplified answer. One forward pass serves both the write and the read. Layer indices shown for Qwen2.5-1.5B/7B; 3B uses $\{16,20,24\}$ , read layer 24.

model	one-pass gate	in-domain	transfer (HellaSwag)
1.5B	naive $\mathbf{d}^{\top}h^{\mathrm{steer}}$ (floor)	$0.913$	$0.836$
1.5B	de-contam $\to\mathbf{d}$ [recon]	$0.944$	$\mathbf{0.888}$
1.5B	ML de-contam $\to\mathbf{d}$ [recon]	$0.959$	$0.869$
1.5B	ML de-contam $+$ BCE boost	$\mathbf{0.982}$	$0.859$
1.5B	ref: 2-pass clean read	0.962	0.918
3B	naive $\mathbf{d}^{\top}h^{\mathrm{steer}}$ (floor)	$0.942$	$0.891$

Table 2: The one-pass sufficiency gate across scale (AUROC full vs. replace;

\alpha

NLI in-domain, HellaSwag zero-shot transfer; single steered pass except the italic reference). De-contamination is label-free; the BCE boost is supervised. At 1.5B capacity trades transfer for in-domain along a clean frontier; at 3B the boosted map reaches in-domain above the 2-pass ceiling at a

0.006

transfer cost; at 7B the identity-anchored map learns

\approx

identity (floor

=

de-contam row) because there is no contamination left to undo. See Figure 3 .

Figure 3: The one-pass capacity–transfer frontier across scale. Each hue is a frozen Qwen2.5 backbone; points are one-pass gates in increasing capacity order (line); the haloed

\bigstar

is our proposed flagship (ML de-contamination

+

BCE boost) and

\times

is the two-pass clean-read reference (

2\times

cost). At 1.5B the frontier slopes down—capacity buys in-domain and costs transfer; at 3B it flattens; at 7B all one-pass gates cluster at the ceiling. The 7B multi-layer reconstruction point is omitted (convergence failure, § 11 ).

policy	1.5B		3B		7B
	$\alpha$ NLI	Hella	$\alpha$ NLI	Hella	$\alpha$ NLI	Hella
frozen baseline $\mathrm{B}_{0}$ (never abstains)	$0.375$	$0.500$	$0.563$	$0.400$	$0.628$	$0.604$
steering-only (never abstains)	$0.590$	$0.502$	$0.625$	$0.502$	$0.668$	$0.592$
gate-only (clean read, no steering)	$0.560$	$0.602$	$0.743$	$0.560$	$0.828$	$0.780$
steering $+$ clean gate (2-pass ref.)	$0.753$	$0.598$	$0.790$	$0.642$	$0.863$	$0.768$
steering $+$ naive steered gate (1 pass)	$0.698$	$0.580$	$0.760$	$0.616$	$0.880$	$0.754$

Table 3: Combined-system benchmark: end-to-end 3-way accuracy (answer correctly on answerable conditions / abstain on the no-information condition), 4-cond validation sets, abstain percentile

p{=}25

via quantile transfer. “Label-free” scopes to the read path : no abstention labels are optimized against (the steering probe in every row is answer-label trained; see § 5 ). The 7B ML de-contam cells inherit that variant’s convergence failure (§ 11 ) and are reported for completeness.

	Qwen2.5-1.5B (tested on)					Qwen2.5-7B (tested on)
trained on	$\alpha$ NLI	Hella	SQuAD2	RepLiQA	MuSiQue	$\alpha$ NLI	Hella	SQuAD2	RepLiQA	MuSiQue
$\alpha$ NLI	.986	.748	.648	.465	.677	.995	.928	.805	.795	.766
HellaSwag	.923	.905	.648	.480	.657	.967	.959	.930	.905	.808
SQuAD2	.766	.872	.863	.941	.817	.801	.828	.932	.961	.815
RepLiQA	.919	.599	.552	.999	.616	.867	.914	.913	.988	.830
MuSiQue	.799	.864	.865	.949	.832	.918	.935	.916	.969	.850

Table 4: Direction-transfer cross-matrix, computed entirely from our stored states. Each row: the sufficiency direction

d

is fit on that dataset’s train split (the sufficiency line’s v2 protocol: per-layer difference of means on an 80% subset,

L^{*}

by held-out AUROC, frozen); each column: ranking AUROC (

\uparrow

) of that frozen direction on the target dataset’s evaluation split, clean states, no recalibration. Diagonal (gray)

=

in-domain.

\alpha

NLI/HellaSwag are our constructed 4-condition sets (sufficient

=

non- replace ); SQuAD2/RepLiQA carry native labels; MuSiQue (multi-hop, supports-first windows, native answerability labels) is both the hardest target column and, at 7B, the strongest source row. Layer sweep restricted to the capture set shared by all five extractions.

Panel of Figure 4 — Figure 4: YOPO-2: one pass, write and read on separate channels. The answer-only steering write (orange) fires through a self-suppressing magnitude gate trained with unlabeled out-of-distribution negatives—active in-domain, near-zero under shift. Abstention never touches the write: a small read-head (blue) scores sufficiency from the layer-18 state of the same pass (stop-gradient), and a label-free quantile threshold turns the score into the abstain decision (green). In-domain this configuration beats the two-pass reference on every seed ( $0.877$ – $0.883$ ); under shift it holds parity with the best alternative ( $0.781\pm 0.012$ vs. $0.784$ ) while the read-head’s score is the best transfer gate we measure ( $0.955$ – $0.964$ AUROC).

backbone	size	baseline	steer only	flagship (1 pass)	2-pass ref.	gate naive $\to$ flagship
Qwen2.5	0.5B	0.318	0.420	0.620	0.545	0.720 $\to$ 0.981
OLMo-2	1B	0.485	0.535	0.743	0.662	0.920 $\to$ 0.987
TinyLlama (Llama arch.)	1.1B	0.375	0.375	0.565	0.500	0.830 $\to$ 0.963
Qwen2.5	1.5B	0.375	0.590	0.797	0.752	0.903 $\to$ 0.989
StableLM-2	1.6B	0.375	0.480	0.695	0.625	0.823 $\to$ 0.987
SmolLM2	1.7B	0.375	0.580	0.787	0.777	0.754 $\to$ 0.982

Table 5: Cross-family end-to-end validation: ten backbones, six families, 0.5B–7B. Same pipeline as Table 3 on the

\alpha

NLI 4-condition policy benchmark (answer/abstain accuracy,

\uparrow

, abstention budget

p{=}25\%

): zero-shot baseline; steering only; the proposed one-pass flagship (steer

+

multi-layer map

+

BCE boost); the two-pass reference (clean re-read,

2\times

cost). Last column: gate ranking AUROC read from the steered pass, naive

\to

flagship (

=

echo size and its repair). Steering probes for non-Qwen backbones come from our earlier backbone studies or are retrained by the same recipe (layers mapped by proportional depth); single seed per cell. Bold

=

one-pass flagship

\geq

two-pass reference.

system	params	SQuAD2	RepLiQA	MQ-C	MQ-F	mean	vs. self	vs. target zs
		in-domain decision acc.				cross-domain (12 arenas)
zero-shot spoken judgment	0	0.854	0.873	0.588	0.535	—	—	—
$+$ forced chain-of-thought	0	0.817	0.886	0.555	0.516	—	—	—
read-only direction	0	0.860	0.953	0.785	0.705	0.799	$-2.7$	$\mathbf{+8.6}$ (9/12)
constrained write (steering)	8.3M	0.853	0.976	0.849	0.695	0.681	$-16.3$	$-3.2$ (6/12)
free write (LoRA)	7.5M	0.888	0.994	0.949	0.803	0.678	$-23.1$	$-3.5$ (5/12)

Table 6: The three-family comparison on the standard four-domain suite (Qwen2.5-7B, judgment decision accuracy; the sufficiency line’s measurement). In-domain, capacity ranks the families (LoRA sweeps all four); across the 12 cross-domain arenas the ranking inverts: only the read-only direction exports a gain over the target’s own zero-shot spoken judgment (

+8.6\,\textrm{pp}

mean, 9 of 12 arenas), both write families landing below it. Forced chain-of-thought lowers the spoken judgment on three of four domains.

domain	(answerable)	zero-shot	$+$ calib. zs	$+$ sufficiency gate	oracle	zs $\to$ $+$ gate
	raw QA	hybrid accuracy				answered-precision
SQuAD2	0.890	0.826	0.830	0.834	0.945	$0.749\to 0.786$
RepLiQA	0.632	0.622	0.748	0.805	0.816	$0.459\to\textbf{0.636}$
MuSiQue-C	0.319	0.625	0.622	0.619	0.660	$0.637\to 0.645$
MuSiQue-F	0.062	0.505	0.503	0.502	0.531	$0.275\to 0.275$

Table 7: Task C (hybrid answer-or-abstain) on the standard four-domain suite with the zero-shot sufficiency gate, Qwen2.5-7B. Columns: raw answerable-QA accuracy; hybrid accuracy under zero-shot, calibrated zero-shot, and OR-composition with the sufficiency gate; the oracle upper bound; and the precision of answers actually given, zero-shot

\to

+

gate. OR-composition on RepLiQA gains

+18.3\,\textrm{pp}

(

98.6\%

of the oracle gap) and lifts answered-precision

0.459\!\to\!0.636

; the multi-hop domains are answering-bound.

target	gen-only	zero-shot $\mathbf{d}$	read-head (flagship)	hardest-source $\mathbf{d}$	oracle
		in-domain		cross-domain
SQuAD2	$0.826$	$0.834$	$\mathbf{0.845}$	$0.828$	$0.945$
RepLiQA	$0.622$	$0.805$	$\mathbf{0.805}$	$0.745$	$0.816$
MuSiQue-C	$0.625$	$0.619$	$\mathbf{0.642}$	$0.625$	$0.660$
MuSiQue-F	$0.505$	$0.502$	$\mathbf{0.505}$	–	$0.531$

Table 8: Task C (hybrid answer-or-abstain) on the standard four-domain suite, Qwen2.5-7B. Hybrid accuracy: answerable rows scored on the answer (normalized containment or token-F1

\geq 0.5

), unanswerable rows on abstention; OR-composition with the generation’s own abstention. In-domain gates are fit on the target’s own split (read-head cells are three-seed means,

\pm\leq 0.002

); the cross-domain column applies the direction fit on the hardest source (MuSiQue-Full, label-free threshold) to each target (its own row is in-domain, marked “–”). The supervised read-head leads every in-domain cell; cross-domain, the label-free direction from the hardest source matches in-domain on two of three transferable targets while the supervised gate collapses off-diagonal. Multi-hop targets are answering-bound (oracle within

3.5/2.7\,\textrm{pp}

of gen-only).

Figure 5: Task C (hybrid answer-or-abstain) on the standard four-domain suite, Qwen2.5-7B (Table 8 ). Blue: the in-domain flagship read-head, top on all four datasets. Orange: the label-free direction fit on the hardest source (MuSiQue-Full), deployed cross-domain with no target labels—it reaches the in-domain bar on SQuAD2 and MuSiQue-C and trails only on RepLiQA (MuSiQue-F is the source, so it has no cross bar). The oracle cap (dashed) and zero-shot floor (solid gray) bound each domain: the wide oracle gap on SQuAD2 is answering headroom, while on the multi-hop domains the oracle nearly meets the floor—these are answering-bound, and no gate has room there.

configuration (shared injection module)	RepLiQA	MuSiQue-C
zero-shot baseline	0.636	0.626
(B) injection off (read-only ablation)	0.822	0.628
(C) layer ordering (sufficiency line, replicated)	0.852	0.717
(B) read-only, zero-training direction	0.874	0.734
(B) read-only, BCE read-head	0.887	0.800

Table 9: Three one-pass fusion architectures on one shared injection module (answer-trained probe, layers

\{16,20,24\}

\alpha{=}0.3

), three-way hybrid accuracy, in-domain. (A) reconstruction is the discriminative-set optimum; under this generative protocol the simplest read-only design (B, BCE read-head) leads both cells, and (C) layer-ordering reproduces the sufficiency line’s reported gains. The write–read interference is protocol-dependent: severe on the controlled set, mild here. Row (B) numbers are seed-checked on both trained components (footnote in text); (C) is single-seed.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

config	$\alpha_{\max}$	$\alpha$ NLI	HellaSwag	PIQA
Qwen2.5-1.5B frozen $\mathrm{B}_{0}$	–	$50.8$	$74.1$	$68.6$
1 layer, last	$0.3$	$65.6$ ( $+14.8$ )	$74.9$ ( $+0.8$ )	$70.6$ ( $+2.0$ )
1 layer, last	$0.5$	$65.5$ ( $+14.6$ )	$74.5$ ( $+0.4$ )	$72.6$ ( $+4.0$ )
1 layer, spread-5	$0.3$	$69.1$ ( $+18.3$ )	$75.8$ ( $+1.7$ )	$73.4$ ( $+4.8$ )
1 layer, spread-5	$0.5$	$70.4$ ( $+19.5$ )	$76.2$ ( $+2.1$ )	$75.8$ ( $+7.2$ )
3 layers, last	$0.3$	$65.5$ ( $+14.7$ )	$74.7$ ( $+0.6$ )	$72.7$ ( $+4.1$ )

Appendix

Table 10: Steering benchmark grid: accuracy (%

\Delta

vs. the frozen baseline

\mathrm{B}_{0}

) across injection layer count, position scheme, and

\alpha_{\max}

, per dataset and backbone.

LLM Abstention

Large language model abstention studies when AI systems should answer, ask for more information, defer, or withhold unsupported claims.

See all 99 papers

INTACT: Inducing Latent Reasoning on a Frozen Backbone
Oct 14, 2026Kutay Tire, Atula Tejaswi, Haris Vikalo +1Efficient Language Model Reasoning Implicit Reasoning in Language Models
The University of Texas at Austin
One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models
Oct 24, 2026Jiarun Shen, Lei Zhao, Xiao LiLLM Inference Fine-Tuning Language Models
The Chinese University of Hong Kong, Shenzhen · Shanghai Jiao Tong University
Locating Answer-Correctness Signals in Frozen Large Language Models
Sep 29, 2026Yuansen Liu, Yixuan Tang, Anthony Kum Hoe TungLLM Answer Verification Language Model Error Detection
School of Computing, National University of Singapore

Oct 14, 2026J/K move · Enter open · S save

INTACT: Inducing Latent Reasoning on a Frozen Backbone

Kutay Tire, Atula Tejaswi, Haris Vikalo, Sujay Sanghavi

The University of Texas at Austin

Efficient Language Model Reasoning Implicit Reasoning in Language Models Continuous Latent Reasoning Numerical Reasoning in Language Models

Open paper AskPDF arXiv

Latent reasoning replaces natural-language chains of thought with compact continuous "soft-token" representations, reducing the number of intermediate tokens that must be decoded autoregressively. We use a small external encoder to induce latent reasoning in a frozen backbone model. The encoder iteratively maps the backbone's hidden states to new latent blocks, constructing a latent reasoning prefix, after which the frozen model writes its remaining reasoning and its answer as text. Our method, INTACT, trains this encoder through distillation to reproduce the effect of written reasoning steps while keeping the model's behavior close to that of the frozen backbone, and then reinforces it on answer correctness. Across four frozen backbones from 1B to 8B, INTACT improves both GSM8K and average out-of-domain accuracy, including GSM8K gains from 75.8% to 79.3% on Llama-3.2-3B and from 77.6% to 79.7% on Qwen2.5-3B. It also reduces end-to-end latency per problem by 19% on Llama-3.2-3B and outperforms the evaluated latent-reasoning baselines at the 3B and 8B scales. Beyond the mathematics setting, the same recipe trained on a multi-domain corpus improves the frozen backbone on twelve evaluation benchmarks spanning mathematics, science, and commonsense, without task-specific adaptation.

Abstract

LLM Abstention

Explore similar work

INTACT: Inducing Latent Reasoning on a Frozen Backbone

One Model, Two Paths: Balancing Single-Pass and Looped Performance in Language Models

Locating Answer-Correctness Signals in Frozen Large Language Models