Consistency Training Suppresses Cue Following Without Teaching Cue Selectivity

Prakhar Gupta; Terry Jingchen Zhang; Zhijing Jin

cs.CLOct 19, 2026

Consistency Training Suppresses Cue Following Without Teaching Cue Selectivity

Authors: Prakhar Gupta, Terry Jingchen Zhang, Zhijing Jin

Organizations: University of Michigan · Jinesis Lab, University of Toronto & Vector Institute · University of Oxford · EuroSafeAI ·

Abstract

Language models often change answers because of a cue, a hint such as "I believe the answer is B" that gives no valid evidence. Consistency training (CT) trains a model to answer the same with and without the cue. Four CT methods apply this idea to output tokens (BCT), residual activations (ACT), attention patterns (AttCT) and sampled-answer rates (RMCT). Prior work suggests some learn different solutions. We train all four on Gemma-2-2B-it and Llama-3.1-8B-Instruct with the same data. We test predictions written in advance and inspect the models with a known circuit that carries cues to the answer. All four make the trained cue less effective, and their fixes share three properties. First, correct cues lose about as much influence as wrong ones. Second, the fix is usually large for cues before the answer options, as in training, and much smaller after them. Llama RMCT depends least on position. Third, the circuit heads still carry most remaining cue-induced errors. Making BCT targets depend on the cue's correctness mostly weakens the fix. CT thus weakens the trained cue without teaching models when a cue is right. Evaluations with only wrong cues at the trained position miss the first two properties.

Figures & tables

Figure 1: Consistency training weakens right and wrong cues alike, mostly before the answer options, and leaves the cue circuit in use. (a) Cue following on the same questions, cue before the options. (b) Harmful-cue following removed at each cue position. (c) Share of cue errors corrected by patching, which resets the circuit heads to uncued values. If CT had removed the circuit, the after-CT bar would drop toward the black lines, which show the best random heads (highest over models and methods). All bars and the lines in (b) average the two models. Bars in (a) and the after-CT bar in (c) also average the four methods. Whiskers show ranges, and Llama uses one seed in (c).

Figure 2: Every method cuts the trained cue, but transfer to new cue types is weaker. Whiskers show 95% intervals and dotted lines the original model (Table 6 ).

Figure 3: A cue that names the correct answer loses its effect as much as a wrong one. Arrows run from the original model (circle) to the model after CT, on questions that both models answer wrongly without a cue (cue before the options). A selective fix would keep the green arrows short.

Figure 4: The fix fades for cues placed after the answer options. Whiskers show 95% intervals. Training puts every cue at the front (shaded). On Gemma, the values of RMCT after the options come mostly from its preference for answering D (§ 4.3 ).

Figure 5: The circuit still carries the leftover errors, and the fix sits mainly before it. (a) Share of the remaining cue-induced errors that patching the circuit heads corrects. (b, c) We copy one block of attention layers from the trained model into the original model and show the share of the method’s fixes that this mixed model reproduces (our planned readout for Llama). The swap cannot test RMCT. Llama uses one seed.

Figure 6: No change to the training data of BCT makes the fix selective. Cue battery, seed means. (a) shows each BCT variant and the self-training control on Gemma, and (b) compares BCT with position-diverse BCT, which trains with the cue at all four positions.

Figure 7: Benign fine-tuning raises cue following in every model, the original one included. Harmful-cue following on the trained cue, with bands for the SD over seeds (the original model is one run).

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

code	registered (EDT)	prediction, as written	outcome
P1	26 Sep, 23:47	efficacy: every arm cuts trained-cue (suggested_answer test) flip $\mid$ knew by $\geq$ 50% vs untrained.	fails
P2	26 Sep, 23:47	transfer: BCT has the lowest mean held-out caving (5 held-out cues) of the four arms.	fails
P3	26 Sep, 23:47	reading vs following (wave-3 W2 pattern): early-reader cue-span mass on the trained cue falls by $\geq$ 30% for ACT and AttCT in all seeds, while on every held-out cue each arm’s relative change stays within $\pm$ 25% (seed mean).	fails
P4	26 Sep, 23:47	where edits land: per-head OV depth centroid (log 82.7 definition) is lower for AttCT and ACT than for RMCT in all seeds.	holds
P5	26 Sep, 23:47	content-blindness: every arm lowers paired helpful-cue uptake (gen3, W4c paired definition) in all seeds.	holds
F1	27 Sep, 14:21	Bayes-BCT(0.8): helpful-uptake drop $\leq 0.5\times$ BCT’s AND harmful-caving drop $\geq 0.5\times$ BCT’s.	fails

Appendix

Table 1: Every scored prediction as registered (2026). P1–P5, F1–F3 and R1 concern Gemma, and B1–B3 concern response-only BCT on Gemma. L1–L5, LF1–LF3, LR1–LR2 and D1 concern Llama. S1–S2 outcomes are given as Gemma / Llama. The 30 Sep 12:07 erratum withdrew the composition numbers of the 30 Sep block, including the 25% in LF1, and the Llama Bayes-BCT data for LF1 were rebuilt on the corrected readout. The P1–P5 section header in the file reads 27 Sep, and an erratum appended one minute later corrects it to 26 Sep. Outcomes are on the corrected, unmerged results.

appended (EDT)	amendment, erratum or note
27 Sep, 00:27	LR rule, Amendment 1 (the header says 00:45, corrected by an erratum). The grid-end test uses the best F1, extensions are sequential and at most two, and an untied published LR yields to the best-F1 LR. RMCT execution v2 uses shared-prefix sampling and a one-epoch run in resumable segments (we verified that the LoRA weights and the Adam step are identical across boundaries).
27 Sep, 02:07	Exploratory RMCT+anchor variant declared (anchor weight 0.5, the trainer’s default), after the pre-registered RMCT runs had collapsed to a constant answer at every LR evaluated so far. It is not part of P1–P5.
27 Sep, 02:31	LR rule, Amendment 2. A checkpoint that gives one letter on at least 95% of validation prompts is degenerate and excluded. If every grid point is degenerate, the grid extends downward.
30 Sep, 09:01	Erratum to D1. RMCT cannot be tested by the layer swap (its adapter trains MLP and unembedding weights), so it is dropped from that analysis.
30 Sep, 11:04	Llama relapse runs use micro-batches of 2 because of memory. The loss and gradient are unchanged up to bf16 numerics.
30 Sep, 12:07–13:27	Llama readout correction, in four notes. The first-token readout must include the “(L” tokens, with a readout gate of at least .97 agreement with greedy generation (v6, 12:07, with gate coverage and handling at 12:24). Because v6 fails for Llama BCT, the readout becomes the maximum over letter forms (v7), and Llama LR selection, test rates and relapse move to generated answers (12:53). Exact bf16 ties are broken as greedy decoding breaks them (v7.1, 13:27).

Appendix

Table 2: Amendments, errata and correction notes after the first scored predictions, in the order appended (2026). Notes on one topic appended within hours of each other share a row. The full texts are in the supplementary pre-registration file.

verdict	deciding quantity	value	95% interval	status
P1 (BCT)	trained flip $\mid$ knew $-$ $0.5\times$ original ( $\leq$ 0 holds)	-0.129	[-0.196, -0.062]	resolved
P1 (ACT)	trained flip $\mid$ knew $-$ $0.5\times$ original ( $\leq$ 0 holds)	-0.004	[-0.077, 0.069]	unresolved
P1 (AttCT)	trained flip $\mid$ knew $-$ $0.5\times$ original ( $\leq$ 0 holds)	+0.012	[-0.058, 0.084]	unresolved
P1 (RMCT)	trained flip $\mid$ knew $-$ $0.5\times$ original ( $\leq$ 0 holds)	-0.153	[-0.211, -0.093]	resolved
P2	BCT held-out $-$ lowest other method ( $<$ 0 holds)	+0.139	[0.094, 0.185]	resolved
L1 (BCT, trained)	trained method $-$ original ( $<$ 0 holds)	-0.204	[-0.293, -0.115]	resolved

Appendix

Table 3: Pre-registered verdicts that turn on a single estimate, with the deciding quantity, threshold, value and 95% interval. The intervals come from a bootstrap over questions for test-set, cue-battery and layer-swap rates, a binomial bootstrap for relapse fractions (Table 20 gives the seed

t

interval) and a seed-level

t

interval for LR2 and S1–S2. A verdict is unresolved when the interval contains the threshold.

model	method	stored	re-selected	changed	validation F1 by LR (unmerged, generated)
Gemma	BCT	1e-4	1e-4	no	1e-6: 0.468 3.16e-6: 0.469 1e-5: 0.449 3.16e-5: 0.509 1e-4: 0.569 3.16e-4: 0.523
Gemma	ACT	3.16e-6	1e-5	yes	1e-6: 0.532 3.16e-6: 0.559 1e-5: 0.563 3.16e-5: 0.557 1e-4: 0.559
Gemma	AttCT	5e-6	1.58e-5	yes	5e-6: 0.548 1.58e-5: 0.553 5e-5: 0.539 1.58e-4: 0.537 5e-4: 0.534
Gemma	RMCT	3.16e-6	3.16e-6	no	3.16e-6: 0.574 1e-5: 0.524 3.16e-5: 0.353 d 1e-4: 0.339 d 2.86e-4: 0.339 d 5e-4: 0.339 d 1e-3: 0.000 d
Gemma	RMCT+anchor †	3.16e-5	3.16e-5	no	1e-5: 0.573 3.16e-5: 0.575 1e-4: 0.022 d 2.86e-4: 0.008 d
Gemma	response-only BCT	1e-3	1e-3	no	1e-5: 0.508 3.16e-5: 0.583 1e-4: 0.583 3.16e-4: 0.598 1e-3: 0.627

Appendix

Table 4: Stored and re-selected learning rates, with the validation F1 at every grid point (unmerged adapters, generated answers). d marks a degenerate point (excluded), and † an exploratory variant.

Gemma-2-2B-it
		merged $-$ unmerged, seed mean
method	agreement (range)	trained cue	held-out	accuracy	rule met
BCT	0.972–0.983	-0.010	-0.001	-0.003	0/3
ACT	0.975–0.988	-0.004	+0.003	-0.005	0/3
AttCT	0.987–0.991	+0.002	-0.002	-0.003	2/3
RMCT	0.924–0.943	+0.033	+0.059	+0.012	0/3

Appendix

Table 5: Merged (bf16 merge_and_unload ) and unmerged adapters, per method over its three seeds. The columns give the range of agreement of the generated test answers, the seed-mean difference (merged

-

unmerged) in trained-cue flip

\mid

knew , held-out flip

\mid

knew and uncued accuracy, and the number of seeds that meet the declared rule. The merge check came before the learning-rate re-selection, so the Gemma ACT and AttCT rows are the checkpoints at the superseded rates. Held-out rates use the 16-token answers before the few-shot correction. † marks an exploratory variant.

Gemma-2-2B-it
method	trained-cue flip $\mid$ knew	$\Delta$ [95% CI]	held-out (5 cues)	$\Delta$ [95% CI]	uncued accuracy ( $\Delta$ )
original	0.490 [0.39, 0.59]	–	0.546 [0.50, 0.59]	–	0.454 [0.39, 0.52]
BCT	0.116 $\pm$ 0.024	-0.37 [-0.48, -0.27]	0.421 $\pm$ 0.018	-0.12 [-0.17, -0.08]	0.406 $\pm$ 0.045 (-0.05)
ACT	0.241 $\pm$ 0.012	-0.25 [-0.34, -0.16]	0.479 $\pm$ 0.004	-0.07 [-0.10, -0.04]	0.468 $\pm$ 0.005 (+0.01)
AttCT	0.257 $\pm$ 0.010	-0.23 [-0.32, -0.15]	0.486 $\pm$ 0.003	-0.06 [-0.08, -0.03]	0.444 $\pm$ 0.005 (-0.01)
RMCT	0.091 $\pm$ 0.015	-0.40 [-0.50, -0.30]	0.282 $\pm$ 0.043	-0.26 [-0.31, -0.22]	0.458 $\pm$ 0.028 (+0.00)

Appendix

Table 6: All four methods reduce following of the trained cue, and transfer depends on the method and the model. Test set, generated answers. Harmful-cue following (flip

\mid

knew ) for the trained cue at its original positions and averaged over the five held-out cue types, and uncued accuracy. Seed mean

\pm

SD.

\Delta

is the method minus the original model, with its 95% question-cluster interval (no interval for accuracy). The cells of the original model show their own interval.

Gemma-2-2B-it
method	trained cue	post hoc	distractor fact	wrong few-shot	distractor arg.	squares
original	0.490	0.794	0.255	0.504	0.737	0.440
BCT	0.116 (-0.37)	0.391 (-0.40)	0.086 (-0.17)	0.513 (+0.01 ∘ )	0.823 (+0.09)	0.292 (-0.15)
ACT	0.241 (-0.25)	0.679 (-0.12)	0.066 (-0.19)	0.543 (+0.04 ∘ )	0.667 (-0.07)	0.438 (+0.00 ∘ )
AttCT	0.257 (-0.23)	0.773 (-0.02 ∘ )	0.056 (-0.20)	0.488 (-0.02 ∘ )	0.688 (-0.05)	0.427 (-0.01 ∘ )
RMCT	0.091 (-0.40)	0.369 (-0.43)	0.015 (-0.24)	0.326 (-0.18)	0.506 (-0.23)	0.196 (-0.24)

Appendix

Table 7: Per-cue flip

\mid

knew on the test set (generated answers, with few-shot cues scored at the test slot). The table gives the original value and, for each method, the seed mean with the change from the original model in parentheses. ∘ marks a change whose 95% question-cluster bootstrap interval includes zero, and † an exploratory variant.

Gemma-2-2B-it
method	trained: method / control	beyond prior	held-out: method / control	beyond prior	modal-letter share
BCT	0.115 / 0.439	0.324 $\pm$ 0.025	0.423 / 0.587	0.164 $\pm$ 0.023	0.31 (D/C/C)
ACT	0.238 / 0.471	0.233 $\pm$ 0.020	0.488 / 0.597	0.109 $\pm$ 0.003	0.27 (C/C/C)
AttCT	0.251 / 0.475	0.224 $\pm$ 0.012	0.516 / 0.603	0.086 $\pm$ 0.008	0.27 (C/C/C)
RMCT	0.098 / 0.336	0.237 $\pm$ 0.020	0.303 / 0.492	0.189 $\pm$ 0.012	0.54 (D/D/D)
RMCT+anchor †	0.142 / 0.462	0.320 $\pm$ 0.053	0.392 / 0.592	0.200 $\pm$ 0.050	0.30 (A/C/A)

Appendix

Table 8: Letter-prior control (exploratory, first-position readout, with the gate-failing Gemma checkpoints on the v7.1 readout). The table gives the flip

\mid

knew of the method and of the original model shifted to the uncued letter distribution of the method, on the trained cue and the held-out mean. It also gives the drop beyond the prior (control

-

method, seed mean

\pm

SD) and the share of uncued answers on the most frequent letter of the method (seed mean, with the letter per seed). † marks an exploratory variant.

Gemma-2-2B-it (original model’s harmful-cue following, front 0.493, pre 0.463, post 0.746, end 0.878)
method	helpful drop	harmful drop	ratio [95% CI]	rel. red. front	pre	post	end
BCT	0.384 $\pm$ 0.057	0.304 $\pm$ 0.077	1.26 [1.08, 1.49]	0.951	0.885	0.403	0.180
ACT	0.309 $\pm$ 0.004	0.252 $\pm$ 0.002	1.23 [1.10, 1.38]	0.920	0.791	0.127	0.093
AttCT	0.280 $\pm$ 0.006	0.212 $\pm$ 0.006	1.32 [1.16, 1.51]	0.912	0.784	0.064	0.025
RMCT	0.416 $\pm$ 0.048	0.377 $\pm$ 0.035	1.10 [0.96, 1.28]	0.924	0.918	0.542	0.382
Llama-3.1-8B-Instruct (original model’s harmful-cue following, front 0.381, pre 0.403, post 0.605, end 0.638)

Appendix

Table 9: Helpful cues drop together with harmful ones, and the fix fades when the cue moves after the options. Cue battery, pooled over six wordings and four positions. Paired drops in helpful- and harmful-cue following (seed mean

\pm

SD), their ratio with a 95% question-bootstrap interval (“–” where the harmful drop exceeds .02 in fewer than 97.5% of samples), and the relative reduction of harmful-cue following at each position (only front is trained).

Gemma-2-2B-it
method	absolute [95% CI]	proportional [95% CI]	log-odds [95% CI]
BCT	1.26 [1.08, 1.49]	0.98 [0.89, 1.09]	1.30 [1.10, 1.54]
ACT	1.23 [1.09, 1.37]	0.99 [0.93, 1.06]	1.35 [1.18, 1.56]
AttCT	1.32 [1.15, 1.51]	1.06 [0.97, 1.15]	1.49 [1.26, 1.75]
RMCT	1.10 [0.95, 1.28]	0.87 [0.78, 0.97]	1.10 [0.93, 1.29]
RMCT+anchor †	1.04 [0.94, 1.15]	0.83 [0.78, 0.88]	1.08 [0.96, 1.22]

Appendix

Table 10: Selectivity on three scales. Each cell is the seed-mean helpful drop divided by the seed-mean harmful drop on the paired item sets, over all positions, with a 95% question-bootstrap interval (B = 2000, and “–” where the harmful drop exceeds .02 in fewer than 97.5% of samples). Exploratory. † marks an exploratory variant.

Gemma-2-2B-it
	helpful / harmful drop: absolute / proportional / log-odds
method	before options	front	pre	post	end
BCT	1.50 / 0.99 / 1.08	1.49 / 0.98 / 0.95	1.51 / 1.00 / 1.15	0.88 / 0.73 / 1.09	0.86 / 0.76 / 1.95
ACT	1.37 / 0.98 / 1.08	1.41 / 0.99 / 1.07	1.33 / 0.96 / 1.06	0.74 / 0.63 / 1.13	0.29 / 0.26 / 0.98
AttCT	1.37 / 0.96 / 1.07	1.42 / 0.98 / 1.10	1.29 / 0.92 / 1.03	0.82 / 0.70 / 1.33	– / – / -0.17
RMCT	1.34 / 0.92 / 0.90	1.37 / 0.94 / 0.94	1.31 / 0.90 / 0.86	0.98 / 0.82 / 1.13	0.68 / 0.60 / 1.31

Appendix

Table 11: Helpful-to-harmful ratio before the options (front and pre pooled) and at each position, with each cell on three scales (“–” on a scale where the harmful drop is below .02). Seed means on the paired item sets. Exploratory. † marks an exploratory variant.

Gemma-2-2B-it
method	correct cue	wrong cue	drop difference [95%]	log-odds gap	change [95%]
BCT	0.62 $\to$ 0.06	0.57 $\to$ 0.05	+0.043 [-0.028, 0.118]	+0.22 $\to$ +0.16	[-0.66, 0.55]
ACT	0.66 $\to$ 0.09	0.58 $\to$ 0.04	+0.028 [-0.033, 0.091]	+0.33 $\to$ +0.82	[-0.02, 1.05]
AttCT	0.66 $\to$ 0.12	0.59 $\to$ 0.08	+0.035 [-0.025, 0.097]	+0.33 $\to$ +0.48	[-0.26, 0.56]
RMCT	0.61 $\to$ 0.11	0.56 $\to$ 0.04	-0.022 [-0.094, 0.054]	+0.21 $\to$ +1.15	[0.33, 1.72]
gold-gated BCT	0.61 $\to$ 0.21	0.58 $\to$ 0.11	-0.074 [-0.153, 0.013]	+0.13 $\to$ +0.85	[0.18, 1.29]

Appendix

Table 12: Same questions, cue before the options. The table gives the following of a cue that names the correct answer and of a cue that names a wrong option on the same questions, original

\to

trained (seed means). The questions are those that both the original and the trained model answer wrongly without a cue, and the option that the wrong cue names is neither model’s uncued answer. The last columns give the difference of the two drops and the change in their log-odds gap, with 95% question-bootstrap intervals. Exploratory.

Gemma-2-2B-it
	strength and letter checks			training-family / novel wordings
method	matched: before / after	after coef.	end, non-correct	front	pre	post	end
BCT	0.93 / 0.46	-0.57	0.13	0.95 / 0.95	0.90 / 0.87	0.40 / 0.41	0.19 / 0.17
ACT	0.86 / 0.25	-0.66	0.08	0.92 / 0.92	0.80 / 0.78	0.11 / 0.15	0.06 / 0.12
AttCT	0.83 / 0.18	-0.72	0.02	0.90 / 0.93	0.79 / 0.78	0.04 / 0.10	0.00 / 0.05
RMCT	0.91 / 0.65	-0.30	0.28	0.92 / 0.93	0.91 / 0.93	0.57 / 0.51	0.41 / 0.35

Appendix

Table 13: Position checks (seed means, exploratory). Matched is the relative reduction of harmful-cue following in wording

\times

position cells whose original harmful-cue following lies in the range shared by cells before and after the options. After coef. is the after-the-options coefficient of a regression of relative reduction on original harmful-cue following and position over the 24 cells. End, non-correct is the end-position relative reduction when every answer other than the correct one counts as following the cue. The right part gives the relative reduction by position for the three training-family wordings and the three novel wordings. † marks an exploratory variant.

Gemma-2-2B-it
	relative reduction of trained-cue flip $\mid$ knew
method	front (trained)	pre	post	front $-$ post [95%]
original flip $\mid$ knew ( $n$ )	0.303 (33)	0.467 (30)	0.686 (35)	–
BCT	0.82	0.98	0.60	+0.22 [-0.077, 0.475]
ACT	0.80	0.78	0.25	+0.55 [0.204, 0.843]
AttCT	0.83	0.82	0.14	+0.69 [0.405, 0.947]

Appendix

Table 14: The trained cue of the test set (Table 6 ) split by the cue position of each source row (post hoc, exploratory). The table gives the original flip

\mid

knew per position with the number of known items in parentheses, the relative reduction of flip

\mid

knew per position (seed means), and the front

-

post gap with a 95% question-bootstrap interval (B = 2000, with the same resampled questions for the original model and every seed). † marks an exploratory variant.

Gemma
checkpoint (seed 42)	harmful drop	helpful drop	ratio abs.	prop.	log-odds	rel. red. front	pre	post	end
BCT, selected LR	0.382	0.446	1.17	0.93	1.16	0.97	0.90	0.55	0.29
BCT, LR 3.16e-5	0.127	0.231	1.82	1.42	2.04	0.87	0.72	-0.10	-0.05
BCT, LR 3.16e-4	0.350	0.449	1.28	0.99	1.28	0.95	0.92	0.61	0.25
ACT, selected LR	0.249	0.304	1.22	0.98	1.35	0.92	0.78	0.13	0.09
ACT, LR 3.16e-6	0.210	0.244	1.16	0.93	1.32	0.79	0.69	0.10	0.08

Appendix

Table 15: Robustness of the pattern to the learning rate and the training data (exploratory, seed 42). The table gives the paired drops on the cue battery, the helpful-to-harmful ratio on three scales and the relative reduction of harmful-cue following by position. It covers the selected learning rate, its non-degenerate neighbors, and the Llama checkpoints trained with the authors’ own data and configuration.

Gemma-2-2B-it
method	circuit [Wilson 95%] (recovered / flips)	writers	readers	circuit, trained-cue flips ( $n$ )	best random set
original	0.818 [0.780, 0.850] (390/477)	0.71	0.26	0.82 (50)	0.07
BCT	0.843 [0.817, 0.866] (694/823)	0.78	0.26	0.97 (30)	0.06
ACT	0.844 [0.821, 0.864] (933/1106)	0.75	0.25	0.92 (72)	0.06
AttCT	0.861 [0.840, 0.880] (981/1139)	0.75	0.27	0.88 (73)	0.07
RMCT	0.910 [0.885, 0.931] (559/614)	0.87	0.43	0.90 (29)	0.12

Appendix

Table 16: Patch recovery. Each cell is the share of a method’s remaining cue-induced flips that return to the correct answer when the uncued answer-position activations of a head set are patched in, pooled over seeds (Gemma three seeds, Llama seed 42). The head sets are the circuit (with a Wilson 95% interval), its writer and reader subsets (the Llama writer subset is L17.H24 alone), the circuit on trained-cue flips only, and the best random head set (the maximum over three layer-matched sets and, for Gemma, over seeds). The Gemma circuit column and the subset and trained-cue columns are exploratory.

method	trained cue	held-out cues	fixed	still followed	fixed / gap	$n$ fixed / still	OV depth centroid
	rel. change, labeled items		drop on followed trained-cue items
BCT	-0.101 $\pm$ 0.153	-0.21 +0.37 -0.26 -0.06 -0.25	14%	13%	0.27	37 / 13	19.9 $\pm$ 0.4
ACT	-0.139 $\pm$ 0.016	-0.05 -0.05 -0.04 -0.12 -0.09	20%	2%	0.29	27 / 23	16.9 $\pm$ 0.7
AttCT	-0.109 $\pm$ 0.010	+0.00 -0.02 -0.01 -0.05 -0.01	20%	1%	0.25	24 / 26	4.9 $\pm$ 0.3
RMCT	-0.110 $\pm$ 0.046	-0.08 -0.05 -0.04 -0.07 -0.11	10%	5%	0.19	40 / 10	20.4 $\pm$ 0.1
RMCT+anchor †	-0.128 $\pm$ 0.051	+0.15 -0.07 +0.01 -0.02 -0.09	–	–	–	–	7.6 $\pm$ 0.5

Appendix

Table 17: Early reader heads of Gemma. The left part gives the relative change of their cue-span attention on the labeled items of the original model (seed mean

\pm

SD, with held-out cues in the order post hoc, distractor fact, wrong few-shot, distractor argument and squares). The middle part (exploratory) uses the trained-cue items that the original model follows. It gives the relative drop on the items the method fixes (its cued answer is no longer the cued letter) and on the items it still follows (seed means), the fixed-item drop as a share of the gap of the original model between items it follows and items it resists, and the item counts per seed. The right part gives the OV depth centroid of the LoRA edit over the attention projections (layer index out of 26). † marks an exploratory variant.

		mediated fraction: binary / own fixes / probability
method	fixed by trained model	EARLY	CARRIER	CIRCUIT	LATE	D1
Llama-3.1-8B-Instruct
BCT	72	0.71 / 0.51 / 0.38	0.72 / 0.53 / 0.73	0.28 / 0.21 / 0.17	0.25 / 0.19 / 0.16	fails (CARRIER)
ACT	64	0.73 / 0.69 / 0.59	0.70 / 0.66 / 0.58	0.11 / 0.09 / 0.06	0.08 / 0.08 / 0.01	holds (EARLY)
AttCT	57	0.72 / 0.65 / 0.61	0.70 / 0.61 / 0.59	0.14 / 0.11 / 0.06	0.07 / 0.05 / 0.01	holds (EARLY)
Gemma-2-2B-it

Appendix

Table 18: Layer swap. Each cell is the forward mediated fraction when one block of the trained model’s q, k, v and o projections is copied into the original model (Llama EARLY 0–8, CARRIER 9–14, CIRCUIT 15–18 and LATE 19–31, Gemma 0–6, 7–12, 13–18 and 19–25), on items whose cue the original model follows. The three readouts are the pre-registered binary readout, the share of the trained model’s own fixes and the share of the fall in the probability of the cued letter. For Llama (seed 42), the table also gives D1 (the largest block is EARLY or CARRIER, and LATE is below 0.15). For Gemma, it gives seed means over three seeds, the range over seeds of the number of fixed items, and the number of seeds that meet the same criterion (exploratory). RMCT cannot be tested this way.

Gemma-2-2B-it
method	trained-cue	held-out	accuracy	helpful drop	harmful drop	CAI	rel. red. front	end
BCT	0.116	0.421	0.406	0.384	0.304	1.26	0.951	0.180
response-only BCT	0.045	0.193	0.444	0.652	0.565	1.15	0.980	0.852
Bayes-BCT ( $r{=}.5$ )	0.161	0.460	0.412	0.308	0.234	1.32	0.935	0.111
Bayes-BCT ( $r{=}.8$ )	0.202	0.504	0.398	0.276	0.191	1.45	0.866	0.077
gold-gated BCT	0.159	0.452	0.438	0.243	0.229	1.06	0.842	0.037

Appendix

Table 19: Changing the training data of BCT. Seed means for the BCT variants of Section 4.5 (defined in Appendix B ). Response-only BCT is run on Gemma only. CAI (content-awareness index) is the helpful drop divided by the harmful drop ( ∗ marks a harmful drop below .05, which the registered criterion does not count as selective). Rel. red. is the relative reduction of harmful-cue following at the front and end positions.

Gemma-2-2B-it ; original model’s drift 0.490 $\to$ 0.589, log-odds +0.40 (held-out 0.596 $\to$ 0.668)
	trained cue					held-out cues
method	flip $\mid$ knew, step 0 $\to$ 125	relapse fraction	seed 95% CI	beyond drift	log-odds rise	fraction	beyond drift
BCT	0.116 $\to$ 0.185	+0.187 $\pm$ 0.083	[-0.020, 0.394]	-0.079 $\pm$ 0.075	+0.54 (1.4 $\times$ )	+0.140	-0.280 $\pm$ 0.103
ACT	0.241 $\to$ 0.330	+0.355 $\pm$ 0.042	[0.252, 0.458]	-0.044 $\pm$ 0.059	+0.44 (1.1 $\times$ )	+1.093	+0.421 $\pm$ 0.058
AttCT	0.257 $\to$ 0.363	+0.452 $\pm$ 0.070	[0.278, 0.626]	+0.026 $\pm$ 0.088	+0.50 (1.2 $\times$ )	+1.369	+0.503 $\pm$ 0.046
RMCT	0.091 $\to$ 0.199	+0.269 $\pm$ 0.109	[0.000, 0.539]	+0.020 $\pm$ 0.106	+0.89 (2.2 $\times$ )	+0.272	+0.028 $\pm$ 0.076

Appendix

Table 20: Relapse under 125 steps of benign LoRA fine-tuning (generated answers, three seeds per method,

\pm

is the seed SD). The relapse fraction is (method at step 125

-

method at step 0)/(original at step 0

-

method at step 0), and its seed 95% CI is a

t

interval over seeds. Beyond original drift subtracts the change of the original model under the same fine-tuning. Log-odds rise is the change in the logit of trained-cue flip

\mid

knew from step 0 to 125, with its ratio to that of the original model in parentheses. The drift-adjusted and log-odds columns are exploratory. Held-out fractions are given only where the held-out gap of the method is at least .05. The held-out columns use the 16-token answers without the few-shot correction (as does the original held-out rate in the header), and the held-out fraction beyond drift subtracts the held-out change of the original model.

checkpoint	MMLU BRR $\pm$ SE	held-out BRR $\pm$ SE	MMLU clean acc.	Anthropic rate
Gemma-2-2B-it original	0.540 $\pm$ 0.018	0.331 $\pm$ 0.019	0.596 [0.565, 0.626]	0.848
Gemma-2-2B-it BCT	0.145 $\pm$ 0.019	0.319 $\pm$ 0.019	0.512 [0.481, 0.543]	0.466
Gemma-2-2B-it ACT	0.081 $\pm$ 0.017	0.160 $\pm$ 0.017	0.602 [0.571, 0.632]	0.838
Gemma-2-2B-it AttCT	0.131 $\pm$ 0.018	0.176 $\pm$ 0.018	0.596 [0.565, 0.626]	0.849
Gemma-2-2B-it response-only BCT	0.011 $\pm$ 0.016	0.076 $\pm$ 0.018	0.536 [0.505, 0.567]	0.651
Gemma-2-2B-it BCT sweep, LR 1e-5	0.543 $\pm$ 0.018	0.421 $\pm$ 0.019	0.594 [0.563, 0.624]	0.656

Appendix

Table 21: The evaluator of the AttCT authors on seed-42 final checkpoints (adapter unmerged, their code). The columns give the MMLU bias rate (BRR = P(cued answer

\mid

cued)

-

P(cued answer

\mid

clean), n = 1000) with its unpaired SE, the held-out BRR (n = 951), clean MMLU accuracy (Wilson 95%) and the Anthropic model-written sycophancy rate (n = 999). Gemma ACT and AttCT use the re-selected learning rates (Appendix B.4 ), and the Gemma BCT sweep rows are three other learning rates of the BCT grid. § marks the authors’ own data and configuration run through their pipeline. Values in parentheses are those that the AttCT paper reports for its Llama checkpoints.

LR	evals	training-bias switch rate	held-out switch rate	authors’ logs (distractor argument)	authors’ logs (held-out)
base	2	0.650 [0.567, 0.724]	0.419 [0.383, 0.456]	0.650	–
1e-4	2	0.497 [0.417, 0.576]	0.414 [0.378, 0.450]	0.325/0.346	0.438/0.414
2.86e-4	2	0.362 [0.284, 0.449]	0.375 [0.337, 0.414]	0.141/0.132	0.320/0.322

Appendix

Table 22: RMCT in the authors’ own regime (their trainer and paper configuration, chain-of-thought answers, evaluated with their protocol on 100 questions per pass). The table gives the switch rate toward the bias on the training bias and pooled over five held-out biases, for final-step checkpoints pooled over two evaluation passes. Next to these are the values recomputed from the evaluation logs that the authors released (one training run, two passes). The authors’ base value comes from their 400-question logs, and their held-out value for the second pass pools four biases, while ours pools five. A second training run at 2.86e-4 (seed 43) gives 0.350 and 0.309 at its final step. We evaluate with the authors’ vLLM pipeline on bf16-merged weights, which weakened RMCT in our setting (Appendix D.3 ). The direction reproduces, but the size does not at 2.86e-4.

Reasoning Consistency in Language Models

Reasoning consistency in language models measures whether conclusions and reasoning remain reliable across equivalent prompts, samples, steps, and contexts.

See all 115 papers

Many Biases, One Circuit: Post-Training Tunes a Pretrained Cue-Routing Mechanism
Oct 16, 2026Prakhar Gupta, Terry Jingchen Zhang, Zhijing JinCognitive Bias in Language Models Circuits in Language Models
University of Michigan · Jinesis Lab, University of Toronto & Vector Institute · University of Oxford +2
Consistency Training while Mitigating Obfuscation via Rate Matching
Jun 1, 2026Sohaib Imran, Prakhar Gupta, Jannes Elstner +1LLM Sycophancy Language Model Robustness
Independent · University of Michigan · UK AI Security Institute
Base Models Can Reason By Taking a Cue From Training Data
Oct 5, 2026Sophie L. Wang, Amil Dravid, Rulin Shao +3LLM Prompting RL for Language Model Reasoning
MIT · UC Berkeley · University of Washington +1

Oct 16, 2026J/K move · Enter open · S save

Many Biases, One Circuit: Post-Training Tunes a Pretrained Cue-Routing Mechanism

Prakhar Gupta, Terry Jingchen Zhang, Zhijing Jin

University of Michigan · Jinesis Lab, University of Toronto & Vector Institute · University of Oxford+2

Cognitive Bias in Language Models Circuits in Language Models Language Model Post-Training Language Model Bias Evaluation LLM Sycophancy Causal Interventions in Language Models

Open paper AskPDF arXiv

Language models often abandon a correct answer when the prompt contains a cue, a hint toward a wrong option such as "I think the answer is B." Sycophancy and related biases show this failure. They are often studied separately and blamed on alignment. We test this view causally, tracing the attention heads behind these biases through the stages of two post-training pipelines. In multiple-choice tests on most models, four cue types rely largely on the same few heads, a shared circuit already present in pretrained models. Most base models follow cues in plain text. Fine-tuning a base model on about 200 generic assistant examples without cues activates these heads in chat format, so cue-following can be a side effect of assistant tuning. The final reinforcement learning (RL) stage strengthens the effect of cues in almost every comparison. In Tulu-3-8B and OLMo-2-7B, we swap the attention patterns of six middle layers between the RL checkpoint and its predecessor. For suggested-answer cues, this shifts each checkpoint's cue effect most of the way to the other's level. Post-training thus seems to change when and how strongly models use this circuit. Mitigation for these multiple-choice biases could target this circuit.

Abstract

Reasoning Consistency in Language Models

Explore similar work

Many Biases, One Circuit: Post-Training Tunes a Pretrained Cue-Routing Mechanism

Consistency Training while Mitigating Obfuscation via Rate Matching

Base Models Can Reason By Taking a Cue From Training Data