Mechanistic Tomography: Designed Measurement for Control-Oriented Interpretability

Vijay Erramilli

cs.LGAug 19, 2026

Mechanistic Tomography: Designed Measurement for Control-Oriented Interpretability

Authors: Vijay Erramilli

Abstract

Mechanistic interpretability seeks quantities that models do not expose directly: represented states, component effects, interactions, and responses to interventions. Patching, gradients, Hessian-vector products, and subset interventions provide different measurements under different access assumptions and may target different quantities. We formulate their shared measurement structure as mechanistic tomography: designed measurement for recovering internal mechanisms and intervention effects. For a chosen basis and intervention family, measurements take the form y = Ax + w, where A describes the interventions, x is the target map, and w contains nonlinear response, sampling error, and basis misspecification. This language gives a practical procedure: start with the least costly measurements, test on held-out interventions at the intended scale, calibrate simple mismatch, and expand the measurement family when structured residuals remain. Control provides a demanding validation setting because an estimate that guides an intervention acts as an observer. In a two-HMM model, control error rises with observer error, while target improvement can hide nuisance-state movement. Under forward-only access, sparse aggregate measurements recover a finite-effect map with fewer interventions than coordinate patching. With gradient access, finite probes improve a local attribution map. Lifted measurements and Hessian-vector products recover interactions missed by first-order maps, while Tracr shows that the required family depends on the basis. On GPT-2-small IOI, the Name Mover-Negative Name Mover interaction is the largest held-out predictive term among three tested cross-group pairs. On Qwen-2.5-7B, finite calibration makes an additive refusal-response map adequate, so held-out error does not support pairwise lifting.

Figures & tables

Method and access	Declared target	Induced measurement
Coordinate patching; forward	finite singleton effects	one-coordinate rows
Subset recovery; forward	additive finite map	multi-coordinate rows; joint fit
AtP; backward	local first-order map $x$	gradient returns coordinates
Lifted recovery; forward	finite main and pair effects	columns include $a_{i}$ and $a_{i}a_{j}$
HVP; white-box second order	local curvature $H_{0}$	directions give curvature projections

Table 1. Mechanistic tomography maps each method to a target, access path, and measurement design.

Figure 1. With the controller and actuator fixed, closed-loop target error rises with observer error across ten observer variants. Oracle and probe lie low-left; coarse, noisy, proxy, and entangled estimates move up and right.

Figure 2. Aggregate recovery of the 32-coordinate finite-effect map. OMP scores Pearson $r=0.989$ and held-out $R^{2}=0.935$ with 12 aggregate measurements. Ridge reaches comparable held-out prediction with 32 measurements, the exhaustive coordinate count.

Scale	raw AtP $R^{2}$	calibrated AtP $R^{2}$	OMP $R^{2}$	fitted gain $\widehat{g}$
$\varepsilon=5$	0.943 [0.920, 0.967]	0.963 [0.933, 0.983]	0.965 [0.952, 0.976]	0.874 [0.799, 0.915]
$\varepsilon=8$	0.818 [0.751, 0.903]	0.960 [0.925, 0.978]	0.969 [0.960, 0.977]	0.742 [0.677, 0.780]

Table 2. Finite-scale calibration on one trained model whose intervention directions are estimated once. We cross three independently generated 256-sequence evaluation batches with three mask-and-split designs. Entries are means [min, max] across the nine cells.

Panel of Figure 3 — Figure 3. Finite-scale calibration across the fixed-direction $3\times 3$ design. Left: fitted gain at each scale for the nine evaluation-batch/design cells. Right: the median held-out $R^{2}$ gap between OMP and calibrated AtP as the calibration budget grows; bands show the 10th–90th percentiles over cells and subset draws.

Figure 4. Interaction-aware recovery at $\varepsilon=8$ with a generous fixed budget. Lifted OMP recovers the planted pair-bearing response, while methods without pair features remain below $0.5$ held-out $R^{2}$ .

Figure 5. Measurement budget for lifted recovery. With four planted nonzero terms among 2,080 main-and-pair features, lifted OMP reaches the prediction and pair-recall thresholds with 64–72 aggregate measurements, compared with 2,016 exhaustive pair probes. First-order OMP has no pair features.

Panel of Figure 6 — Figure 6. White-box interaction recovery. Designed HVPs recover the local pair map, and the quadratic harness supplies the known conversion to finite-scale pair coefficients. Accurate prediction also requires the AtP main-effect map.

Figure 7. Noise robustness with 96 aggregate measurements. Pair support can remain recoverable after held-out finite-response prediction degrades, separating support identification from coefficient accuracy.

Figure 8. Basis comparison in Tracr circuits. The current-token control is additive in both bases. The AB detectors are additive in the native compiler basis because it contains an explicit detector coordinate. In the restricted pre-conjunction basis, additive features explain only about half the variance, while lifted pair features recover the intended conjunction and restore held-out $R^{2}=1.0$ .

Figure 9. Detector-group writeback in Tracr circuits. Left: ablating the final-residual detector group destroys output prediction for the AB programs, while writing back the lifted reconstruction restores the original held-out $R^{2}$ . Right: additive features only partially reconstruct the detector groups, whereas lifted features reconstruct them completely.

Pair (heads)	mean ablation $I$ [95%]	zero ablation $I$ [95%]	$I$ /pair (mean/zero)
$P\times B$ (24)	1.080 [.912, 1.262]	1.268 [1.013, 1.558]	0.045/0.053
$P\times E$ (6)	1.830 [1.569, 2.076]	1.681 [1.474, 1.889]	0.305/0.28
$B\times E$ (16)	.502 [0.429, 0.576]	0.518 [0.416, 0.618]	0.031/0.032

Table 3. Direct IOI group interactions across four lexical frames in ABBA and BABA order, with 128 shared matched name pairs. Intervals use a crossed bootstrap that resamples lexical frames and one shared name-pair vector while retaining both orders. The last column divides by the number of possible cross-group head pairs; it does not assign the same effect to every individual pair.

Panel of Figure 10 — Figure 10. IOI interaction measurements. Left: direct group interactions on the original-template mean-ablation surface. Right: held-out gains from adding one count-product term to the count-additive model. $P\times E$ gives the largest of the three tested gains; the interval for $B\times E$ crosses zero.

Response map	parameters	MAE	RMSE	$R^{2}$
Calibrated additive	20	.003790	.004699	.9829
Lifted pairwise	48	.003801	.004613	.9835

Table 4. Held-out prediction on the Qwen-2.5-7B refusal-response surface. The test contains 128 actions at the unseen scale

0.75

and 224 prompts.

Regime	Available access	Start with	Escalate when
Additive, white-box	gradients and finite evaluations	AtP, then a few finite probes	held-out error does not reduce to a small gain correction
Additive, forward-only	scalar intervention responses	sparse subset measurements	the map is dense or held-out prediction fails
Interactional, white-box	gradients, HVP queries, and finite evaluations	AtP plus HVPs for local curvature	local support fails, or finite-scale probes show that calibration is needed
Interactional, forward-only	scalar intervention responses	lifted subset measurements	lifted coverage or conditioning fails
Uncertain basis	candidate coordinates and interventions	simplest declared basis	residuals persist or writeback fails
Downstream use	fixed controller and actuator	lowest-dimensional validated observer	target, collateral, or transfer tests fail

Table 5. Observer selection as a measurement decision.

Mechanistic Interpretability

Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.

See all 391 papers

ObserverBench: Testing Mechanistic Estimates for Intervention and Control
Sep 2, 2026Vijay ErramilliMechanistic Interpretability AI Safety Evaluation
Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability
Jul 9, 2026Amir AsiaeeMechanistic Interpretability Causal Interventions in Language Models
Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN 37232, USA
Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol
Jun 30, 2026Hussein Chouman, Wataru Sasaki, Tomokazu Matsui +2Transformer Interpretability Mechanistic Interpretability
Ubiquitous Computing Systems Laboratory, Nara Institute of Science and Technology, Nara, Japan.

Sep 2, 2026J/K move · Enter open · S save

ObserverBench: Testing Mechanistic Estimates for Intervention and Control

Vijay Erramilli

Mechanistic Interpretability AI Safety Evaluation Inference-Time Intervention Explainability Evaluation

Open paper AskPDF arXiv

Mechanistic interpretability is increasingly used to guide interventions such as activation steering, circuit removal, and safety monitoring. Yet an internal estimate that is accurate on average can still choose a poor action. We present ObserverBench, a benchmark framework for testing whether an internal estimator---an observer---is adequate for the intervention, control, or safety task it directs. Each task fixes the model, information boundary, allowed actions, decision rule, held-out cases, and loss. The benchmark reports estimation accuracy separately from the loss caused by the chosen action. Theory and experiments show why both are needed. In closed-loop control, observer errors matter at the starting point and along directions the allowed intervention can reach. On circuit-intervention tasks in GPT-2-small and Qwen2.5-7B, pairwise observers predict unseen effects more accurately without always choosing better actions; observers trained on action loss choose lower-loss actions. In safety triage, a score that perfectly separates violations can allocate a fixed intervention budget poorly when violations have different costs. Across Qwen2.5-7B, Gemma-2-9B-it, and prospectively frozen Qwen3.5-9B APPS tasks, AUROC can rank monitors differently from deployment loss, and the best information source changes across models. Sparse SAE readouts also trail their layer-matched dense controls on the reported Qwen panels, under disclosed activation-density or checkpoint mismatches. ObserverBench provides fixed task contracts, runnable baselines, and table-based submissions for evaluating interpretability methods through the actions they enable.

Abstract

Mechanistic Interpretability

Explore similar work

ObserverBench: Testing Mechanistic Estimates for Intervention and Control

Certified Interventional Fidelity: Anytime-Valid, Adaptive Evaluation of Causal Claims in Mechanistic Interpretability

Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol