Can Cross-Layer Transcoders Replace Vision Transformer Activations? An Interpretable Perspective on Vision

Gerasimos Chatzoudis; Konstantinos D. Polyzos; Zhuowei Li; Difei Gu; Gemma E. Moran; Hao Wang; Dimitris N. Metaxas

cs.CVApr 14, 2026

Can Cross-Layer Transcoders Replace Vision Transformer Activations? An Interpretable Perspective on Vision

Authors: Gerasimos Chatzoudis, Konstantinos D. Polyzos, Zhuowei Li, Difei Gu, Gemma E. Moran, Hao Wang, Dimitris N. Metaxas

Organizations: Rutgers University · University of California San Diego

Abstract

Understanding the internal activations of Vision Transformers (ViTs) is critical for building interpretable and trustworthy models. While Sparse Autoencoders (SAEs) have been used to extract human-interpretable features, they operate on individual layers and fail to capture the cross-layer computational structure of Transformers, as well as the relative significance of each layer in forming the last-layer representation. Alternatively, we introduce the adoption of Cross-Layer Transcoders (CLTs) as reliable, sparse, and depth-aware proxy models for MLP blocks in ViTs. CLTs use an encoder-decoder scheme to reconstruct each post-MLP activation from learned sparse embeddings of preceding layers, yielding a linear decomposition that transforms the final representation of ViTs from an opaque embedding into an additive, layer-resolved construction that enables faithful attribution and process-level interpretability. We train CLTs on CLIP ViT-B/32 and ViT-B/16 across CIFAR-100, COCO, and ImageNet-100. We show that CLTs achieve high reconstruction fidelity with post-MLP activations while preserving and even improving, in some cases, CLIP zero-shot classification accuracy. In terms of interpretability, we show that the cross-layer contribution scores provide faithful attribution, revealing that the final representation is concentrated in a smaller set of dominant layer-wise terms whose removal degrades performance and whose retention largely preserves it. These results showcase the significance of adopting CLTs as an alternative interpretable proxy of ViTs in the vision domain.

Figures & tables

Figure 1: Overview of the Cross-Layer Transcoder (CLT) framework. Left : Each CLT encodes LN2 activations into sparse codes $z_{\ell}$ and reconstructs MLP outputs $y_{\ell}$ via triangular decoders. Middle : During inference, CLTs replace MLPs across layers, preserving zero-shot performance. Right : Using CLTs as an interpretable proxy for ViTs to understand the cross-layer contributions of different token types and identify the most significant layers in shaping the ViT’s final representations.

Figure 2: Reconstruction performance of Cross-Layer Transcoders (CLTs) across transformer layers on CIFAR-100 using ViT-B/32. We compare three sparsity variants, i.e., JumpReLU , ReLU-Top-

k

, and Abs-Top-

k

(with

k{=}128

) using (left) cosine similarity, (center) mean squared error (MSE per token, log scale), and (right) variance explained (

R^{2}

). Metrics are averaged over all tokens in the test set.

CIFAR-100
Dataset	Backbone	Tokens	3 $\rightarrow$ 11			7 $\rightarrow$ 11			10 $\rightarrow$ 11			All
			JR	RTK	ATK	JR	RTK	ATK	JR	RTK	ATK	JR	RTK	ATK
	ViT-B/32	CLS	61.38	61.69	61.58	61.41	61.86	61.26	61.06	61.39	61.16	61.33	61.74	61.43
		Patches	53.65	54.20	55.83	61.51	61.69	63.18	61.49	61.25	61.56	49.63	51.12	48.68
		All	54.10	54.84	55.69	61.54	62.08	62.85	61.01	61.20	61.11	49.40	51.12	48.41
	ViT-B/16	CLS	66.10	65.81	65.39	66.15	66.15	65.86	66.06	65.90	65.90	66.04	65.92	65.38

Table 3: Top-1 classification accuracy (%) across datasets, backbones, and token types. Columns correspond to layer range and sparsity combinations: JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. For reference, baseline top-1 accuracies (in %) are: CIFAR-100 (B/32: 61.65, B/16: 65.97), COCO (B/32: 43.12, B/16: 43.56), and ImageNet-100 (B/32: 80.42, B/16: 84.34).

Panel of Figure 3 — Figure 3: Cross-layer attribution scores $C^{\mathrm{proj}}_{i\rightarrow j}$ for ViT-B/16 on CIFAR-100, averaged over the validation set. Scores along each column yield a partition of attribution across depth. Patch tokens (right) exhibit strong diagonal dominance, while [CLS] (left) draws attribution broadly across source layers.

Model	Dataset	Tokens	Baseline		Full CLT		Drop Top-1		Keep Top-4
Model	Dataset	Tokens	Acc $\uparrow$	KL $\downarrow$	Acc $\uparrow$	KL $\downarrow$	Acc $\uparrow$	KL $\downarrow$	Acc $\uparrow$	KL $\downarrow$
ViT-B/32	cifar100	all (cls+patch)	61.65%	-	61.31%	0.0104	59.68%	0.0800	59.96%	0.0714
		cls	61.65%	-	61.31%	0.0104	59.91%	0.0704	60.72%	0.0582
	coco	all (cls+patch)	43.12%	-	43.26%	0.0103	42.98%	0.1619	43.02%	0.0156
		cls	43.12%	-	43.26%	0.0103	42.78%	0.1157	43.00%	0.0155
	imagenet100	all (cls+patch)	80.42%	-	80.54%	0.0073	74.94%	0.2320	80.56%	0.0109

Table 4: Faithful attribution via projection-based ablation on the final layer of ViT-B/32. Accuracy (%) and KL divergence from the unmodified logits are reported for [CLS] and all tokens ( [CLS] +patch).

Figure 4: Layerwise visual retrieval using CLT sparse codes of CLS for a test image. Each row shows the top-3 retrieved training samples across transformer layers, revealing the semantic evolution of representations.

Figure 5: Layerwise visual retrieval using CLT sparse codes of Patches for a test image. Each row shows the top-3 retrieved training samples across transformer layers, revealing the semantic evolution of representations.

	Pred. Distrib.			Top-k Agree		Embedding			Prompt Sens.
Layers	$\Delta$ Acc	Flip $\downarrow$	KL $\downarrow$	Top-1	Top-5	Cos	CKA	Spear.	$r$	KL $\downarrow$
0-11 CLS	+0.07	10.4%	.034	89.6	88.5	.984	.948	.929	.995	.036
7-11 CLS	+0.01	9.5%	.030	90.5	89.0	.985	.954	.937	.992	.031
10-11 CLS	$-$ 0.07	7.2%	.018	92.8	91.2	.991	.979	.973	.994	.019
11 CLS	$-$ 0.04	5.4%	.010	94.6	93.2	.996	.991	.991	.997	.010
0-11 Patches	$-$ 11.1	35.2%	.412	64.8	70.0	.957	.817	.759	.942	.383

Table 5: CLT surrogate faithfulness metrics.

Figure 6: Reconstruction performance of CLTs across transformer layers on CIFAR-100 using ViT-B/16. We report cosine similarity (left), MSE per token in log scale (center), and

R^{2}

(right) for JumpReLU , ReLU-Top-

k

, and Abs-Top-

k

sparsity variants (

k{=}128

), averaged across all tokens in the validation set.

Figure 7: Reconstruction performance of CLTs across transformer layers on COCO using ViT-B/32. We show cosine similarity (left), MSE per token (log scale, center), and

R^{2}

(right) for the three sparsity variants JumpReLU , ReLU-Top-

k

, and Abs-Top-

k

(

k{=}128

), averaged across all tokens in the validation set.

Figure 8: Reconstruction performance of CLTs across transformer layers on COCO using ViT-B/16. As in the main paper, we compare JumpReLU , ReLU-Top-

k

, and Abs-Top-

k

(

k{=}128

) with cosine similarity (left), MSE per token (log scale, center), and

R^{2}

(right), averaged over the validation set.

Figure 9: Reconstruction performance of CLTs across transformer layers on ImageNet-100 using ViT-B/32. We plot cosine similarity (left), MSE per token in log scale (center), and

R^{2}

(right) for the three sparsity variants JumpReLU , ReLU-Top-

k

, and Abs-Top-

k

(

k{=}128

), averaged across all tokens in the validation set.

Figure 10: Reconstruction performance of CLTs across transformer layers on ImageNet-100 using ViT-B/16. Cosine similarity (left), MSE per token (log scale, center), and

R^{2}

(right) are reported for JumpReLU , ReLU-Top-

k

, and Abs-Top-

k

sparsity (

k{=}128

), averaged across all tokens in the test set.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	61.33	61.74	61.43
1 $\rightarrow$ 11	61.43	61.76	61.56
2 $\rightarrow$ 11	61.39	61.72	61.61
3 $\rightarrow$ 11	61.38	61.69	61.58
4 $\rightarrow$ 11	61.20	61.76	61.66
5 $\rightarrow$ 11	61.21	61.59	61.48

Table 6: Top-1 classification accuracy (%) on CIFAR-100 for ViT-B/32 with CLS tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 61.65 and ViT-B/16: 65.97.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	49.63	51.12	48.68
1 $\rightarrow$ 11	49.92	51.61	52.40
2 $\rightarrow$ 11	51.95	52.86	53.46
3 $\rightarrow$ 11	53.65	54.20	55.83
4 $\rightarrow$ 11	56.59	57.02	58.86
5 $\rightarrow$ 11	58.89	59.52	61.38

Table 7: Top-1 classification accuracy (%) on CIFAR-100 for ViT-B/32 with Patches tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 61.65 and ViT-B/16: 65.97.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	49.40	51.12	48.41
1 $\rightarrow$ 11	49.90	51.91	51.97
2 $\rightarrow$ 11	51.84	53.51	53.16
3 $\rightarrow$ 11	54.10	54.84	55.69
4 $\rightarrow$ 11	56.62	57.32	58.23
5 $\rightarrow$ 11	59.21	59.96	61.19

Table 8: Top-1 classification accuracy (%) on CIFAR-100 for ViT-B/32 with All tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 61.65 and ViT-B/16: 65.97.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	66.04	65.92	65.38
1 $\rightarrow$ 11	66.05	65.82	65.62
2 $\rightarrow$ 11	66.02	65.78	65.73
3 $\rightarrow$ 11	66.10	65.81	65.39
4 $\rightarrow$ 11	66.05	65.87	65.61
5 $\rightarrow$ 11	66.15	65.80	65.65

Table 9: Top-1 classification accuracy (%) on CIFAR-100 for ViT-B/16 with CLS tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 61.65 and ViT-B/16: 65.97.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	62.40	58.57	56.59
1 $\rightarrow$ 11	62.58	59.40	60.68
2 $\rightarrow$ 11	63.15	60.07	61.97
3 $\rightarrow$ 11	63.54	60.76	63.31
4 $\rightarrow$ 11	64.29	62.32	64.85
5 $\rightarrow$ 11	65.24	63.50	66.11

Table 10: Top-1 classification accuracy (%) on CIFAR-100 for ViT-B/16 with Patches tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 61.65 and ViT-B/16: 65.97.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	62.45	58.82	56.72
1 $\rightarrow$ 11	62.70	59.23	60.30
2 $\rightarrow$ 11	62.95	59.74	61.54
3 $\rightarrow$ 11	63.71	60.75	62.87
4 $\rightarrow$ 11	64.14	62.16	64.57
5 $\rightarrow$ 11	65.20	63.60	65.86

Table 11: Top-1 classification accuracy (%) on CIFAR-100 for ViT-B/16 with All tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 61.65 and ViT-B/16: 65.97.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	43.12	43.36	43.00
1 $\rightarrow$ 11	43.14	43.32	43.08
2 $\rightarrow$ 11	43.10	43.26	42.96
3 $\rightarrow$ 11	43.08	43.22	43.12
4 $\rightarrow$ 11	43.18	43.22	43.06
5 $\rightarrow$ 11	43.22	43.30	43.06

Table 12: Top-1 classification accuracy (%) on COCO for ViT-B/32 with CLS tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 43.12 and ViT-B/16: 43.56.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	39.04	38.94	39.68
1 $\rightarrow$ 11	39.38	39.44	40.02
2 $\rightarrow$ 11	39.54	40.24	40.42
3 $\rightarrow$ 11	40.58	40.92	41.48
4 $\rightarrow$ 11	41.38	41.88	42.10
5 $\rightarrow$ 11	42.06	42.28	42.28

Table 13: Top-1 classification accuracy (%) on COCO for ViT-B/32 with Patches tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 43.12 and ViT-B/16: 43.56.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	38.60	39.12	40.06
1 $\rightarrow$ 11	39.02	39.62	40.54
2 $\rightarrow$ 11	39.98	40.10	40.44
3 $\rightarrow$ 11	41.00	41.34	41.46
4 $\rightarrow$ 11	41.62	41.68	41.92
5 $\rightarrow$ 11	41.88	42.28	42.26

Table 14: Top-1 classification accuracy (%) on COCO for ViT-B/32 with All tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 43.12 and ViT-B/16: 43.56.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	43.62	43.62	42.76
1 $\rightarrow$ 11	43.56	43.62	42.40
2 $\rightarrow$ 11	43.60	43.64	42.34
3 $\rightarrow$ 11	43.68	43.64	42.64
4 $\rightarrow$ 11	43.66	43.72	42.64
5 $\rightarrow$ 11	43.58	43.80	42.98

Table 15: Top-1 classification accuracy (%) on COCO for ViT-B/16 with CLS tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 43.12 and ViT-B/16: 43.56.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	42.72	42.00	35.46
1 $\rightarrow$ 11	42.84	41.98	37.28
2 $\rightarrow$ 11	42.68	42.20	38.36
3 $\rightarrow$ 11	43.00	42.34	39.46
4 $\rightarrow$ 11	43.00	42.96	40.86
5 $\rightarrow$ 11	42.98	43.00	42.58

Table 16: Top-1 classification accuracy (%) on COCO for ViT-B/16 with Patches tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 43.12 and ViT-B/16: 43.56.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	43.00	42.08	34.16
1 $\rightarrow$ 11	42.92	42.08	36.06
2 $\rightarrow$ 11	42.74	42.20	36.82
3 $\rightarrow$ 11	42.78	42.66	38.18
4 $\rightarrow$ 11	42.84	43.10	39.80
5 $\rightarrow$ 11	43.04	43.12	42.22

Table 17: Top-1 classification accuracy (%) on COCO for ViT-B/16 with All tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 43.12 and ViT-B/16: 43.56.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	80.92	80.86	80.26
1 $\rightarrow$ 11	80.84	80.82	80.32
2 $\rightarrow$ 11	80.82	80.86	80.24
3 $\rightarrow$ 11	80.86	80.78	80.16
4 $\rightarrow$ 11	80.80	80.86	80.24
5 $\rightarrow$ 11	80.74	80.86	80.44

Table 18: Top-1 classification accuracy (%) on ImageNet-100 for ViT-B/32 with CLS tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 80.42 and ViT-B/16: 84.34.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	71.96	68.74	60.26
1 $\rightarrow$ 11	72.34	70.10	64.80
2 $\rightarrow$ 11	73.10	72.24	67.54
3 $\rightarrow$ 11	75.58	75.26	72.46
4 $\rightarrow$ 11	77.86	77.18	75.88
5 $\rightarrow$ 11	78.92	79.10	77.54

Table 19: Top-1 classification accuracy (%) on ImageNet-100 for ViT-B/32 with Patches tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 80.42 and ViT-B/16: 84.34.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	71.60	68.90	60.26
1 $\rightarrow$ 11	71.92	70.10	63.82
2 $\rightarrow$ 11	72.88	72.32	66.72
3 $\rightarrow$ 11	75.12	75.36	71.44
4 $\rightarrow$ 11	77.56	77.10	74.68
5 $\rightarrow$ 11	78.74	78.84	77.30

Table 20: Top-1 classification accuracy (%) on ImageNet-100 for ViT-B/32 with All tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 80.42 and ViT-B/16: 84.34.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	84.54	84.02	83.28
1 $\rightarrow$ 11	84.50	84.04	83.52
2 $\rightarrow$ 11	84.44	83.98	83.50
3 $\rightarrow$ 11	84.56	84.08	83.74
4 $\rightarrow$ 11	84.44	84.04	83.82
5 $\rightarrow$ 11	84.42	84.10	84.20

Table 21: Top-1 classification accuracy (%) on ImageNet-100 for ViT-B/16 with CLS tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 80.42 and ViT-B/16: 84.34.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	83.04	81.40	73.06
1 $\rightarrow$ 11	83.00	81.44	76.74
2 $\rightarrow$ 11	83.18	81.88	77.54
3 $\rightarrow$ 11	83.16	82.76	79.36
4 $\rightarrow$ 11	83.56	83.34	80.98
5 $\rightarrow$ 11	83.78	83.88	82.50

Table 22: Top-1 classification accuracy (%) on ImageNet-100 for ViT-B/16 with Patches tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 80.42 and ViT-B/16: 84.34.

Range	JR	RTK	ATK
0 $\rightarrow$ 11	83.12	80.94	70.78
1 $\rightarrow$ 11	83.08	81.58	74.70
2 $\rightarrow$ 11	83.28	81.90	76.30
3 $\rightarrow$ 11	83.00	82.64	78.08
4 $\rightarrow$ 11	83.62	83.08	80.12
5 $\rightarrow$ 11	84.16	83.86	81.76

Table 23: Top-1 classification accuracy (%) on ImageNet-100 for ViT-B/16 with All tokens, across layer ranges

s\rightarrow 11

. JR = JumpReLU, RTK = ReLU-Top-

k

, ATK = Abs-Top-

k

. Baseline top-1 accuracies (in %) are ViT-B/32: 80.42 and ViT-B/16: 84.34.

Panel of Figure 11 — Figure 11: Cross-layer contribution scores $C_{s\rightarrow\ell}$ on CIFAR-100 with ViT-B/32. Columns vary the sparsifier (JumpReLU, ReLU-Top- $k$ , Abs-Top- $k$ ) and rows show CLS (top) and patch tokens (bottom). Each heatmap visualizes the proportional contribution of source layer $s$ to the reconstructed activation at target layer $\ell$ , averaged over the validation set.

Panel of Figure 12 — Figure 12: Cross-layer contribution scores $C_{s\rightarrow\ell}$ on CIFAR-100 with ViT-B/16. Columns vary the sparsifier (JumpReLU, ReLU-Top- $k$ , Abs-Top- $k$ ) and rows show CLS (top) and patch tokens (bottom). Each heatmap visualizes the proportional contribution of source layer $s$ to the reconstructed activation at target layer $\ell$ , averaged over the validation set.

Panel of Figure 13 — Figure 13: Cross-layer contribution scores $C_{s\rightarrow\ell}$ on COCO with ViT-B/32. Columns vary the sparsifier (JumpReLU, ReLU-Top- $k$ , Abs-Top- $k$ ) and rows show CLS (top) and patch tokens (bottom). Each heatmap visualizes the proportional contribution of source layer $s$ to the reconstructed activation at target layer $\ell$ , averaged over the validation set.

Panel of Figure 14 — Figure 14: Cross-layer contribution scores $C_{s\rightarrow\ell}$ on COCO with ViT-B/16. Columns vary the sparsifier (JumpReLU, ReLU-Top- $k$ , Abs-Top- $k$ ) and rows show CLS (top) and patch tokens (bottom). Each heatmap visualizes the proportional contribution of source layer $s$ to the reconstructed activation at target layer $\ell$ , averaged over the validation set.

Panel of Figure 15 — Figure 15: Cross-layer contribution scores $C_{s\rightarrow\ell}$ on ImageNet-100 with ViT-B/32. Columns vary the sparsifier (JumpReLU, ReLU-Top- $k$ , Abs-Top- $k$ ) and rows show CLS (top) and patch tokens (bottom). Each heatmap visualizes the proportional contribution of source layer $s$ to the reconstructed activation at target layer $\ell$ , averaged over the validation set.

Panel of Figure 16 — Figure 16: Cross-layer contribution scores $C_{s\rightarrow\ell}$ on ImageNet-100 with ViT-B/16. Columns vary the sparsifier (JumpReLU, ReLU-Top- $k$ , Abs-Top- $k$ ) and rows show CLS (top) and patch tokens (bottom). Each heatmap visualizes the proportional contribution of source layer $s$ to the reconstructed activation at target layer $\ell$ , averaged over the validation set.

Explore similar work

CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
Mar 22, 2026Florent Draye, Vedant Palit, Abir Harrasse +8LLM Interpretability Mechanistic Interpretability
Max Planck Institute for Intelligent Systems, Tübingen, Germany · Jinesis AI Lab, University of Toronto & Vector Institute · CMU +2
SpIn-ViT: Designing a Sparsity-Induced Vision Transformer That Is Mechanistically Interpretable
Aug 14, 2026Philip H. Lee, Parth PadalkarExplainable Image Classification Vision Transformer
Independent Researcher · Texas State University
"World Knowledge" in the Weights: Reading Concept Circuits of Vision Transformers
Sep 8, 2026Yanlin Chen, Tang Li, Xi PengTransformer Interpretability Vision Foundation Models
Department of Computer & Information Sciences, University of Delaware, USA · Department of Computer Science, University of Virginia, USA

Mar 22, 2026J/K move · Enter open · S save

CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs

Florent Draye, Vedant Palit, Abir Harrasse, Tung-Yu Wu+7

Max Planck Institute for Intelligent Systems, Tübingen, Germany · Jinesis AI Lab, University of Toronto & Vector Institute · CMU+2

LLM Interpretability Mechanistic Interpretability Feature Attribution

Open paper AskPDF arXiv

Mechanistic interpretability seeks to understand how Large Language Models (LLMs) represent and process information. Recent approaches based on dictionary learning and transcoders enable representing model computation in terms of sparse, interpretable features and their interactions, giving rise to feature attribution graphs. However, these graphs are often large and redundant, limiting their interpretability in practice. Cross-Layer Transcoders (CLTs) address this issue by sharing features across layers while preserving layer-specific decoding, yielding more compact representations, but remain difficult to train and analyze at scale. We introduce an open-source library for end-to-end training and interpretability of CLTs. Our framework integrates scalable distributed training with model sharding and compressed activation caching, a unified automated interpretability pipeline for feature analysis and explanation, attribution graph computation using Circuit-Tracer, and a flexible visualization interface. This provides a practical and unified solution for scaling CLT-based mechanistic interpretability. Our code is available at: https://github.com/LLM-Interp/CLT-Forge.

Abstract

Explore similar work

CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs

SpIn-ViT: Designing a Sparsity-Induced Vision Transformer That Is Mechanistically Interpretable

"World Knowledge" in the Weights: Reading Concept Circuits of Vision Transformers