Mechanistic interpretability aims to uncover how neural networks use their parameters to implement their impressive neural algorithms. Although previous work has uncovered substantial structure in the intermediate representations that networks use, little progress has been made to understand how the parameters and nonlinearities of networks perform computations on those representations. In this work, we present a method that brings us closer to this understanding by decomposing a language model's parameters into subcomponents that each implement only a small part of the model's learned algorithm, while simultaneously requiring only a small fraction of those subcomponents to account for the network's behavior on any input. The method, adVersarial Parameter Decomposition (VPD), optimizes for decompositions of neural network parameters into simple subcomponents that preserve the network's input-output behavior even when many subcomponents are ablated, including under ablations that are adversarially selected to destroy behavior. This encourages learning subcomponents that provide short, mechanistically faithful descriptions of the network's behavior that should aggregate appropriately into more global descriptions of the network's learned algorithm. We study how sequences of interactions between these parameter subcomponents produce the network's output on particular inputs, enabling a new kind of 'circuit' analysis. While more work remains to be done to deepen our understanding of how neural networks use their parameters to compute their behavior, our work suggests an approach to identify a small set of simple, mechanistically faithful subcomponents on which further mechanistic analysis can be based.
Figures & tables
Figure 1: Parameter decomposition methods decompose target model parameters into vectors in parameter space (parameter components) that are optimized to approximate the model’s mechanisms.
Figure 2: Our target model is a standard decoder-only transformer language model.
Attributes of our target model
Layers
4
Residual stream dimension
768
MLP intermediate dimension
3072
Attention heads
6
Attention head dimension
128
Context length
512
Table 1:
Layer
C
Alive
Mean L0
L0/Alive
Layer 0
9728
3709
44.6
0.012
Layer 1
9728
848
18.9
0.022
Layer 2
9728
1943
49.5
0.025
Layer 3
9728
3472
92.0
0.026
Total
38912
9972
205.0
0.021
Table 2: Table: Per-layer decomposition summary statistics: Subcomponent dictionary sizes C ; alive subcomponents (subcomponents with mean causal importances above 10−6 at the end of training); average L0 scores of subcomponents with causal importance >0 per batch and sequence position; and fraction of all subcomponents with causal importance >0 per batch and sequence position.
Masking mode (excluding Δ -components)
Validation CE Loss
Pretraining Compute Recovered (%)
Target Model
2.71
100%
Unmasked (All masks(=)1)
2.72
82.4%
Stochastic Masks
2.84
26.9%
Rounded Masks (Mask(=)1 if CI(>)0)
2.94
11.8%
Rounded Masks (Mask(=)1 if CI(>)0.1)
2.95
11.3%
Causal Importance values (CIs) used as Masks
2.99
9.4%
Table 3:
Adversarial optimization steps nadv
KL divergence to target model
20
0.8280
40
1.3539
80
3.8381
160
25.2560
320
40.2200
Table 4: While the decomposition is somewhat robust to approximately 20 steps of adversarial optimization, it is clearly not at all robust to 160 steps or more. 15 15 15 To provide some sense of scale, zero-ablating all of the target model’s weight matrices gives a KL-divergence of ca. 67.2449 .
Figure 3: Side-by-side generation comparisons across masking strategies. Static snapshot of an interactive figure; explore it at the web version .
Figure 4: CE degradation when simultaneously replacing all 4 MLP layers with sparse reconstructions from each method. (a) Active subcomponents per module (raw L0). (b) Active subcomponents per MLP reconstruction, adjusting for CLT’s cross-layer writes and VPD’s paired modules. (c) Total active parameters. VPD (purple markers) Pareto-dominates the activation-based methods under all three sparsity measures. The dashed line indicates zero-ablation (all MLP outputs set to zero). Lower is better.
Figure 5: Browse all VPD parameter subcomponents by weight matrix. Green highlights indicate causal importances; colored underlines show subcomponent activations. Static snapshot of an interactive figure; explore it at the web version .
Figure 6: Intruder detection scores for various CLT and PLT latents, and VPD subcomponents at different CI thresholds. Error bars are 95% bootstrap CIs on the mean. Dashed line is random chance accuracy (20%). Higher is better.
Figure 7: Number of alive subcomponents as a function of total subcomponent capacity. PLTs and CLTs scale roughly linearly with dictionary size, staying close to the y=x line. VPD (purple) remains flat at 6,500-7,000 alive subcomponents regardless of capacity, indicating that additional capacity is not used for feature splitting. Dashed line: y=x (all subcomponents alive).
Figure 8: The norm of the weights of each WQ and WK subcomponent in each head. No parameter subcomponent is exclusively localized in a single head, suggestive of computations that are distributed across attention heads.
Figure 9: The Standardized Static Interaction Strengths of pairs of parameter subcomponents in the Q and K projection matrices in each head (bottom grid) and all heads (top). The ten pairs with the largest interaction strengths at any offset are shown in color, with the rest in grey. The 1.attn.q:316 and 1.attn.k:329 pair exhibit strong positive Static Interaction Strength at early offsets, indicating this pair’s involvement in cross-head previous token behavior (and, more generally, ’recent token behavior’.
Figure 10: Attention scores Z illustrated as a sum of Data Dependent Interaction Strengths between pairs of subcomponents. Static snapshot of an interactive figure; explore it at the web version .
Figure 11: The attention score consists of a sum of Data Dependent Interaction Strengths. This panel shows the same prompt as the figure above, but here you can control which pairs of subcomponents to include in the sum, allowing you to study their individual effects on the reconstructed attention score and attention pattern. Static snapshot of an interactive figure; explore it at the web version .
Figure 12: Identifying the previous token head: Mean attention across multiple inputs on offset τ=1 , i.e. the previous token. Left : Average over sequences of random tokens, as per [ 66 ] . Right : Average over sequences sampled from the dataset. The plots reveal L1H1 is the most canonical “previous token head”. But note other heads place substantial average attention at offset τ=1 .
Figure 13: Effect of ablations: Ablating 1.attn.q:316 very strongly reduces attention to tokens in the recent past across all heads that otherwise attended there strongly. The effects of ablating other W_Q components has no distinguishable effect compared with the baseline and are therefore not shown. Here the baseline is the unablated average attention pattern.
Figure 14: Data-weighted cosine similarities between each head’s WOVh read and write matrices, and the cosine similarity between each head’s raw WOVh . Here, data-weighting uses data where subcomponent 1.attn.k:329 is causally important.
Figure 15: Data Dependent Interaction Strength for the 1.attn.k:119 - 1.attn.q:316 interaction only, and the corresponding attention pattern if only that interaction contributed to the attention pattern. On the right hand side, the ground truth attention pattern is shown for comparison. Static snapshot of an interactive figure; explore it at the web version .
Figure 16: Static snapshot of an interactive figure; explore it at the web version .
Figure 17: Static snapshot of an interactive figure; explore it at the web version .
Figure 18: Static snapshot of an interactive figure; explore it at the web version .
Figure 19: Static snapshot of an interactive figure; explore it at the web version .
Figure 20: Static snapshot of an interactive figure; explore it at the web version .
Figure 21: Static snapshot of an interactive figure; explore it at the web version .
Figure 22: Static snapshot of an interactive figure; explore it at the web version .
Figure 23: Static snapshot of an interactive figure; explore it at the web version .
Figure 24: To exclude indirect effects (i.e. effects that one parameter subcomponent has on another that are mediated by intermediate parameter subcomponents), we stop the gradients flowing through all subcomponents that are not the source subcomponent.
Figure 25: Attribution graph for predicting ·her on the prompt ‘‘The princess lost her crown.’’, pruned with adversarial sampling. 25 25 25 Coefficient 0.5 for cross-entropy reconstruction with stochastic sampling, coefficient 0.5 for cross entropy with 4 steps of PGD, lr 1 , importance minimality coefficient 0.09 , p=0.3 , 2000 optimization steps. There are 150 subcomponents in the graph. Static snapshot of an interactive figure; explore it at the web version .
Figure 26: Attribution graph for predicting ·her on the prompt ‘‘The princess lost her crown.’’, pruned with causal importance masking. 29 29 29 Coefficient 1.0 for cross-entropy reconstruction with causal importance masking, importance minimality coefficient 1.0 , p=0.3 , 2000 optimization steps. Static snapshot of an interactive figure; explore it at the web version .
Figure 27: Attribution graph for predicting ·his on the prompt ‘‘The prince lost his crown.’’, pruned with adversarial sampling. 35 35 35 Coefficient 0.5 for cross-entropy reconstruction with stochastic sampling, coefficient 0.5 for cross entropy with 4 steps of PGD, lr 1 , importance minimality coefficient 0.05 , p=0.3 , 2000 optimization steps. There are 160 subcomponents in the graph. The target model assigns probability 0.512 to ·his . Static snapshot of an interactive figure; explore it at the web version .
Figure 28: Attribution graph for predicting ·his on the prompt “The prince lost his crown.”, pruned with causal importance masking. Static snapshot of an interactive figure; explore it at the web version .
Figure 29: Attribution graph for predicting > after v on the prompt < u , v > , pruned with adversarial sampling. Static snapshot of an interactive figure; explore it at the web version .
Figure 30: Attribution graph for predicting > on the prompt < u , v > after u , pruned with adversarial sampling. 41 41 41 Coefficient 0.5 for cross-entropy reconstruction with stochastic sampling, coefficient 0.5 for cross entropy with 4 steps of PGD, lr 1 , importance minimality coefficient 0.1 , p=0.3 , 4000 optimization steps. There are 162 subcomponents in the graph. The target model assigns probability 0.119 to > . Static snapshot of an interactive figure; explore it at the web version .
Figure 31: Attribution graph for predicting > on the prompt < u , v > , pruned with causal importance masking. Static snapshot of an interactive figure; explore it at the web version .
Figure 32: Comparing VPD-based edits with a LoRA-augmented target model trained to complete emoticons with o . The orange background shows the per-token KL divergence between the edited model and the target model. The purple underline shows the probability assigned to o . The left panel shows the direct VPD-based manual edit, obtained by adding the unembedding vector for o to the emoticon subcomponent’s U vector (scaled by prefactor α=3 ). The right panel shows a LoRA-augmented target model trained on n=947 examples, each consisting of a token the emoticon subcomponent was causally important on and the 20 tokens either side of it. Static snapshot of an interactive figure; explore it at the web version .
Figure 33: Model editing for emoticon completions, LoRA vs. manual subcomponent edit. Manual edits were performed by adding the unembedding vector for o to the U vector of the emoticon subcomponent with different prefactors α . LoRAs were trained on n=10 and n=947 examples, each consisting of a token the emoticon subcomponent was causally important on, and the 20 tokens immediately preceding and following it, with a KL-regularisation term weighted by λ . The y-axis shows the average probability the edited model assigns to o on tokens the emoticon subcomponent is active on. The x-axis in the left plot shows DKL,Surrounding , the KL divergence between the edited model and the target model on the 20 other tokens immediately preceding and following tokens the emoticon subcomponent is causally important on, across a holdout set of 50 examples. The x-axis in the right plot shows DKL,Global , the average KL-divergence between the edited model and the target model on all other tokens across samples from the whole dataset.
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
CI model dimension ( dmodel )
2048
Transformer layers ( N )
8
Attention heads
16
Head dimension
128
FFN hidden dimension ( dff )
8192
Positional encoding
RoPE (base =10,000 , max length =512 )
Appendix
Table 5: Table: Hyperparameters for the causal importance function Γ 48 48 48 Recall that the target model is a 4-layer Llama-style transformer with dmodel=768 and dintermediate=3072 , decomposed across L=24 weight matrices (6 per layer: c_fc , down_proj , q_proj , k_proj , v_proj , o_proj ), yielding a total of Ctotal=38,912 subcomponents and an input dimension of D=27,648 .
Lfrequency-minimality (superlinear CI frequency penalty)
Equation 3
1×10−4
Appendix
Table 6: VPD loss terms and their coefficients. The importance minimality loss uses p -annealing from 2.0 to 0.4 . In practice we implement Limportance-minimality and Lfrequency-minimality as a single fused term with an inner weight β=0.5 on the frequency part (see above); this is functionally equivalent to the two losses with the coefficients shown. All reconstruction losses use KL divergence.
Module type
Subcomponents ( C ) per layer
c_fc (MLP Up-projection, 768×3072 )
3072
down_proj (MLP Down-projection, 3072×768 )
3584
q_proj (query projection, 768×768 )
512
k_proj (key projection, 768×768 )
512
v_proj (value projection, 768×768 )
1024
o_proj (output projection, 768×768 )
1024
Appendix
Table 7: Number of subcomponents per module type at initialization.
Loss
Value
Total
24.62
Delta-L2 (MSE)
0.00000240
StochasticReconSubsetLoss (KL)
0.2419
PersistentPGDReconLoss (KL)
0.5733
ImportanceMinimalityLoss
1102.0
Appendix
Table 8: Training losses (Measured at final step).
Loss
Value
StochasticReconSubsetLoss (KL)
0.2381
PGDReconLoss (KL)
0.9268
StochasticHiddenActsReconLoss (MSE)
0.4130
CIHiddenActsReconLoss (MSE)
0.8464
Appendix
Table 9: Evaluation reconstruction losses.
Used in
What
WandB link
Figure 4 ; Figure 35
PLT/CLT, local-MSE objective, k∈{8,16,32,64}
dict_4k , dict_32k
Figure 34
PLT/CLT, end-to-end KL objective, k∈{8,16,32,64} , three training modes ( cascading = error-propagating, parallel = clean-input, independent = single-layer)
5 VPD seed runs (otherwise identical to the main decomposition)
VPD multiseed
Table 11 — hidden-activation aux-loss VPD
VPD trained with an auxiliary stochastic-forward-pass hidden-activation MSE loss
VPD hidden-act run
Figure 7 ; Figure 35
VPD capacity sweep ( 0.5× , 1× , 2× , 4× subcomponents); 1× is the main run above
capacity_sweep
Appendix
Table 43
Figure 34: CE degradation vs. L0 (active features per module) for end-to-end KL-trained methods under three evaluation protocols. (a) Error-propagating: each encoder sees the modified residual stream. (b) Clean-input: each encoder sees the clean residual stream. (c) Single-layer replacement, averaged over layers. PLTs (blue) and CLTs (orange) perform well in their training mode but degrade by 5-20x in the mismatched mode. VPD (purple markers) is relatively stable across all three protocols. Linestyle indicates training mode: solid = error-propagating, dashed = clean-input, dotted = single-layer.
Figure 35: Cross-model latent/subcomponent splitting (decoder vector cosine similarity >0.5 ). Each cell shows the percentage of alive subcomponents or latents in the source model (row) that have more than one match in the target model (column). VPD shows low cosine similarity across models. PLTs and CLTs show high mutual cosine similarity, suggesting substantial splitting.
Method
Cross-seed Mean Max Cos Sim
VPD U vectors
0.4808
VPD V vectors
0.5156
PLT (e2e)
0.4390
CLT (e2e, parallel)
0.3468
TC (local MSE)
0.8063
CLT (local MSE)
0.6078
Appendix
Table 11:
Figure 36: Comparison between a decomposition with and without adversarial loss. The training configuration is otherwise identical. The CE loss is especially improved for small values of r .
Figure 37: The norm of the weights of each WV and WO subcomponent in each head.
Figure 38: The probability of each K subcomponent being active when a given V subcomponent is active, and vice versa. This tells us what WK subcomponents are responsible for moving information from particular kinds of WV subcomponents. The 1.attn.k:329 subcomponent is always active, and therefore moves all kinds of WV subcomponents.
Figure 39: Data-weighted cosine similarities between each head’s WOVh read- and write matrices, and the cosine similarity between each head’s raw WOVh . Here, data-weighting uses data where subcomponent 1.attn.k:119 is causally important.
Rank
Comp
Alignment
Label
1
v.22
74.9453
punctuation, syntax, and formatting tokens
2
v.984
68.3169
fires on punctuation and symbols
3
v.1000
64.4708
fires on punctuation, delimiters, and structural boundaries
4
v.346
62.6574
distinguishes function words (positive) and content words (negative)
5
v.568
59.1917
fires on word prefixes and partial words
Appendix
Table 51
Rank
Comp
Alignment
Label
1
o.923
231.5484
first token of the sequence
2
o.411
203.4490
code, markup, and technical formatting syntax
3
o.630
191.1107
punctuation, symbols, and syntax in technical text
4
o.753
181.5993
closing parentheses and brackets in code and math
5
o.300
173.9110
code and structured text syntax/indentation
Appendix
Table 52
Rank
Comp
Alignment
Label
1
v.984
59.9101
fires on punctuation and symbols
2
v.346
56.6177
distinguishes function words (positive) and content words (negative)
3
v.22
52.5358
punctuation, syntax, and formatting tokens
4
v.1000
52.4579
fires on punctuation, delimiters, and structural boundaries
5
v.946
45.8153
distinguishes content words from function words/symbols
Appendix
Table 53
Rank
Comp
Alignment
Label
1
o.362
167.0992
fires on names, citations, proper nouns and formatting tokens
2
o.490
164.5913
line start and indentation tokens
3
o.337
149.7576
fires inside parentheses or mathematical formulas
4
o.895
132.7422
variables, math symbols, and syntax in technical text
5
o.986
116.9061
sentence/paragraph boundaries and transition words
Appendix
Table 54
Rank
Comp
Alignment
Label
1
v.984
83.9813
fires on punctuation and symbols
2
v.22
79.5923
punctuation, syntax, and formatting tokens
3
v.346
69.2600
distinguishes function words (positive) and content words (negative)
4
v.1000
68.2968
fires on punctuation, delimiters, and structural boundaries
5
v.531
66.0043
opening parentheses, brackets, braces, and quotes
Appendix
Table 55
Rank
Comp
Alignment
Label
1
o.923
432.9877
first token of the sequence
2
o.578
249.4896
heterogeneous subcomponent / lack of clear pattern
3
o.180
227.2081
diffuse firing on tokens within words/phrases
4
o.866
208.1991
predicts newlines and separators at line ends
5
o.336
205.5378
diverges between function words and complex technical terms
Appendix
Table 56
Rank
Comp
Alignment
Label
1
v.984
70.1569
fires on punctuation and symbols
2
v.745
57.7404
formatting symbols, operators, and spatial alignment
3
v.946
53.8508
distinguishes content words from function words/symbols
4
v.494
53.3743
predicts line breaks or indentation in formatted text
5
v.346
52.9415
distinguishes function words (positive) and content words (negative)
Appendix
Table 57
Rank
Comp
Alignment
Label
1
o.311
333.3696
fires universally on most tokens
2
o.630
263.6867
punctuation, symbols, and syntax in technical text
3
o.37
257.2931
continuations of multi-token entities and compound words
4
o.300
251.2589
code and structured text syntax/indentation
5
o.180
177.4386
diffuse firing on tokens within words/phrases
Appendix
Table 58
Rank
Comp
Alignment
Label
1
v.346
146.1802
distinguishes function words (positive) and content words (negative)
2
v.22
145.5668
punctuation, syntax, and formatting tokens
3
v.984
142.4020
fires on punctuation and symbols
4
v.745
129.9418
formatting symbols, operators, and spatial alignment
5
v.1000
116.1570
fires on punctuation, delimiters, and structural boundaries
Appendix
Table 59
Rank
Comp
Alignment
Label
1
o.753
832.6620
closing parentheses and brackets in code and math
2
o.630
591.0163
punctuation, symbols, and syntax in technical text
3
o.411
538.0296
code, markup, and technical formatting syntax
4
o.860
522.9536
structural and formatting markers vs content words
5
o.292
503.4616
fires broadly on various tokens, promoting line breaks and punctuation
Appendix
Table 60
Rank
Comp
Alignment
Label
1
v.1000
57.2500
fires on punctuation, delimiters, and structural boundaries
2
v.946
56.3107
distinguishes content words from function words/symbols
3
v.346
53.8432
distinguishes function words (positive) and content words (negative)
4
v.428
51.4916
fragments of proper nouns, foreign, and technical words
5
v.984
50.2268
fires on punctuation and symbols
Appendix
Table 61
Rank
Comp
Alignment
Label
1
o.923
227.1903
first token of the sequence
2
o.311
178.6259
fires universally on most tokens
3
o.630
160.5296
punctuation, symbols, and syntax in technical text
4
o.180
156.3924
diffuse firing on tokens within words/phrases
5
o.300
144.0436
code and structured text syntax/indentation
Appendix
Table 62
Figure 40: Entries of the interaction matrix Ic,c′ for the layer 0 MLP Up projection matrix subcomponents. Entries greater than 1.0 are clamped to 1 . Indices are sorted by the components that each subcomponent belong to. Entries ≥1.0 in a row indicate that the nonlinear interaction between subcomponents c′ and c is large compared to the self-interaction of c on the diagonal. Many interactions are either block-diagonal, indicating they take place inside higher-rank components, or arranged along vertical and horizontal lines, indicating they are caused by a relatively small number of highly interacting components.
Figure 41: Histogram of matrix entries of the interaction matrix Ic,c′ for the layer 0 up projection subcomponents. Most entries are much smaller than the self-interaction 1.0 .
Figure 42: Entries of the interaction matrix Ic,c′ for the layer 1 MLP Up projection subcomponents. Indices are sorted by the components subcomponents belong to.
Figure 43: Histogram of the entries of the interaction matrix Ic,c′ for the layer 1 MLP Up projection subcomponents.
Figure 44: Entries of the interaction matrix Ic,c′ for the layer 2 MLP Up projection subcomponents. Indices are sorted by the components subcomponents belong to.
Figure 45: Histogram of the entries of the interaction matrix Ic,c′ for the layer 2 MLP Up projection subcomponents.
Figure 46: Entries of the interaction matrix Ic,c′ for the layer 3 MLP Up projection subcomponents. Indices are sorted by the components subcomponents belong to.
Figure 47: Histogram of the entries of the interaction matrix Ic,c′ for the layer 3 MLP Up projection subcomponents.
Activation space and parameter space provide complementary views of model computation. Activations represent information, while weights read, transform, and write that information. Yet existing interpretability methods largely study the two spaces separately, leaving the connection between represented information and parameter-level computation underexplored. We introduce Activation-Supported Parameter Decomposition (ASPD), which jointly decomposes activation and parameter spaces and grounds each learned weight component in the activation features it reads or writes. This grounding constrains otherwise non-unique parameter decompositions using the model's internal activations, while an internal reconstruction objective provides a local learning signal at the weight matrix being analyzed. Together, these properties enable scalable, interpretable, and causally editable parameter decomposition in pretrained large language models, demonstrated on Qwen-3-8B. The learned read--write components can also be composed into parameter-level mechanism circuits. We use ASPD to recover mechanisms underlying the classic IOI circuit and trace semantic transformations through model weights.
Tue M. Cao, Lisiane Pruinelli, My T. Thai
Department of Computer and Information Science and Engineering, University of Florida, Gainesville, FL 32611, USA · College of Nursing, University of Florida, Gainesville, FL 32611, USA
Parameter decomposition (PD) decomposes neural networks into interpretable computational components that faithfully reflect the original network's operations. However, scaling PD to large models requires vast compute, making it a costly and risky endeavor. Here we propose targeted PD (tPD), which identifies only the components that process specific inputs of interest -- from isolated prompts to large subtasks -- by introducing a high-rank catch-all component that handles all non-target data. We validate tPD on toy models and on transformer language models trained on The Pile, where it recovers reproducible, mechanistically faithful circuits. We extract a CSS-only submodel of a 4-block transformer using 7% of the FLOPs of its published decomposition, and in a 12-block transformer we surgically ablate and rewire memorized sequences, with negligible side effects on other inputs.
Understanding parameter functions helps elucidate the internal mechanisms of large language models (LLMs). However, how to connect parameters from different modules to verifiable output effects and further characterize the relationship between their functional organization and model capability remains to be explored. To this end, we introduce the downstream vocabulary lens (DV-Lens), a parameter-level interpretability framework that links native parameter directions to their downstream vocabulary responses. Specifically, we first estimate module-specific downstream Jacobians over a reference prompt set for attention query, key, value, and output (Q/K/V/O) projections and feed-forward networks (FFNs). Second, we use these mappings to project native parameter columns into the final vocabulary space, obtaining signed readouts that characterize their average local output responses. Third, we group parameter columns by their vocabulary readouts and introduce downstream vocabulary complexity (DV-Complexity), which quantifies within-group structural variation using normalized reconstruction residuals of the original weights. At the parameter level, randomized controls and finite-difference tests show that DV-Lens readouts capture non-random vocabulary structure and predict local logit changes with 98.0% coordinate-orientation agreement across 720 cases from nine models. These readouts further guide parameter ablation, steering, and swapping across 21 models, shifting target-token probabilities in the predicted directions under controlled conditions. At the model level, the joint-parameter score of DV-Complexity achieves a Spearman correlation of 0.904 with benchmark-based capability rankings across 48 language models. Together, these results provide intervention-based evidence for DV-Lens interpretations and reveal an association between DV-Complexity and model capability.
Chenhang Cui, Jian Yu, Shuyi Miao +5
National University of Singapore · Nanjing University of Science and Technology · Beihang University +2