The SuperActivator Mechanism: Transformers Concentrate Reliable Concept Signals in the Tail
Organizations: University of Pennsylvania
Abstract
Concept vectors aim to enhance model interpretability by linking learned internal representations with human-understandable semantics, but their practical utility is often limited by noisy and inconsistent activations. In this work, we discover the SuperActivator Mechanism: a transformer dynamic that amplifies concept activation gaps, concentrating the most reliable concept evidence into a small set of highly activated tokens, which we call SuperActivators. We theoretically characterize this mechanism, proving that concept-aligned attention produces two competing effects: a tail amplification effect, where gaps among already-high activations grow disproportionately fast, and an equalization effect that eventually dominates as attention concentrates on extreme tokens. We validate the predicted tail amplification empirically, finding that in-concept activation distributions develop well-separated upper tails consistently across models, image and text modalities, and concept types. Moreover, we demonstrate the practical implications of SuperActivators for concept detection and localization. First, using only high-activation tokens as concept indicators improves detection by up to 14 F1 points over competing baselines. Second, restricting detection to the extreme activation tail is not merely sufficient but essential: including activations beyond the top 5-10% of the in-concept distribution introduces noise that degrades performance. Finally, using SuperActivators as instance-specific attribution targets yields more accurate and faithful concept localization heatmaps than using global concept vectors.
Figures & tables
| Concept Detection Methods | ||||||
| RandTok | LastTok Chen et al. (2025) | MeanTok ( McKenzie et al., 2025 ) | CLSTok ( Yu et al., 2024 ) | Prompt ( Wu et al., 2025 ) | SuperAct | |
| COCO | 0.41 | 0.36 | 0.67 | 0.57 | 0.69 | 0.83 |
| OpenSurfaces | 0.38 | 0.37 | 0.51 | 0.46 | 0.49 | 0.57 |
| Pascal | 0.45 | 0.46 | 0.71 | 0.64 | 0.68 | 0.82 |
| PERSUADE | 0.42 | 0.42 | 0.51 | 0.55 | 0.29 | 0.64 |
| El Capitan | 0.47 | 0.45 | 0.52 | 0.58 | 0.51 | 0.70 |
| Attribution Method | Dataset | Attribution ( is better) | Insertion Score ( is better) | Deletion Score ( is better) | |||
| Concept | SuperActs | Concept | SuperActs | Concept | SuperActs | ||
| LIME [-0.25ex] ( Ribeiro et al., 2016 ) | COCO | ||||||
| PERSUADE | |||||||
| SHAP [-0.25ex] ( Lundberg and Lee, 2017 ) | COCO | ||||||
| PERSUADE | |||||||
| RISE [-0.25ex] ( Petsiuk et al., 2018 ) | COCO | ||||||
Appendix figures & tables58 assets
Supplementary material from the paper’s appendix.
Appendix
| Baseline | COCO | Open Surfaces | Pascal | PERSUADE | El Capitan | GoEmotions | CLEVR | iSarcasm |
| Constant Positive | ||||||||
| Random |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| CLIP | Avg | 0.368 0.013 | 0.347 0.012 | 0.531 0.014 | 0.679 0.012 | – | 0.718 0.012 |
| Linsep | 0.514 0.014 | 0.516 0.014 | 0.681 0.013 | 0.702 0.012 | – | 0.791 0.011 | |
| K-Means | 0.350 0.013 | 0.350 0.013 | 0.543 0.014 | 0.582 0.012 | – | 0.701 0.012 | |
| K-Linsep | 0.347 0.013 | 0.368 0.014 | 0.591 0.014 | 0.570 0.013 | – | 0.713 0.012 | |
| Llama | Avg | 0.340 0.012 | 0.315 0.011 | 0.551 0.013 | 0.531 0.014 | 0.686 0.010 | 0.754 0.012 |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| CLIP | Avg | 0.360 0.013 | 0.354 0.012 | 0.465 0.014 | 0.488 0.014 | – | 0.546 0.015 |
| Linsep | 0.416 0.014 | 0.413 0.013 | 0.523 0.014 | 0.499 0.014 | – | 0.557 0.014 | |
| K-Means | 0.340 0.013 | 0.339 0.013 | 0.469 0.014 | 0.437 0.014 | – | 0.533 0.015 | |
| K-Linsep | 0.348 0.013 | 0.345 0.013 | 0.481 0.015 | 0.432 0.014 | – | 0.530 0.015 | |
| Llama | Avg | 0.352 0.012 | 0.332 0.011 | 0.480 0.014 | 0.439 0.014 | 0.491 0.014 | 0.533 0.014 |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| CLIP | Avg | 0.426 0.007 | 0.420 0.006 | 0.581 0.007 | 0.720 0.006 | – | 0.792 0.006 |
| Linsep | 0.497 0.007 | 0.492 0.006 | 0.698 0.006 | 0.746 0.006 | – | 0.824 0.005 | |
| K-Means | 0.423 0.007 | 0.443 0.007 | 0.614 0.007 | 0.652 0.006 | – | 0.786 0.006 | |
| K-Linsep | 0.414 0.007 | 0.450 0.007 | 0.648 0.007 | 0.636 0.007 | – | 0.774 0.006 | |
| Llama | Avg | 0.403 0.006 | 0.393 0.006 | 0.624 0.007 | 0.622 0.007 | 0.680 0.006 | 0.786 0.006 |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| Llama | Avg | 0.412 0.020 | 0.401 0.020 | 0.484 0.024 | 0.486 0.026 | 0.286 0.026 | 0.591 0.025 |
| Linsep | 0.422 0.024 | 0.419 0.022 | 0.510 0.025 | 0.551 0.027 | 0.286 0.026 | 0.639 0.026 | |
| K-Means | 0.398 0.020 | 0.400 0.021 | 0.434 0.021 | 0.421 0.021 | 0.286 0.026 | 0.605 0.024 | |
| K-Linsep | 0.396 0.020 | 0.405 0.020 | 0.445 0.024 | 0.421 0.021 | 0.286 0.026 | 0.608 0.026 | |
| Qwen | Avg | 0.426 0.021 | 0.403 0.019 | 0.477 0.022 | 0.459 0.025 | 0.493 0.022 | 0.525 0.024 |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| Llama | Avg | 0.430 0.023 | 0.434 0.023 | 0.505 0.024 | 0.503 0.026 | 0.512 0.029 | 0.525 0.023 |
| Linsep | 0.471 0.025 | 0.452 0.023 | 0.524 0.025 | 0.580 0.026 | 0.512 0.029 | 0.696 0.022 | |
| K-Means | 0.407 0.024 | 0.437 0.024 | 0.569 0.026 | 0.494 0.026 | 0.512 0.029 | 0.682 0.022 | |
| K-Linsep | 0.397 0.024 | 0.446 0.024 | 0.540 0.027 | 0.494 0.026 | 0.512 0.029 | 0.679 0.022 | |
| Qwen | Avg | 0.421 0.022 | 0.434 0.024 | 0.515 0.024 | 0.535 0.026 | 0.533 0.021 | 0.674 0.022 |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| Llama | Avg | 0.131 0.021 | 0.172 0.021 | 0.177 0.021 | 0.274 0.026 | 0.252 0.028 | 0.330 0.026 |
| Linsep | 0.187 0.026 | 0.210 0.023 | 0.275 0.025 | 0.348 0.029 | 0.252 0.028 | 0.449 0.027 | |
| K-Means | 0.083 0.017 | 0.156 0.022 | 0.175 0.021 | 0.165 0.022 | 0.252 0.028 | 0.387 0.026 | |
| K-Linsep | 0.122 0.022 | 0.171 0.023 | 0.305 0.029 | 0.165 0.021 | 0.252 0.028 | 0.452 0.028 | |
| Qwen | Avg | 0.146 0.023 | 0.172 0.023 | 0.230 0.026 | 0.336 0.028 | 0.263 0.017 | 0.435 0.026 |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| CLIP | Avg | 0.479 0.035 | 0.484 0.036 | 0.566 0.037 | 0.983 0.010 | – | 0.989 0.008 |
| Linsep | 0.475 0.035 | 0.527 0.040 | 0.805 0.030 | 0.994 0.005 | – | 0.994 0.006 | |
| K-Means | 0.447 0.038 | 0.578 0.041 | 0.670 0.042 | 0.957 0.016 | – | 0.993 0.006 | |
| K-Linsep | 0.448 0.039 | 0.613 0.041 | 0.779 0.035 | 0.980 0.011 | – | 0.976 0.012 | |
| Llama | Avg | 0.483 0.037 | 0.532 0.038 | 0.774 0.028 | 0.960 0.015 | 0.987 0.008 | 1.000 0.000 |
| Model | Concept Type | Concept Detection Methods | |||||
| RandTok | LastTok | MeanTok | CLSTok | Prompt | SuperAct | ||
| Llama | Avg | 0.679 0.047 | 0.823 0.045 | 0.712 0.050 | 0.860 0.038 | 0.789 0.047 | 0.817 0.044 |
| Linsep | 0.708 0.054 | 0.905 0.033 | 0.859 0.039 | 0.892 0.035 | 0.789 0.047 | 0.918 0.029 | |
| K-Means | 0.678 0.050 | 0.820 0.044 | 0.791 0.045 | 0.808 0.043 | 0.789 0.047 | 0.754 0.050 | |
| K-Linsep | 0.648 0.053 | 0.870 0.037 | 0.815 0.044 | 0.810 0.046 | 0.789 0.047 | 0.867 0.037 | |
| Qwen | Avg | 0.682 0.050 | 0.790 0.047 | 0.761 0.050 | 0.889 0.035 | 0.811 0.043 | 0.792 0.044 |
| Concept Detection | ||||
| Dataset | Runner-up Baseline | Runner-up Score | SuperAct (Tuned) | SuperAct ( ) |
| CLEVR | Prompt | 0.987 | 0.997 | 1.000 |
| COCO | Prompt | 0.686 | 0.827 | 0.803 |
| OpenSurfaces | MeanTok | 0.514 | 0.567 | 0.547 |
| Pascal | MeanTok | 0.709 | 0.823 | 0.777 |
| PERSUADE | CLSTok | 0.551 | 0.639 | 0.516 |
| Attribution Method | Concept Type | CLIP | Llama | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.75 0.01 | 0.74 0.01 | 0.90 0.00 | 0.87 0.00 |
| Linsep | 0.76 0.01 | 0.75 0.01 | 0.97 0.00 | 0.94 0.00 | |
| K-Means | 0.74 0.01 | 0.74 0.01 | 0.52 0.00 | 0.52 0.00 | |
| K-Linsep | 0.73 0.01 | 0.71 0.01 | 0.51 0.00 | 0.51 0.00 | |
| LIME | Avg | 0.49 0.02 | 0.55 0.04 | 0.76 0.03 | 0.81 0.02 |
| Attribution Method | Concept Type | CLIP | Llama | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.56 0.00 | 0.51 0.00 | 0.59 0.00 | 0.51 0.00 |
| Linsep | 0.63 0.00 | 0.53 0.00 | 0.65 0.00 | 0.50 0.00 | |
| K-Means | 0.47 0.00 | 0.48 0.00 | 0.42 0.00 | 0.42 0.00 | |
| K-Linsep | 0.47 0.00 | 0.49 0.00 | 0.41 0.00 | 0.40 0.00 | |
| LIME | Avg | 0.32 0.01 | 0.38 0.02 | 0.47 0.01 | 0.51 0.02 |
| Attribution Method | Concept Type | CLIP | Llama | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.61 0.00 | 0.55 0.00 | 0.51 0.00 | 0.44 0.00 |
| Linsep | 0.67 0.00 | 0.54 0.00 | 0.59 0.00 | 0.44 0.00 | |
| K-Means | 0.50 0.00 | 0.50 0.00 | 0.39 0.00 | 0.39 0.00 | |
| K-Linsep | 0.50 0.00 | 0.51 0.00 | 0.40 0.00 | 0.38 0.00 | |
| LIME | Avg | 0.50 0.02 | 0.52 0.02 | 0.69 0.02 | 0.71 0.03 |
| Attribution Method | Concept Type | CLIP | Llama | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.32 0.00 | 0.30 0.00 | 0.26 0.00 | 0.24 0.00 |
| Linsep | 0.35 0.00 | 0.29 0.00 | 0.31 0.00 | 0.25 0.00 | |
| K-Means | 0.27 0.00 | 0.26 0.00 | 0.19 0.00 | 0.19 0.00 | |
| K-Linsep | 0.25 0.00 | 0.24 0.00 | 0.18 0.00 | 0.20 0.00 | |
| LIME | Avg | 0.42 0.03 | 0.50 0.01 | 0.55 0.03 | 0.62 0.01 |
| Attribution Method | Concept Type | Llama | Qwen | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.25 0.00 | 0.19 0.00 | 0.10 0.00 | 0.07 0.00 |
| Linsep | 0.35 0.01 | 0.21 0.00 | 0.27 0.00 | 0.16 0.00 | |
| K-Means | 0.10 0.00 | 0.15 0.00 | 0.08 0.00 | 0.08 0.00 | |
| K-Linsep | 0.10 0.00 | 0.17 0.00 | 0.07 0.00 | 0.08 0.00 | |
| LIME | Avg | 0.28 0.05 | 0.33 0.05 | 0.18 0.03 | 0.22 0.04 |
| Attribution Method | Concept Type | Llama | Qwen | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.29 0.00 | 0.25 0.00 | 0.33 0.00 | 0.30 0.00 |
| Linsep | 0.44 0.00 | 0.36 0.00 | 0.42 0.00 | 0.33 0.00 | |
| K-Means | 0.31 0.00 | 0.31 0.00 | 0.30 0.00 | 0.30 0.00 | |
| K-Linsep | 0.30 0.00 | 0.31 0.00 | 0.29 0.00 | 0.29 0.00 | |
| LIME | Avg | 0.46 0.08 | 0.48 0.09 | 0.41 0.10 | 0.44 0.10 |
| Attribution Method | Concept Type | Llama | Qwen | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.21 0.01 | 0.15 0.00 | 0.46 0.01 | 0.39 0.01 |
| Linsep | 0.52 0.01 | 0.42 0.01 | 0.48 0.01 | 0.40 0.01 | |
| K-Means | 0.19 0.01 | 0.17 0.00 | 0.42 0.01 | 0.42 0.01 | |
| K-Linsep | 0.43 0.01 | 0.42 0.01 | 0.42 0.01 | 0.41 0.01 | |
| LIME | Avg | 0.20 0.03 | 0.25 0.01 | 0.27 0.01 | 0.31 0.02 |
| Attribution Method | Concept Type | Llama | Qwen | ||
| Concept | SuperActs | Concept | SuperActs | ||
| Direct Alignment | Avg | 0.72 0.01 | 0.65 0.01 | 0.66 0.01 | 0.62 0.01 |
| Linsep | 0.83 0.01 | 0.72 0.01 | 0.85 0.01 | 0.74 0.01 | |
| K-Means | 0.68 0.01 | 0.68 0.01 | 0.61 0.01 | 0.61 0.01 | |
| K-Linsep | 0.74 0.01 | 0.69 0.01 | 0.76 0.01 | 0.73 0.01 | |
| LIME | Avg | 0.71 0.02 | 0.78 0.01 | 0.63 0.02 | 0.67 0.03 |
| Attribution Method | Dataset | Insertion Score | Deletion Score | ||
| ( is better) | ( is better) | ||||
| Concept | SuperActs | Concept | SuperActs | ||
| LIME | COCO | 0.397 0.008 | 0.438 0.009 | 0.007 0.001 | 0.005 0.001 |
| Pascal | 0.379 0.006 | 0.405 0.005 | 0.007 0.001 | 0.006 0.001 | |
| OpenSurfaces | 0.197 0.003 | 0.213 0.004 | 0.013 0.003 | 0.010 0.002 | |
| CLEVR | 0.521 0.010 | 0.556 0.012 | 0.005 0.001 | 0.004 0.001 | |
Transformer Interpretability
Transformer interpretability studies how attention, feed-forward layers, and hidden representations support model predictions and behavior.
See all 355 papers