All classifiers, including state-of-the-art vision models, possess invariants, partially rooted in the geometry of their linear mappings. These invariants, which reside in the null-space of the classifier, induce equivalent sets of inputs that map to identical outputs. The semantic content of these invariants remains vague, as existing approaches struggle to provide human-interpretable information. To address this gap, we present Semantic Interpretation of the Null-space Geometry (SING), a method that constructs equivalent images, with respect to the network, and assigns semantic interpretations to the available variations. We use a mapping from network features to multi-modal vision language models. This allows us to obtain natural language descriptions and visual examples of the induced semantic shifts. SING can be applied to a single image, uncovering local invariants, or to sets of images, allowing a breadth of statistical analysis at the class and model levels. For example, our method reveals that ResNet50 leaks relevant semantic attributes to the null space, whereas DinoViT, a ViT pretrained with self-supervised DINO, is superior in maintaining class semantics across the invariant space.
Figures & tables
Figure 2: Method Overview. The approach consists of: (a) decomposing the final linear weights to obtain principal and null projectors; (b) training a translator that maps features from the network embedding space to the CLIP image space; (c) creating an equivalent pair to the feature we want to examine. (d) translate the set into CLIP image embedding space, and apply our metrics and visualizations.
Figure 3: Model-level comparison (1,000 classes). (a) Attribute Score (AS) quantifies class-dependent semantic leakage into the null space; Image Score (IS) quantifies tolerance to class-independent (non–class-dependent) semantic variation within the invariant subspace. Desirably, AS is low and IS is high (relative to AS). In our results, DinoViT performs best in this regard. (b) We summarize the trade-off with the IS/AS ratio (higher is better), DinoViT has the highest ratio and ResNext101 the lowest.
Figure 4: Class Comparison. DinoViT consistently preserves low semantic leakage across classes, whereas ResNet50 exhibits a pronounced imbalance, with certain classes, such as Porcupine and Sports-Car, leaking substantially more semantic information into the null space.
Figure 5: Open-vocabulary concept analysis. For DinoViT, we sample ∼1300 images per class and compute the CLIP angle (degrees; lower is more similar) to a set of concepts for (a) “Arabian Camel” class and (b) “Jellyfish” class. Blue dots denote original features; red dots denote null-removed (equivalent) features. Green arrows connect each pair and represent the Attribute Score after null removal. Longer arrows indicate larger ∣AS∣ (greater class-dependent semantic leakage); shorter arrows indicate minimal leakage.
Figure 6: Null-space semantic steering (ResNet50). From each original image (left), we add a small perturbation aligned with the indicated prompt (column headers) but constrained to the classifier head’s null space (projected-gradient direction). Although only the invariant component is modified, the feature’s semantics shift toward the target concepts, illustrating how null-space directions can alter meaning without changing the discriminative subspace.
ResNet50
EfficientNet
BiTresnet
DinoViT
ResNext101
∣AS∣ towards target
12.04 ± 0.25
12.38 ± 0.52
9.19 ± 0.31
5.0 ± 0.59
11.15 ± 0.53
Table 1: Text-gradient null perturbations. For a fair comparison, each model is perturbed by a fixed null-space step calibrated to IS=40∘ . We report ∣AS∣ toward the target prompt (mean ± standard deviation; lower is better). DinoViT attains the lowest value (marked in bold), indicating the greatest resistance to directed null-space manipulation, whereas ResNext101 remains comparatively susceptible.
Figure 7: Training losses for the different translator objectives. Minimizing the MSE loss also improves cosine similarity, whereas cosine-only training leaves the MSE substantially higher.
Architecture
Mean cosine similarity
Validation MSE
3-layer MLP
0.9049
0.082246
Residual MLP
0.9045
0.082366
4-layer MLP
0.9023
0.084346
Linear
0.8946
0.091355
Table 2: Validation results for different translator architectures on a 2 000-image validation subset from 16 classes. None of the non-linear architectures shows a significant advantage over the linear translator.
Figure 8: Cosine similarity distribution of different architectures between the translated and the original CLIP features, over 2k ImageNet features from 16 classes. All the histograms are leaned towards high correlation
Figure 9: Distribution of logit changes Δℓ(f,δ) for null-space, random, and principal perturbations. Null-space perturbations leave logits almost unchanged, whereas principal perturbations induce large logit shifts.
Figure 10: Example images with different attribute scores (AS) and image scores (IS), illustrating the relationship between angular distance and perceived semantic change. Small angles correspond to nearly identical images, while larger angles reflect more significant semantic changes.
Figure 11: UnCLIP generations from a single classifier feature translated by different translator architectures. Despite small quantitative differences in cosine similarity, the resulting visualizations are qualitatively consistent.
Model
ImageNet Top-1 Acc (%)
VGG-16 [ 48 ]
71.6
VGG-19 [ 48 ]
72.4
DenseNet-121 [ 24 ]
74.4
ResNet50 [ 21 ]
76.1
DinoViT [ 9 ]
84.0
EfficientNet-B0 (NS) [ 52 ]
78.7
Table 3: List of models from CNNs to ViTs that we used as our test subjects.
Figure 12: Additional model-level validations used in the camera-ready update.
Figure 13: UnCLIP generations of a single feature under three perturbation types (null, random, principal) across four random seeds. Null-space perturbations preserve the global class semantics, while random and principal perturbations produce more noticeable semantic changes.
Figure 14: Per-class distribution of null-space semantic angle changes across three architectures. Each violin corresponds to a single class; narrow distributions around zero indicate classes largely invariant to null-space perturbations. The consistent pattern across architectures with different inductive biases confirms the generality of our observations.
Figure 15: Open-vocabulary analysis for the “Arabian Camel” class in DinoViT. We show a larger set of prompts and their CLIP similarities to the class, illustrating the semantic neighborhood used in our analysis.
Figure 16: Open-vocabulary analysis for the “Jellyfish” class in DinoViT. The graph highlights related concepts and their CLIP similarities, showing that the concepts discussed in the main paper are part of a consistent semantic cluster.
A popular route to interpretable zero-shot classification asks a large language model (LLM) to describe each class name and prompts CLIP with the resulting descriptors. We show that these descriptors carry little visual evidence of their own: removing the class name from the prompt collapses ImageNet accuracy from 59.5% to 15.5%. The diagnosis is that the descriptors are conditioned on the label rather than on the images, so they describe the concept in general and mislead exactly when the data shifts; an LLM insists that strawberries are red, but every strawberry in ImageNet-Sketch is a colorless line drawing. We therefore select attributes from the target image collection instead: we score a large attribute pool against the images in CLIP's joint embedding space and keep the top-scoring attributes per class. Selected this way, class-name-free attribute prompts reach 23.8% on ImageNet (against 15.5% for LLM descriptors), the gain holds on four shifted ImageNet variants, and reselecting from the LLM's own pool isolates the selection mechanism as the cause. With one image per class, the selected attributes outperform the prompt-tuning method CoOp by 3 points while fitting in under a minute instead of 14 hours, with no learned soft prompt to obscure the decision. Because the attribute set is chosen by the data, it doubles as a readable summary of a dataset, which we use to describe distribution shift in words. Our code and results are available on our project page: https://ggare-cmu.github.io/AttributeSelect/
As Vision-Language Models are increasingly deployed in safety-critical applications, the trustworthiness of their explanations becomes crucial. Explainable AI (XAI) methods for Vision-Language Models often suffer from semantic hallucination, where attribution maps highlight prominent image regions even when prompted with incorrect text descriptions (e.g., highlighting a dog when prompted ``cat''). Although this problem is widespread, a formal mathematical analysis of XAI methods and CLIP embeddings is largely missing in the literature. We demonstrate that this phenomenon is not specific to a single architecture but is a fundamental consequence of Linear Semantic Leakage in high-dimensional embedding spaces. We propose a unified theoretical framework, Linear Semantic Attribution (LSA), which generalizes across discriminative methods. We introduce OSP, a geometric intervention that utilizes the residual property of OMP to disentangle unique semantic signals from shared concepts. We prove theoretically and demonstrate empirically that OSP minimizes hallucination by orthogonalizing the query vector against distractor concepts, rendering the attribution model blind to shared features while preserving fidelity for correct prompts. Our code is available at: https://github.com/emirhanbilgic/Orthogonal-Semantic-Projection
Emirhan Bilgiç, Baptiste Caramiaux, Zhi Yan +1
U2IS, ENSTA, Institut Polytechnique de Paris, Palaiseau · ISIR, Université Sorbonne, Pierre et Marie Curie, Paris · AMIAD, Pôle Recherche, Palaiseau
As vision-language models are deployed at scale, understanding their internal mechanisms becomes increasingly critical. Existing interpretability methods predominantly rely on activations, making them dataset-dependent, vulnerable to data bias, and often restricted to coarse head-level explanations. We introduce SITH (Semantic Inspection of Transformer Heads), a fully data-free, training-free framework that directly analyzes CLIP's vision transformer in weight space. For each attention head, we decompose its value-output matrix into singular vectors and interpret each one via COMP (Coherent Orthogonal Matching Pursuit), a new algorithm that explains them as sparse, semantically coherent combinations of human-interpretable concepts. We show that SITH yields coherent, faithful intra-head explanations, validated through reconstruction fidelity and interpretability experiments. This allows us to use SITH for precise, interpretable weight-space model edits that amplify or suppress specific concepts, improving downstream performance without retraining. Furthermore, we use SITH to study model adaptation, showing how fine-tuning primarily reweights a stable semantic basis rather than learning entirely new features.
Francesco Gentile, Nicola Dall'Asen, Francesco Tonini +3
University of Trento · University of Pisa · Fondazione Bruno Kessler