cs.CVNov 16, 2026

Certified, Not Explained: Exact Token Erasure in Gated Vision State Space Models

Authors: Sadia Parveen, Tehseen Zia

Organizations: Department of Computer Science, COMSATS University Islamabad, Pakistan

Abstract

Vision models built on Mamba are often explained through their selective step size Delta, on the assumption that a token the scan does not admit cannot influence the prediction. We test that in a concept bottleneck model for lung-nodule malignancy built on a single gated, bidirectional selective state-space block. The answer becomes an interpretability framework: a guarantee, a procedure that checks it, and a pre-registered audit of what it explains. For a fixed gate pattern, masking the input of the causal convolution of the block and giving closed tokens zero weight at readout makes the concept and task logits exactly invariant to the closed tokens. Gating Delta is neither necessary nor sufficient. On LIDC-IDRI, a path-by-path knockout over 15 cross-validated models agrees with this in every run: the largest logit change is exactly 0 whenever the conditions hold and between 4.1 and 6.7 when they do not. The gated model reaches a pooled out-of-fold AUROC of 0.8425 +/- 0.0085, level with an ungated Mamba bottleneck (0.8428). The guarantee belongs to the masked model, not to the gate as an explanation. Post-hoc rankers, which inherit the guarantee on this model, order tokens better than the gate. Gate maps from a model trained on permuted labels are as close to the real maps as two real-label seeds are to each other. Under retraining, the tokens of the gate carry more task information than random tokens and, on shifted crops, more than central ones, but no more than the tokens chosen by hidden attention. We therefore read the gate as a certified support set produced by mask supervision. Splitting the Oracle Impurity Score into own-concept and cross-concept parts reveals differences the aggregate score misses. Partitioning the readout reduced leakage slightly under an earlier recipe, and the final model sits at, rather than below, the level implied by correlations among the true concepts.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

May 20, 2026

Ablate-to-Validate: Are Vision-Language Models Really Using Continuous Thought Tokens?

Vision-language models (VLMs) are increasingly augmented with continuous or latent non-textual tokens intended to support "visual thinking." Despite improved task accuracy, this alone does not show that models actually use these tokens for reasoning -- gains may arise from confounds such as added context length, special-token anchoring, or training-time regularization. We formalize a diagnostic principle, Ablate-to-Validate, for testing whether latent-token content is genuinely utilized, and instantiate it as the Token Replacement Test (TRT), a standardized suite of content-replacement ablations. TRT holds the prompt, image, token budget, and decoding fixed while replacing intermediate tokens with zero, random, first-repeat, or oracle alternatives, isolating whether performance depends on token content or merely on token presence. As a controlled testbed, we study relative depth reasoning with LLaVA-13B and Qwen2.5-VL-3B, training models to predict and consume continuous or discrete depth spans across multiple frozen encoders (SigLIP2, CLIP, DINOv2) and token budgets. We additionally apply TRT to three off-the-shelf visual-thinking systems (Mirage, Mull-Tokens, CoVT) on BLINK, VSP, and CV-Bench. Across all settings, accuracy gains are a misleading proxy for latent-token reasoning: VLMs retain most improvement even when token content is corrupted or replaced, revealing a persistent gap between having a latent channel and using it as an information bottleneck. We recommend TRT as a standard diagnostic alongside accuracy for any method introducing continuous thought tokens.
Feb 22, 2026

DD-CAM: Minimal Sufficient Explanations for Vision Models Using Delta Debugging

We introduce a gradient-free framework for identifying minimal, sufficient, and decision-preserving explanations in vision models by isolating the smallest subset of representational units whose joint activation preserves predictions. Unlike existing approaches that aggregate all units, often leading to cluttered saliency maps, our approach, DD-CAM, identifies a 1-minimal subset whose joint activation suffices to preserve the prediction (i.e., removing any unit from the subset alters the prediction). To efficiently isolate minimal sufficient subsets, we adapt delta debugging, a systematic reduction strategy from software debugging, and configure its search strategy based on unit interactions in the classifier head: testing individual units for models with non-interacting units and testing unit combinations for models in which unit interactions exist. We then generate minimal, prediction-preserving saliency maps that highlight only the most essential features. Our experimental evaluation demonstrates that our approach can produce more faithful explanations and achieve higher localization accuracy than the state-of-the-art CAM-based approaches.
Aug 5, 2026

Not All Redundant Tokens Are Alike: Analyzing Visual Token Pruning through Token Roles

Vision-language models (VLMs) process an image as a sequence of visual tokens, which creates a substantial computational bottleneck during inference. Recent visual token pruning methods address this issue by removing seemingly redundant tokens, yet it remains unclear how these pruning decisions relate to the functional roles of visual tokens. In this work, we analyze visual token pruning through the lens of token roles identified by EmbedLens. We first show that representative pruning methods exhibit distinct token-role biases, but these biases do not directly correlate with downstream performance. To better understand this behavior, we refine the token-role assignment procedure and evaluate role-protected pruning variants. Our results show that preserving non-alive tokens can sometimes maintain or improve performance, suggesting that tokens with weak direct semantic alignment may still affect model behavior under pruning. Our code is publicly available at https://github.com/jaykim9870/Not_All_Redundant_Tokens_Are_Alike.