Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models

Laura Yao; Shixing Yu; Jie Peng; Promit Ghosal; Kyra Gan; Tianlong Chen

cs.AIOct 9, 2026

Cross-Modal Latent Influence: Patch-Level Training Sample Attribution for Vision-Language Models

Authors: Laura Yao, Shixing Yu, Jie Peng, Promit Ghosal, Kyra Gan, Tianlong Chen

Organizations: UNITES Lab, University of North Carolina at Chapel Hill · Electrical and Computer Engineering, Cornell Tech · Foci Labs · Department of Statistics, University of Chicago ·

Abstract

Explaining a vision--language model (VLM) prediction requires tracing training-example influence to the multimodal evidence that mediates it, yet existing attribution methods largely operate only at the sample level. Recent latent-mediated influence methods provide feature-level attribution in language models, but lack the multimodal and spatial structure needed to ground influence in VLMs. We introduce MOSAIC (Multimodal Orthogonalized Sparse Autoencoders for Influence Characterization). It learns a shared sparse image--text latent space whose concepts are 1) sparse within each modality, 2) supported on contiguous image regions, 3) aligned to the phrases that describe them, and 4) approximately decorrelated. It then applies EK-FAC to decompose training-example influence through these concepts and localize their contributions to specific text phrases and image regions. We evaluate MOSAIC on Qwen and Gemma across captioning, chest X-ray understanding, and visual question answering. The learned features are more localized and coherent without degrading task behavior, and ablating influence-ranked latents changes model outputs more than ablating latents ranked by activation or frequency. We find that visual evidence can substantially reorganize training-data attribution while preserving more of the shared latent structure. MOSAIC thus connects where model behavior comes from with what multimodal evidence carries that influence.

Figures & tables

Figure 1: (a) Input attribution localizes evidence but lacks training-data specific attribution. (b) Multimodal SAEs expose internal concepts without tracing their training origins. (c) Latent influence traces training examples through features but lacks multimodal grounding. (d) MOSAIC connects influential training examples to sparse multimodal concepts and localized evidence.

Figure 2: Overview of MOSAIC. (a) Intermediate VLM image and text representations. (b) Shared TopK SAE with a common sparse dictionary. (c) Spatially structured sparsity for region-coherent visual features. (d) Contrastive phrase–region alignment using frozen VLM attention. (e) Latent decorrelation for a better-separated attribution space. (f) EK-FAC influence tracing predictions to training examples, latents, phrases, and image regions.

Dataset	Model	Metric	Base	FT	+SAE
VizWiz	Qwen	CIDEr	0.0738	0.7569	0.7471
VizWiz	Gemma	CIDEr	0.0686	0.5799	0.5793
CheXpert	Qwen	Acc.	0.3320	0.4290	0.4295
CheXpert	Gemma	Acc.	0.0040	0.4530	0.4560
GQA	Qwen	Acc.	0.5943	0.6513	0.6361
GQA	Gemma	Acc.	0.4019	0.5914	0.5867

Table 1: Task performance before and after SAE insertion. CIDEr for caption similarity to human references; accuracy is used for chest X-ray classification and VQA.

Figure 3: Comparison on CheXpert. Highlighted regions and words indicate latent activations. (a) Base SAE yields diffuse support and weaker radiological alignment. (b) Ours improves localization and clinically relevant alignment.

Figure 4: Latent breakdown for a VizWiz example with Qwen. The top two latents following Fig. 9 capture medicine- and bottle-related evidence; highlighted regions and words indicate representative activations.

Table 6

Figure 5: VizWiz necessity–sufficiency. Qwen [top], Gemma [bottom]; (a) necessity, (b) sufficiency.

Figure 6: CheXpert necessity–sufficiency. Qwen [top], Gemma [bottom]; (a) necessity, (b) sufficiency.

Figure 7: GQA necessity–sufficiency. Qwen [top], Gemma [bottom]; (a) necessity, (b) sufficiency.

Figure 8: Effect of removing visual evidence on VizWiz influence rankings with Qwen. (a) Latent rankings remain relatively stable. (b) Training-example rankings change substantially.

Figure 9: Training-example breakdown for a VizWiz example with Qwen. Training examples are ranked by influence; we show the top contributing latents, with peak image activations and caption words. Additional results are in the appendix.

Figure 10: GQA influence decompositions. Incorrect prediction [left]. Correct prediction [right].

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Setting	CIDEr $\uparrow$	METEOR $\uparrow$	B-1 $\uparrow$	B-2 $\uparrow$	B-3 $\uparrow$	B-4 $\uparrow$	R-L $\uparrow$
Qwen	Finetuned, no SAE	0.7569	0.2641	0.7076	0.5171	0.3736	0.2660	0.4767
Qwen	+ SAE	0.7471	0.2612	0.7099	0.5180	0.3737	0.2659	0.4758
Qwen	Base (prompted)	0.0738	0.2385	0.3452	0.2070	0.1205	0.0706	0.3079
Qwen	Base (zero-shot)	0.0001	0.0026	0.0001	0.0000	0.0000	0.0000	0.0108
Gemma	Finetuned, no SAE	0.5799	0.2405	0.6608	0.4558	0.3068	0.2030	0.4419
Gemma	+ SAE	0.5793	0.2389	0.6622	0.4557	0.3057	0.2016	0.4416

Appendix

Table 4: Full VizWiz captioning results. We report CIDEr, METEOR, BLEU-1–4, and ROUGE-L. “Prompted” uses the task-specific captioning prompt, while “zero-shot” denotes evaluation without task-specific prompting.

Dataset	Metric	Text	Pre-latent	SAE latent
VizWiz	Near-orthogonal ( $\|\rho\|<0.1$ ) $\uparrow$	$7.6\%$	$20.8\%$	$\mathbf{99.5\%}$
	Mean $\|\mathrm{offdiag}(G)\|$ $\downarrow$	$0.798$	$0.280$	$\mathbf{0.0020}$
	Mean $\mathrm{offdiag}(G)^{2}$ $\downarrow$	$0.751$	$0.115$	$\mathbf{0.0008}$
	Stable rank (raw / norm.)	$1.16/0.0053$	$4.01/0.0020$	$9.93/0.0012$
CheXpert	Near-orthogonal ( $\|\rho\|<0.1$ ) $\uparrow$	$24.9\%$	$25.4\%$	$\mathbf{99.2\%}$
	Mean $\|\mathrm{offdiag}(G)\|$ $\downarrow$	$0.510$	$0.239$	$\mathbf{0.0036}$

Appendix

Table 5: Orthogonality of Qwen representations before and after SAE encoding at layer

\ell=15

(

d_{\mathrm{latent}}=8192

k=128

Setting	SAE Layer	GQA Acc.
Base model	–	0.4019
Fine-tuned model	–	0.5914
Fine-tuned + SAE	19	0.5867
Fine-tuned + SAE	12	0.3315

Appendix

Table 6: Sensitivity of Gemma on GQA to SAE placement.

Dataset / Model	Pool J.	Latent $\rho$	Example $\rho_{\mathrm{union}}$	Example $\rho_{\mathrm{shared}}$	Top-8
GQA / Qwen	$0.123$	$0.599$	$-0.564$	$0.260$	$0.073$
GQA / Gemma	$0.051$	$0.543$	$-0.755$	0.183	$0.000$
VizWiz / Qwen	$0.021$	$0.584$	$-0.793$	$0.279$	$0.010$
VizWiz / Gemma	$0.052$	$0.571$	$-0.774$	$-0.063$	$0.000$
CheXpert / Qwen	$0.145$	$0.737$	$-0.606$	$0.570$	$0.031$
CheXpert / Gemma	$0.026$	$0.795$	$-0.797$	$0.850$	$0.000$

Appendix

Table 7: Influence with and without visual evidence. Latent

\rho

compares absolute latent influence, example

\rho_{\mathrm{union}}

compares training examples active in either condition, and example

\rho_{\mathrm{shared}}

is restricted to candidates retrieved in both conditions.

Figure 11: Example latent comparison on VizWiz. (a) Base SAE. (b) Our approach. Our latent exhibits a clearer semantic association and more spatially coherent visual support, while the base latent activates more diffusely across the image.

Figure 12: Example latents on GQA. Representative latents learned by our approach exhibits a meaningful semantic association together with localized activation over the corresponding visual evidence.

Figure 13: Detailed latent decomposition for a VizWiz medicine example. Representative test latents capture complementary evidence including the bottle , medicine -related semantics, and the hand holding the medicine . The localized activations show that the representation separates object identity, semantic function, and relevant object–person interaction while remaining grounded in the corresponding image regions.

Figure 14: Detailed latent decomposition for a VizWiz eye-drop example. The learned latents recover the central semantics of the test image, including eyes , eye drops , and bottle , while also representing localized contextual evidence such as the wooden table on which the object appears. This example illustrates how distinct latents jointly encode both answer-relevant concepts and faithful scene context.

Figure 15: Influential training examples for a VizWiz screen query. The test example contains a screen, and the highest-influence training examples consistently exhibit closely related screen or display content. This qualitative consistency shows that influential examples reflect the salient visual semantics of the test input.

Figure 16: Influential training examples for a GQA clothing query. The test question asks what the man is wearing, with jeans as the predicted answer. The top three influential training examples are all associated with jeans , directly matching the answer semantics of the test query.

Figure 17: Influential training examples for a GQA identity query. The test question asks who is looking out the window, with man as the answer. The highlighted answers in the highest-influence training examples are likewise man , demonstrating strong agreement between the training supervision and the test prediction.

Figure 18: Influential training examples for a CheXpert cardiomegaly prediction. The correctly predicted cardiomegaly case receives support from training examples spanning multiple queried findings, including cardiomegaly and other cardiopulmonary abnormalities. This illustrates that influence is not determined solely by exact diagnostic-label matching.

Figure 19: Influential training examples for a CheXpert edema prediction. The highest-influence example is queried for consolidation but its report describes pulmonary vascular congestion, providing evidence closely related to edema. The ranking therefore captures clinically relevant cross-label correspondence rather than only exact label agreement.

Figure 20: Influential training examples for a CheXpert pleural effusion prediction. The influential set contains both a directly matching pleural-effusion example and examples associated with other thoracic findings, demonstrating a mixture of direct and cross-label support for the prediction.

Method	Alive	Balance $\uparrow$	Footprint $\downarrow$	Contig. $\downarrow$	Word conc. $\downarrow$	Img. tok./fire $\downarrow$
MOSAIC	8156	26.7	8.3%	0.548	0.219	3.30
JSAE	99	3.3	100%	1.000	0.552	192
SPARC	69	0.0	100%	1.001	0.992	192

Appendix

Table 8: Comparison with pooled multimodal SAE baselines on VizWiz with Qwen. Balance is computed over selective live latents. Footprint measures the fraction of the image patch grid activated by a latent, and contiguity is normalized against a size-matched random support. Lower footprint, contiguity, and word concentration indicate more selective features.

Forward path	Mean CE $\downarrow$	$\Delta$
Identity (no SAE)	1.848	$-0.180$
MOSAIC	2.027	0
JSAE	9.464	$+7.44$
Mean-broadcast control	10.462	$+8.43$
SPARC	12.691	$+10.66$

Appendix

Table 9: Effect of inserting each dictionary position-wise into the VLM forward path. Mean cross-entropy is evaluated over 60 VizWiz validation examples.

\Delta

is relative to MOSAIC.

VLM Interpretability

Vision-language model interpretability explains how image and language representations shape predictions, evidence use, and multimodal reasoning.

See all 268 papers

Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning
Sep 29, 2026Bowen Yuan, Danny Wang, Ruihong Qiu +2Large Vision-Language Models VLM Interpretability
The University of Queensland
Circuit Tracing in Vision-Language Models: Understanding the Internal Mechanisms of Multimodal Thinking
Feb 23, 2026Jingcheng Yang, Tianhu Xiong, Shengyi Qian +2VLM Interpretability VLM Reasoning
University of Illinois Urbana-Champaign · Independent Researcher
Where To Look? : Causal Tracing of Vision Encoders in VLM
Aug 11, 2026Naren Kumar S, Tirth Bhatt, Mayank SinghLarge Vision-Language Models VLM Interpretability
LINGO Research Group, Indian Institute of Technology Gandhinagar, India

Sep 29, 2026J/K move · Enter open · S save

Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning

Bowen Yuan, Danny Wang, Ruihong Qiu, Zijian Wang+1

The University of Queensland

Large Vision-Language Models VLM Interpretability Multimodal Model Interpretability Faithfulness in VLMs

Open paper AskPDF arXiv

Large vision-language models (LVLMs) exhibit strong reasoning capabilities, yet the visual and textual evidence supporting the generated responses remains difficult to identify. Faithful token attribution explains an LVLM's response by assigning scores that rank image and prompt tokens by how much the model relies on them, such that removing higher-ranked tokens causes the likelihood of the generated response to drop more rapidly. However, existing token-attribution methods have been developed mainly for text-based language models, and our empirical study reveals two challenges when complex multimodal sources are involved. First, the joint image-text attribution can underrepresent visual evidence relative to text, obscuring the image regions supporting the response. Second, visual evidence may influence the generated response through multiple intermediate reasoning paths, while existing methods trace only a limited subset of these paths, causing important visual contributions to be underestimated. Motivated by these insights, we introduce VTrace, a multimodal token-attribution framework that traces input contributions through intermediate reasoning and calibrates attribution scores across modalities. VTrace constructs pairwise attributions that highlight token-specific contributions and aggregates all forward attribution paths in closed form to account for both direct and indirect contributions. Cross-modal calibration then rescales image and text attribution scores using modality contributions estimated from response-likelihood changes, enabling a unified ranking of input tokens. Evaluations against seven baselines across six visual reasoning benchmarks demonstrate the superior attribution faithfulness. Project page: https://vtrace-attribution.github.io/.

Abstract

VLM Interpretability

Explore similar work

Tracing the Evidence: Faithful Token Attribution Through Vision-Language Reasoning

Circuit Tracing in Vision-Language Models: Understanding the Internal Mechanisms of Multimodal Thinking

Where To Look? : Causal Tracing of Vision Encoders in VLM