cs.CVNov 11, 2026

A General Concept-based Decomposition for Vision-Language Embeddings

Authors: Simone Alberto Peirone, Ortal Senouf, Francesca Pistilli, Giuseppe Averta, Pascal Frossard

Organizations: Politecnico di Torino Torino, Italy · EPFL Lausanne, Switzerland

Abstract

Vision-Language Models are designed to capture the compositional structure of the world through multi-modal alignment; however, it remains unclear whether their representations truly align with the way a human would naturally describe the same visual input. Prior works attempt to decompose visual embeddings into their concept-level contributions, but their instance-level optimization is inherently local and does not capture the model's global semantic structure, which is essential for compositional generalization across different inputs, tasks, and domains. We introduce GRACE, a method that learns this concept-based structure at the feature space level, producing semantic and general concept decompositions. To achieve this, we propose a novel training objective that preserves the geometric relations of different samples in the concept space, enabling a better understanding of how concepts compose and relate to each other. We validate GRACE on several image and video benchmarks, showing that its concept decomposition is aligned with human perception and achieves faithful decompositions across different tasks and domains. With minimal overhead, GRACE makes existing Vision-Language Models more interpretable, while preserving most of their original performance.

Figures & tables

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Conceptualizing Embeddings: Sparse Disentanglement for Vision-Language Models

    May 21, 2026Piotr Kubaty, Patryk Marszałek, Łukasz Struski +3Disentangled Representation LearningMultimodal Disentangled Representation Learning

  2. H-GRPO: Permutation-Invariant Reinforcement Learning for Improving Spatial Grounded Reasoning

    Jun 29, 2026Eric Peh, Debaditya Roy, Chen Li +1RL for VLMsVisual Reasoning

  3. Decompose, Look, and Reason: Reinforced Latent Reasoning for VLMs

    Apr 8, 2026Mengdan Zhu, Senhao Cheng, Liang ZhaoVision-Language GroundingVLM Interpretability