Compositional Reasoning in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

37 papers13 papers in the last four weeks, up 225% on the four weeks before. 0.1% of all new papers.

Factorized MLLM Embeddings for Composed Image Retrieval
Nov 3, 2026WonJun Moon, Seongchan Kim, Dahun Kim +1Composed Image Retrieval Multimodal Embedding
KAIST AI · Google DeepMind
Perturb, Repair, Verify: Self-Play Vision-Language Verifiers for Compositional Understanding
Nov 3, 2026Zijun Huang, Yingjun Du, Cees G. M. SnoekCompositional Reasoning in VLMs Visual Evidence Sensitivity in VLMs
University of Southern California · VisLab, University of Amsterdam
SurgEye: Instruction Tuning-Free Perception-Reasoning Framework for Surgical Scene Understanding
Nov 2, 2026Qing Xu, Xiechen Shen, Xianjian He +9Surgical Video Understanding Medical VLMs
School of Computer Science, University of Nottingham Ningbo China · University of Nottingham · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University
ConstraintMatch: Compositional Image-Text Matching through Constrained Regional Evidence Selection
Oct 31, 2026Yuhong Zhou, Yubo Gao, Xiangbo ShuCompositional Reasoning in VLMs Cross-Modal Retrieval
Nanjing University of Science and Technology · The Hong Kong University of Science and Technology (Guangzhou)
CompoSET: A Controlled Single-Edit Benchmark for Vision-Language Compositionality
Oct 30, 2026Laure Delisle, Pietro PeronaCompositional Reasoning in VLMs VLM Evaluation
California Institute of Technology
Drone-PUR: From Visual Clue Discovery to Understanding and Reasoning in UAV Scenes
Oct 29, 2026Jiahe Wu, Bing Cao, Xingyue Guo +5Visual Spatial Reasoning VLM Evaluation
School of Artificial Intelligence, Tianjin University, Tianjin 300072, China · College of Computer Science, VCIP, Nankai University, Tianjin 300350, China · School of Computer Science and Technology, Tianjin University, Tianjin 300072, China +3
What do MLLMs know about 'Waldo'? Evaluating Visual Reasoning under Multi-Instance Multi-Object Challenges
Oct 28, 2026Saumya Mundra, Ajoy Mondal, Vineeth N Balasubramanian +1Visual Counting MLLM Evaluation
IIIT Hyderabad · Microsoft Research India · IIT Hyderabad
RuleForge: Inferring and Instantiating Latent Visual Rules as Executable Programs
Oct 28, 2026Weifeng Xu, Yaxiong Wang, Zhedong ZhengVisual Reasoning Visual Program Synthesis
Institute of Artificial Intelligence and Brain Sciences, University of Macau Macau, China · Faculty of Information Science and Computing, University of Macau Macau, China · Hefei University of Technology Hefei, China
MIS-VQA: An Annotation-Driven Surgical Vision-Language Dataset for Complex Surgical Understanding
Oct 25, 2026Mohamed Hamdy, Iheb Zouari, Fatmaelzahraa Ahmed +6Medical Visual Question Answering Surgical Video Understanding
Department of Computer Science and Engineering, Qatar University, Doha, Qatar · Hamad Medical Corporation, Doha, Qatar
All the Facts, Still the Wrong Answer: Integration Gaps in Vision-Language Models
Oct 24, 2026Yuyang Dai, Yuxia Wang, Zhuohan XieVLM Evaluation Compositional Reasoning in VLMs
Florida State University · INSAIT, Sofia University “St. Kliment Ohridski” · MBZUAI
Cross-Scale Sparse Attention for Referring Segmentation in Remote Sensing
Oct 24, 2026Yuxi Li, Lu Si, Xia Liao +4Referring Image Segmentation Remote Sensing Image Segmentation
Westlake Institute for Advanced Study, Hangzhou, Zhejiang, China · School of Engineering, Westlake University, Hangzhou, Zhejiang, China · Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, China +2
Compositionality with Primitives: A Unified View from Euclidean to Hyperbolic Vision-Language Models
Oct 21, 2026Puru Vaish, Meelad Dashti, Christoph Brune +3Compositional Reasoning in VLMs Zero-Shot Learning
University of Twente, The Netherlands · Politecnico di Torino, Italy · University of Amsterdam, The Netherlands
Beyond Correct Answers: How Compositional Cues Shape Visual Judgments
Oct 20, 2026Xiaoyu Ma, Zheyuan Zhang, Kaiwen Shi +1Visual Question Answering Compositional Reasoning in VLMs
University of Notre Dame
RACE: Learning Multimodal Search through Referential Abstraction and Compositional Evidence
Oct 20, 2026Eric Tao, Minhao Sun, Liyan Liu +9Multimodal Search Agents Multimodal Grounding
Huawei · Beijing Institute of Technology
DCR-TSF: Dual-Level Compositional Reasoning with Triadic Spherical Fusion for Training-Free Zero-Shot Composed Image Retrieval
Oct 19, 2026Gwang-Ho Na, Ho-Joong Kim, Seong-Whan LeeComposed Image Retrieval Compositional Reasoning in VLMs
Dept. of Artificial Intelligence, Korea University, Seoul, Korea
Can the Parts Fool the Test? Counterfactual Pairing Cycles for Relational OOD
Oct 16, 2026Yibo Zhou, Yirong XiangCounterfactual Evaluation of VLMs Compositional Reasoning in VLMs
Beihang University · Air Force Medical Center
Composition, Not Conversation: VLMs Lose the Scene, Not the Thread
Sep 29, 2026L. D. M. S. Sai Teja, Ufaq Khan, N. Siva Gopala Krishna +5VLM Evaluation Visual Question Answering
NIT Silchar · MBZUAI · BML Munjal University +1
From Perception to Integration: Revisiting the Internal Dynamics of Reasoning in Vision-Language Models
Sep 28, 2026Rong Yu Xu, Prayag Tiwari, Shaolei ZhangEfficient VLM Inference VLM Interpretability
Shenzhen College of International Education · Halmstad University, Sweden · Renmin University of China
Diagnosing the Sources of Compositional Failure in Vision-Language Models: A Controlled Analysis
Sep 25, 2026Mona Gandhi, Cenk Merih Olcay, Kuan-Chieh Lo +3Compositional Reasoning in VLMs VLM Reasoning
The Ohio State University · Netflix Research
CS-CLIP: Compositional Scene Graph-guided CLIP for Robust Compositional Reasoning
Sep 8, 2026SeongJun Jeong, Minjoon Jung, Woo Suk Choi +2VLM Robustness Contrastive Learning
Seoul National University
The Limits of Binding in Dual Encoders
Aug 16, 2026Kin Ian LoCompositional Reasoning in VLMs Contrastive Learning
University College London
VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use
Aug 9, 2026Juan S. SantillanaVLM Evaluation Tool Use in VLMs
Globant
Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models
Jul 25, 2026Sultan Alshehri, Zhantao Yang, Han Zhang +1Vision-Language Models VLM Reasoning
Carnegie Mellon University, Pittsburgh, PA, USA.
H-GRPO: Permutation-Invariant Reinforcement Learning for Improving Spatial Grounded Reasoning
Jun 29, 2026Eric Peh, Debaditya Roy, Chen Li +1RL for VLMs Visual Reasoning
Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, India +1
Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray
Jun 24, 2026Yonathan Michael, Mohamad Alansari, Natnael Takele +2Multimodal Grounding Compositional Reasoning in VLMs
Computer Science Department Khalifa University · Center of Cyber-physical Systems (C2PS) Khalifa University
Self-Questioning Vision-Language Models: Reinforcement Learning for Compositional Visual Reasoning
Jun 14, 2026Saraswathy AmjithVisual Question Answering Visual Reasoning
Massachusetts Institute of Technology
Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality
Jun 11, 2026Wei Li, Zhen Huang, Xinmei TianCross-Modal Learning Vision-Language Models
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · Independent Researcher
HyperVis: Continuous Latent Visual Relational Graphs on the Lorentz Hyperboloid for Compositional Reasoning
Jun 4, 2026Moshiur Farazi, Sameera Ramasinghe, Mahbub Ahmed Turza +1Hyperbolic Representation Learning Visual Reasoning
Data Science and AI, University of Doha for Science and Technology, Qatar · Pluralis Research, Australia · Department of Electrical and Computer Engineering, North South University, Bangladesh
ReGuLaR: Relation-Grounded Latent Reasoning for Large Vision-Language Models
May 28, 2026Zihu Wang, Karthik Somayaji N. S, Hejia Geng +1Latent Visual Reasoning Large Vision-Language Models
University of California, Santa Barbara
Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding
Apr 14, 2026Eun Woo Im, Dhruv Madhwal, Vivek GuptaCompositional Reasoning in VLMs VLM Reasoning
Arizona State University

Nov 3, 2026J/K move · Enter open · S save

Factorized MLLM Embeddings for Composed Image Retrieval

WonJun Moon, Seongchan Kim, Dahun Kim, Seungryong Kim

KAIST AI · Google DeepMind

Composed Image Retrieval Multimodal Embedding Compositional Representation Learning Compositional Reasoning in VLMs Multimodal Large Language Models

Open paper AskPDF arXiv

Composed Image Retrieval (CIR) aims to retrieve a target image given a reference image and a modification text. Although recent approaches leveraging Multimodal Large Language Models (MLLMs) have shown significant progress, we argue that the MLLM's compositional understanding remains underutilized. This stems from directly mapping composed queries into a retrieval embedding via contrastive supervision alone, without explicitly resolving the intended visual modifications. In this work, we introduce Factorized MLLM Embeddings (FactEmbed), which decouples compositional query comprehension from building discriminative embeddings. Rather than directly projecting the composed query into a single embedding without internal reasoning, FactEmbed employs a set of factorized latents that simulate the intended visual modifications via an intermediate target imagination objective. This process encourages the latents to ground into target semantics before forming discriminative representations for retrieval. Subsequently, to propagate these factorized representations throughout the model for retrieval, we optimize a factor-level contrastive loss while distilling the decomposed representations into a deeper attention layer of the MLLM. Finally, for efficient inference, we retain the [EOS] token to establish a global-level embedding on top of these factorized latents, while we exploit the mean-pooled factorized latents to rerank a short candidate list at negligible cost, as both are produced by the same forward pass. Extensive evaluations demonstrate that FactEmbed outperforms prior works across CIR benchmarks.