Compositional Reasoning in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

22 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Can the Parts Fool the Test? Counterfactual Pairing Cycles for Relational OOD
Oct 16, 2026Yibo Zhou, Yirong XiangCounterfactual Evaluation of VLMs Compositional Reasoning in VLMs
Beihang University · Air Force Medical Center
Composition, Not Conversation: VLMs Lose the Scene, Not the Thread
Sep 29, 2026L. D. M. S. Sai Teja, Ufaq Khan, N. Siva Gopala Krishna +5VLM Evaluation Visual Question Answering
NIT Silchar · MBZUAI · BML Munjal University +1
From Perception to Integration: Revisiting the Internal Dynamics of Reasoning in Vision-Language Models
Sep 28, 2026Rong Yu Xu, Prayag Tiwari, Shaolei ZhangEfficient VLM Inference VLM Interpretability
Shenzhen College of International Education · Halmstad University, Sweden · Renmin University of China
Diagnosing the Sources of Compositional Failure in Vision-Language Models: A Controlled Analysis
Sep 25, 2026Mona Gandhi, Cenk Merih Olcay, Kuan-Chieh Lo +3Compositional Reasoning in VLMs VLM Reasoning
The Ohio State University · Netflix Research
CS-CLIP: Compositional Scene Graph-guided CLIP for Robust Compositional Reasoning
Sep 8, 2026SeongJun Jeong, Minjoon Jung, Woo Suk Choi +2VLM Robustness Contrastive Learning
Seoul National University
The Limits of Binding in Dual Encoders
Aug 16, 2026Kin Ian LoCompositional Reasoning in VLMs Contrastive Learning
University College London
VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use
Aug 9, 2026Juan S. SantillanaVLM Evaluation Tool Use in VLMs
Globant
Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models
Jul 25, 2026Sultan Alshehri, Zhantao Yang, Han Zhang +1Vision-Language Models VLM Reasoning
Carnegie Mellon University, Pittsburgh, PA, USA.
H-GRPO: Permutation-Invariant Reinforcement Learning for Grounded Visual Reasoning
Jun 29, 2026Eric Peh, Debaditya Roy, Basura FernandoRL for VLMs Visual Reasoning
Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, India +1
Falcon: Functional Assembly and Language for Compositional Reasoning in X-ray
Jun 24, 2026Yonathan Michael, Mohamad Alansari, Natnael Takele +2Multimodal Grounding Compositional Reasoning in VLMs
Computer Science Department Khalifa University · Center of Cyber-physical Systems (C2PS) Khalifa University
Self-Questioning Vision-Language Models: Reinforcement Learning for Compositional Visual Reasoning
Jun 14, 2026Saraswathy AmjithVisual Question Answering Visual Reasoning
Massachusetts Institute of Technology
Cross-Modal Masked Compositional Concept Modeling for Enhancing Visio-Linguistic Compositionality
Jun 11, 2026Wei Li, Zhen Huang, Xinmei TianCross-Modal Learning Vision-Language Models
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China · Independent Researcher
HyperVis: Continuous Latent Visual Relational Graphs on the Lorentz Hyperboloid for Compositional Reasoning
Jun 4, 2026Moshiur Farazi, Sameera Ramasinghe, Mahbub Ahmed Turza +1Hyperbolic Representation Learning Visual Reasoning
Data Science and AI, University of Doha for Science and Technology, Qatar · Pluralis Research, Australia · Department of Electrical and Computer Engineering, North South University, Bangladesh
ReGuLaR: Relation-Grounded Latent Reasoning for Large Vision-Language Models
May 28, 2026Zihu Wang, Karthik Somayaji N. S, Peng LiLatent Visual Reasoning Large Vision-Language Models
University of California, Santa Barbara
Concrete Jungle: Towards Concreteness Paved Contrastive Negative Mining for Compositional Understanding
Apr 14, 2026Eun Woo Im, Dhruv Madhwal, Vivek GuptaCompositional Reasoning in VLMs VLM Reasoning
Arizona State University
Revisiting Compositionality in Dual-Encoder Vision-Language Models: The Role of Inference
Apr 13, 2026Imanol Miranda, Ander Salaberria, Eneko Agirre +1Compositional Reasoning in VLMs Vision-Language Models
HiTZ Center – Ixa, University of the Basque Country (UPV/EHU)
SignReasoner: Compositional Reasoning for Complex Traffic Sign Understanding via Functional Structure Units
Apr 12, 2026Ruibin Wang, Zhenyu Lin, Xinhai ZhaoTraffic Sign Detection Compositional Reasoning in VLMs
Huawei Noah’s Ark Lab
Inference-Time Structural Reasoning for Compositional Vision-Language Understanding
Mar 28, 2026Amartya BhattacharyaCompositional Reasoning in VLMs VLM Reasoning
Biomedical Data Science, Geisel School of Medicine, Dartmouth College
No Hard Negatives Required: Concept Centric Learning Leads to Compositionality without Degrading Zero-shot Capabilities of Contrastive Models
Mar 26, 2026Hai X. Pham, David T. Hoffmann, Ricardo Guerrero +1Vision-Language Models Compositional Reasoning in VLMs
Samsung AI Center Cambridge, UK
Edge Reliability Gap in Vision-Language Models: Quantifying Failure Modes of Compressed VLMs Under Visual Corruption
Mar 24, 2026Mehmet Kaan ErolCompositional Reasoning in VLMs VLM Evaluation
Marmara University, Institute of Pure and Applied Sciences
Rethinking Visual Privacy: A Compositional Privacy Risk Framework for Severity Assessment with VLMs
Mar 23, 2026Efthymios Tsaprazlis, Tiantian Feng, Anil Ramakrishna +3VLM Evaluation Privacy Auditing
University of Southern California, Los Angeles CA 90089, USA · Meta Superintelligence Labs · Amazon AGI
VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chaining
Mar 16, 2026Xuanyu Zhu, Yuhao Dong, Rundong Wang +9Compositional Reasoning in VLMs Tool Use in VLMs
PKU · Meituan · NTU +4

Oct 16, 2026J/K move · Enter open · S save

Can the Parts Fool the Test? Counterfactual Pairing Cycles for Relational OOD

Yibo Zhou, Yirong Xiang

Beihang University · Air Force Medical Center

Counterfactual Evaluation of VLMs Compositional Reasoning in VLMs OOD Detection

Open paper AskPDF arXiv

Relational out-of-distribution errors are easy to describe and hard to evaluate cleanly. An image may be natural, a caption may be fluent, and all mentioned objects may be common, yet the caption may describe the wrong relation in the image. Many evaluations intended to test this behavior can be solved without checking the pairing at all, because negatives differ in caption style, template, length, object frequency, or image statistics. We propose counterfactual pairing cycles, a blocked evaluation primitive that reuses the same two images and the same two captions on both sides of the comparison, changing only which image is paired with which caption. This makes image-only, text-only, and additive component shortcuts cancel exactly in finite samples. The same contrast also gives a training loss for relational outlier exposure and a diagnostic for bilinear vision-language scores. Across synthetic controls, GQA, and COCO, cycles separate coarse image-caption matching from directional role-binding failures, while also identifying geometric size relations that remain strongly visible in global features. In a shortcut-contaminated COCO outlier-exposure stress test, a model selected by a near-perfect shortcut validation AUROC retains substantial relational signal but remains below cycle training on exact cycles; cycle training improves CycleAcc from 0.882 to 0.927. The main lesson is simple: to test whether a model understands a pairing, the benchmark should keep the parts fixed, change the pairing, and ask only about coupling. The official implementation is available online.