cs.CVNov 7, 2026

They Can See It but Not Say It: Iterative Self Knowledge Re-expression in Visual Reasoning Relative Pose Identification

Authors: Mengyu Wang, Ken Deng, Yftah Ziser, Shay B. Cohen, Tiejun Ma

Organizations: University of Edinburgh · University of Oxford · 4Nvidia Research

Abstract

The task of visual reasoning relative pose identification (VRRPI) has exposed critical weaknesses in the multi-view 3D spatial reasoning capabilities of modern vision-language models (VLMs). We present a systematic analysis of VLMs' behavior in VRRPI, uncovering stable output biases where VLMs disproportionately predict specific motion patterns regardless of visual evidence. To mitigate these inherent biases, we propose self knowledge re-expression with iterative debiasing (SKR-ID), an annotation-free pipeline that adapts VLMs for VRRPI using only randomly extracted unannotated image pairs. SKR-ID transitions VLMs' output mechanism from generic next-token prediction to task-optimized classification heads. The pipeline proceeds in two stages: (1) a cold-start initialization using debiasing object-centric prompts to anchor motion inference in 2D visual evidence; and (2) a recursive logit-ranking mechanism that enforces statistical parity by using the median logit value as a dynamic threshold. Experimental evaluation across frontier VLMs on standard benchmarks demonstrates that SKR-ID significantly mitigates systematic biases. Our method improves open-source VLMs' Macro-F1 scores from 35.0-45.4% to 53.7-58.0%, without using any annotated data. These results show that SKR-ID can effectively improve 3D reasoning performance from a severely collapsed baseline. The code is publicly available at https://github.com/astudentuser/Codes_of_SKR-ID.

Figures & tables

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ProSR: Process-Shaped Spatial Reasoning for Reliable Chain-of-Thought in VLMs

    May 25, 2026Jiangyang Li, Cong Wan, Changjie Wu +8Visual Spatial ReasoningVLM Reasoning

  2. Position: The Systemic Lack of Agency in Visual Reasoning

    Jun 11, 2026Yizhao Huang, Haoyang Chen, Shiqin Wang +6VLM EvaluationVLM Reasoning

  3. Why Far Looks Up: Probing Spatial Representation in Vision-Language Models

    May 28, 2026Cheolhong Min, Jaeyun Jung, Daeun Lee +5Representation LearningVisual Spatial Reasoning