cs.CVNov 10, 2026

Reasoning-Aware Relational Representation Learning for Open-Vocabulary Scene Graph Generation

Authors: Jiawei Xiao, Boya Wang, Longtian Qiu, Shan Ning, Xuming He

Organizations: ShanghaiTech University, Shanghai, China · Lingang Laboratory, Shanghai, China · Shanghai Engineering Research Center of Intelligent Vision and Imaging

Abstract

Open-Vocabulary Scene Graph Generation (OV-SGG) requires models to recognize visual relationships beyond the training vocabulary, yet existing methods often rely on dataset-specific object--predicate co-occurrence patterns. Under long-tailed distributions and noisy object localization, such reliance leads to biased, entangled relation representations, limiting both rare-relation recognition and generalization to unseen predicates. To address these challenges, we propose RaRe, a reasoning-aware relational representation learning framework that transforms MLLM-generated chain-of-thought (CoT) rationales into structured relation embeddings. Rather than using CoT only as an intermediate reasoning trace for final prediction, RaRe directly optimizes the rationale-derived representation space to improve inter-relation discriminability. Specifically, RaRe first aligns rationale embeddings with textualized relation descriptions through self-supervised contrastive learning, and then refines rationale generation with a specificity-oriented reinforcement learning objective that encourages semantically object-pair-specific relational evidence. Experiments on VG150 and PSG demonstrate improved category-balanced recognition over existing baselines, while the additional gains from learning vary across datasets and proposal settings. Code available at: https://artanic30.github.io/project_pages/RaRe.

Figures & tables

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ReLIC-SGG: Relation Lattice Completion for Open-Vocabulary Scene Graph Generation

    Apr 24, 2026Amir Hosseini, Sara Farahani, Xinyi Li +1Scene Graph GenerationPositive-Unlabeled Learning

  2. SGG-R3^{\rm 3}: From Next-Token Prediction to End-to-End Unbiased Scene Graph Generation

    Mar 9, 2026Jiaye Feng, Qixiang Yin, Yuankun Liu +2Scene Graph GenerationLLM Reasoning with Graphs

  3. CAGE-SGG: Counterfactual Active Graph Evidence for Open-Vocabulary Scene Graph Generation

    Apr 24, 2026Suiyang Guang, Chenyu Liu, Ruohan Zhang +1Vision-Language Grounding