Paper ID: 2405.16213

Learning Visual-Semantic Subspace Representations for Propositional Reasoning

Gabriel Moreira, Alexander Hauptmann, Manuel Marques, João Paulo Costeira

Learning representations that capture rich semantic relationships and accommodate propositional calculus poses a significant challenge. Existing approaches are either contrastive, lacking theoretical guarantees, or fall short in effectively representing the partial orders inherent to rich visual-semantic hierarchies. In this paper, we propose a novel approach for learning visual representations that not only conform to a specified semantic structure but also facilitate probabilistic propositional reasoning. Our approach is based on a new nuclear norm-based loss. We show that its minimum encodes the spectral geometry of the semantics in a subspace lattice, where logical propositions can be represented by projection operators.

Submitted: May 25, 2024