Orthogonal Representation Learning

Ask about this topic

Overview

Orthogonal representation learning studies how to arrange learned features into perpendicular directions or subspaces so that different factors carry less redundant information. Common approaches include orthogonality and cross-covariance penalties, Stiefel-manifold or Householder/Cayley parameterizations, and decompositions into shared and task-specific or common and style components; related methods use orthogonal projections and Jacobian constraints. These designs are applied in contrastive and multimodal learning, continual learning, recommendation, scientific modeling, and generative disentanglement, with evaluations spanning robustness, domain transfer, compositional generalization, and task accuracy. A central tension is that stronger separation can improve isolation and stability but may also discard useful interactions or restrict capacity, making the choice of subspace dimension and constraint strength an ongoing challenge.

Latest papers

CardsList

36 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

  1. OrthoRF: Exploring Orthogonality in Object-Centric Representations

    Oct 26, 2026Despoina Touska, Bastiaan Onne Fagginger Auer, Alexandru Onose +6Object-Centric Representation LearningUnsupervised Visual Concept Discovery

    Faculty of Science, University of Amsterdam, Amsterdam, the Netherlands · ASML Netherlands B.V., Veldhoven, the Netherlands · Advanced Research Center for Nanolithography, Amsterdam, the Netherlands +1

  2. Greedy Orthogonal Polynomial Manifolds with Lipschitz Control for Dimensionality Reduction and Reduced Modeling

    Oct 24, 2026Tomoki Koike, Benjamin Peherstorfer, Elizabeth QianDimensionality ReductionManifold Learning

    School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA · Institute of Mathematics, EPFL, Lausanne, Switzerland · School of Aerospace Engineering and School of Computational Science and Engineering, Georgia Institute of Technology, Atlanta, GA, USA

  3. Making LoRA Identifiable: Orthogonal Alignment for Continual Learning

    Oct 12, 2026Pratik Rakesh Singh, Mohammadi Zaki, Akash Saha +2Orthogonal Procrustes AlignmentContinual LoRA

    Sony Research India

  4. LoomSC: Scalable Deep Subspace Clustering with Projector Factorization and Exact Spectral Reduction

    Oct 7, 2026Nairouz Mrabah, Youssef Melki, Mohamed Bouguessa +3ClusteringSpectral Clustering

    École de technologie supérieure (ÉTS), Montréal, QC, Canada · University of Bahrain (UOB), Kingdom of Bahrain · Université du Québec à Montréal (UQAM), Montréal, QC, Canada +1

  5. Learning Orthogonal Multi-Index Models Beyond Small Initialization: Incremental Learning, Competitive Dynamics and Symmetry

    Sep 9, 2026Mo Zhou, Weihang Xu, Simon S. Du +1Shallow Neural NetworksOrthogonal Representation Learning

    University of Washington · Amazon, Inc.

  6. Orthogonal JEPA: Factorized Predictive States for Latent World Models

    Aug 20, 2026Taoyong Cui, Pheng Ann Heng, Wanli OuyangJoint-Embedding Predictive ArchitectureOrthogonal Representation Learning

    The Chinese University of Hong Kong (CUHK)

  7. Composed Historical Image Retrieval by Modeling Temporal Representations

    Aug 19, 2026Adrià Molina Rodríguez, Oriol Ramos Terrades, Josep Lladós CanetTemporal Representation LearningRepresentation Learning

    Centre de Visió per Computador Universitat Autònoma de Barcelona Bellaterra, Catalonia · Computer Science Department Universitat Autònoma de Barcelona, Bellaterra, Catalonia

  8. Sparse Orthogonal Regression Technique: A Spectral Framework for Equation Discovery, Approximation, and Integration

    Aug 13, 2026Sabin Roman, Ljupco Todorovski, Saso DzeroskiSparse Identification of Nonlinear DynamicsNonlinear System Identification

    Department of Knowledge Technologies, Jožef Stefan Institute · Faculty of Mathematics and Physics, University of Ljubljana

  9. Exploring Oversmoothing with Householder Matrices

    Aug 12, 2026Bhaskar KarolGraph Neural NetworksGNN Oversmoothing

  10. OrEdge: Efficient Multi-Modal Anomaly Detection in Distributed Software Systems via Orthogonal-Domain Learning

    Jul 31, 2026Amr M. Zaki, Farhoud Jafari Kaleibar, Honggeun Ji +2Edge InferenceOrthogonal Representation Learning

    School of Engineering, York University, Toronto, ON, Canada.

  11. Projection Pursuit CPCANet for Domain Generalization

    Jul 24, 2026Yu-Hsi Chen, Abd-Krim SeghouanePrincipal Component AnalysisDomain Generalization

    The University of Melbourne

  12. Gradient-Based Latent Decomposition Reveals Mechanisms of Feature Degradation in Weakly Supervised Mammography

    Jul 24, 2026Vinceline Bertrand, Ionut CardeiVariational AutoencodersHierarchical Representation Learning

    Florida Atlantic University, Boca Raton, FL, USA

  13. Escaping the Procrustean Bed: Groupwise Orthogonal Connectors for Audio-Language Models

    Jul 7, 2026Ho-Lam Chung, Ke-Han Lu, Yi-Cheng Lin +3Audio Representation LearningRepresentation Learning

    Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · ASUS AICS, Taipei, Taiwan

  14. Orthogonal Dendritic Intrinsic Networks: An Architecture for Significance-Ordered, Orthogonal Latent Spaces

    Jul 6, 2026Jeanie Schreiber, Tyrus Berry, Zeeshan AhmedRepresentation LearningPrincipal Component Analysis

    Department of Mathematical Sciences George Mason University Fairfax, VA 22030, USA · Sensor Science Division, Physical Measurement Laboratory NIST Gaithersburg, MD 20899, USA

  15. Unsupervised Disentanglement Without Compromises : How Functional Orthogonality Enforces Identifiability

    Jun 19, 2026Mathieu Cyrille Simon, Pascal Frossard, Christophe De VleeschouwerUnsupervised LearningDisentangled Representation Learning

    1UCLouvain, ICTEAM, Louvain-la-Neuve, Belgium · 2EPFL, LTS4 laboratory, Lausanne, Switzerland.

  16. Dual-Granularity Orthogonal Disentanglement for Generalizable Audio Deepfake Detection

    Jun 15, 2026Zhuodong Liu, Hugen Lv, Xiangyu Li +1Representation LearningAudio Deepfake Detection

    Beijing Jiaotong University, China · Shanghai Jiao Tong University, China · ITMO University, Russia

  17. OSCS-SupCon: Orthogonal Sigmoid-based Common and Style Supervised Contrastive Learning for Robust Feature Disentanglement

    May 29, 2026Bin Wang, Fadi DornaikaContrastive LearningDisentangled Representation Learning

    University B.of theWang1Basqueand Country · IKERBASQUE,2

  18. Learning in Low-Dimensional Subspaces: Orthogonal Bottlenecks for Reinforcement Learning

    May 25, 2026Aleksandar Todorov, Matthia SabatelliRepresentation LearningReinforcement Learning

    University of Groningen, Groningen, The Netherlands

  19. AURORA: Contextual Orthogonalization for Geometric Representation Learning in Healthcare Foundation Models

    May 18, 2026Yuanyun Zhang, Shi LiDisentangled Representation LearningRepresentation Learning

    University of the Chinese Academy of Sciences · Columbia University

  20. The EΔΔ-MHC-Geo Transformer: Adaptive Geodesic Operations with Guaranteed Orthogonality

    May 7, 2026Arash ShahmansooriTransformerResidual Learning

    Independent Researcher

  21. Disentangling Shared and Task-Specific Representations from Multi-Modal Clinical Data

    May 5, 2026He Lyu, Huolin Zeng, Junren Wang +7Disentangled Representation LearningRepresentation Learning

    Department of Anesthesiology and West China Biomedical Big Data Center West China Hospital, Sichuan University, Chengdu, China · Med-X Center for Informatics, Sichuan University, Chengdu, China · Department of National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu, China +4

  22. CCAR: Intrinsic Robustness as an Emergent Geometric Property

    Apr 18, 2026Akash Samanta, Manish Pratap Singh, Debasis ChaudhuriDisentangled Representation LearningRepresentation Learning

    Techno India University, Salt Lake, Kolkata, India · DRDO Young Scientist Laboratory - CT, Chennai, India

  23. MOSAIC: Multi-Domain Orthogonal Session Adaptive Intent Capture for Prescient Recommendations

    Apr 11, 2026Abderaouf Bahi, Mourad Boughaba, Ibtissem Gasmi +2Recommender SystemsUser Preference Modeling

    Computer Science and Applied Mathematics Laboratory (LIMA), Faculty of Science and Technology, Chadli Bendjedid University, P.O. Box 73, El Tarf 36000, Algeria · Electro-Mechanical Systems Laboratory (LSEM), Faculty of Technology, Badji Mokhtar University, P.O. Box 12, Annaba 23000, Algeria · Mathematical Modeling and Numerical Simulation Laboratory (LAM2SIN), Faculty of Technology, Badji Mokhtar University, P.O. Box 12, Annaba 23000, Algeria

  24. Bridging MRI and PET physiology: Untangling complementarity through orthogonal representations

    Apr 8, 2026Sonja Adomeit, Kartikay Tehlan, Lukas Förner +7Medical ImagingMultimodal Fusion

    Dept. of diagnostic and interventional Radiology and Neuroradiology, University Hospital Augsburg, Germany · Digital Medicine, University Hospital Augsburg, Germany · Chair for Computer Aided Medical Procedures and Augmented Reality, Technical University of Munich, Germany +4

  25. Orthogonal Quadratic Complements for Vision Transformer Feed-Forward Networks

    Apr 8, 2026Wang ZixianTransformer FFNsVision Transformer

    China Mobile Communications Group Shandong Co., Ltd.

  26. OrthoFuse: Training-free Riemannian Fusion of Orthogonal Style-Concept Adapters for Diffusion Models

    Apr 6, 2026Ali Aliev, Kamil Garifullin, Nikolay Yudin +5Model MergingDiffusion Model Adaptation

    HSE University · FusionBrain Lab · AXXX

  27. OCP: Orthogonal Constrained Projection for Sparse Scaling in Industrial Commodity Recommendation

    Mar 19, 2026Chen Sun, Beilin Xu, Boheng Tan +6Recommender SystemsOrthogonal Representation Learning

    JD.com Beijing, China

  28. This Looks Distinctly Like That: Grounding Interpretable Recognition in Stiefel Geometry against Neural Collapse

    Mar 9, 2026Junhao Jia, Jiaqi Wang, Yunyou Liu +4Prototype-Based ClassificationExplainable Image Classification

    Medical Artificial Intelligence Lab, Westlake University, Hangzhou, China · Hangzhou Dianzi University, Hangzhou, China · Tencent Jarvis Lab, Shenzhen, China +1

  29. Learning embeddings of non-linear PDEs: the Burgers' equation

    Mar 8, 2026Pedro Tarancón-Álvarez, Leonid Sarieddine, Pavlos Protopapas +1Physics-Informed MLPDE Solving

    Departament de F´ısica Qu´antica i Astrof´ısica, Universitat de Barcelona Institut de Ciencies del Cosmos (ICC), Universitat de Barcelona, Marti i Franques 1, ES-08028, Barcelona, Spain. · Institute for Applied Computational Science Harvard University Cambridge, MA, USA · Institut de Ciencies del Cosmos (ICC), Universitat de Barcelona, Marti i Franques 1, ES-08028, Barcelona, Spain. +1

  30. Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models

    Feb 27, 2026Arnas Uselis, Andrea Dittadi, Seong Joon OhNeural Representation GeometryVisual Representation Learning

    Tübingen AI Center, University of Tübingen · Helmholtz · Technical University of Munich