Contrastive RL

RL: Reinforcement Learning

Contrastive reinforcement learning uses comparisons—often through InfoNCE objectives—to learn which states, actions, or trajectories are compatible with a goal, enabling reward-free or goal-conditioned control. Work spans online and offline navigation, locomotion, and manipulation, with methods such as contrastive critics, action chunks, and direction-conditioned policies; language-model training also uses contrasts between successful and failed reasoning or tool-use trajectories. These approaches can reduce reliance on hand-crafted rewards and support transfer, but depend strongly on which examples are contrasted and how well learned representations match task geometry. A central tension is that a contrastive score may rank actions well without being a calibrated value, making reliable action selection, exploration, and generalization to rarely visited goals open challenges.

Latest papers

28 papers3 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Outperforming 1000 Euclidean Layers With Hyperbolic Goal-Conditioned RL
Oct 14, 2026Timo Klein, Thomas Lang, Andrii Shkabrii +3Reinforcement Learning Contrastive RL
Faculty of Computer Science, University of Vienna, Vienna, Austria · UniVie Doctoral School Computer Science, University of Vienna, Vienna, Austria · Department of Machine Learning and Systems Biology, Max Planck Institute of Biochemistry, Martinsried, Germany +1
Counterfactual Rollouts for Offline Contrastive Goal-Conditioned Reinforcement Learning
Oct 10, 2026Tingrui Huang, Ai Bo, Junzhe ZhangReinforcement Learning Offline RL
Eindhoven University of Technology, Eindhoven, The Netherlands · Department of Electrical Engineering and Computer Science, Syracuse University, Syracuse, NY, USA
Direction-Conditioned Policies for Online Goal-Conditioned Reinforcement Learning
Oct 4, 2026S K Swaminathan, Damiya Gondha, Theyanesh Eswaramoorthy Rajahkrishnan +1Reinforcement Learning Contrastive RL
Department of Computer Science and Engineering Indian Institute of Technology Kharagpur, India · Department of Mechanical Engineering Indian Institute of Technology Kharagpur, India
Learning to Zoom Efficiently with a Contrastive Curriculum
Sep 2, 2026Falko Helm, Iryna GurevychTool Use in VLMs Contrastive RL
Ubiquitous Knowledge Processing Lab (UKP Lab) Department of Computer Science and Hessian Center for AI (hessian.AI) Technical University of Darmstadt
Three Steps at a Time: Learning Representations from Action Sequences in Contrastive RL
Aug 31, 2026Michal Korniak, Kamil Dybek, Benjamin Eysenbach +2Reinforcement Learning Offline RL
ETH Zurich · University of Warsaw · Princeton University +3
Arrive and Survive: Scaling Safe Goal-Conditioned Policy Learning from One-Bit Failure Signals
Aug 27, 2026Guopeng Li, Yiyang Duan, Yiru Jiao +1Goal-Conditioned RL Safe RL
Southeast University Nanjing, China · Lab, Yinwang Intel. Tech. Co. Ltd. Shanghai, China
Contrastive Branch Policy Optimization
Aug 25, 2026Ying Wang, Changlin Qiu, Bang Lin +4RL for Language Model Reasoning Step-Level Credit Assignment in RL
Alibaba Group, Hangzhou, China · Harbin Institute of Technology, Harbin, China
Beyond Success and Failure: Length-Aware Contrastive Learning for GUI Agents
Aug 22, 2026Chengyang Gu, Le Zhang, Jingbo Zhou +6RL for GUI Agents Contrastive RL
Artificial Intelligence Thrust, Information Hub The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Baidu Inc. Beijing, China · Department of Electrical and Computer Engineering University of Alberta, Edmonton, Alberta, Canada
Contrastive Reinforced Policy Optimization via Privileged Self-Distillation
Jul 30, 2026Xingjian Wu, Junlin Liu, Xingchen Liu +6Agentic RL RL for Language Model Reasoning
Meituan
Goal Retrieval versus Action-Value Ordering: Contrastive Critics under Best-of- $K$ Selection
Jul 29, 2026Ayushman Singh, Siddharth AphaleLearning to Rank Contrastive RL
Stanford University · Sesame AI
RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Jul 20, 2026Yuxin Xiong, Xunyi Jiang, Rohan Surana +8Reinforcement Learning RL for Language Model Reasoning
University of California San Diego · 2Adobe Research
GCT-MARL: Graph-Based Contrastive Transfer for Sample-Efficient Cooperative Multi-Agent Reinforcement Learning
Jun 23, 2026Animesh Animesh, Satheesh K Perepu, Kaushik DeyGraph Contrastive Learning Contrastive RL
Department of Artificial Intelligence, Indian Institute of Technology Kharagpur, India · Work done during research internship at Ericsson Research, Bangalore, India · Ericsson Research, Bangalore, India
Context-Aware RL for Agentic and Multimodal LLMs
Jun 15, 2026Peiyang Xu, Bangzheng Li, Sijia Liu +4Multimodal Grounding Contrastive RL
1Princeton University · 2UC Davis
Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning
Jun 15, 2026Swaminathan S K, Damiya Gondha, Theyanesh Eswaramoorthy Rajahkrishnan +1Contrastive RL Policy Learning
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India · Department of Mechanical Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India.
Learning Object Manipulation from Scratch via Contrastive Interaction
Jun 10, 2026Tongle Shen, Caleb Chuck, Fan Feng +1Dynamical Systems Reinforcement Learning
1UC San Diego · 2UT Austin
Harmony in Diversity: Multi-domain Contrastive Policy Optimization for Large Reasoning Models
May 25, 2026Zongji Yu, Wenshui Luo, Yiliu Sun +5Reinforcement Learning RL for Language Model Reasoning
Shanghai Jiao Tong University · Shanghai AI Laboratory · Shandong University
Self-Supervised On-Policy Reinforcement Learning via Contrastive Proximal Policy Optimisation
May 13, 2026Asim Osman, Sasha Abramowitz, Mark Bergh +13Reinforcement Learning Self-Supervised RL
InstaDeep, AIMS · InstaDeep, University of Stellenbosch
Revisiting Reinforcement Learning with Verifiable Rewards from a Contrastive Perspective
May 13, 2026Feng Zhang, Xinhong Ma, Ziqiang Dong +5RL for Language Model Reasoning Group Relative Policy Optimization
Beijing Institute of Technology · Qwen Business Unit of Alibaba · The Chinese University of Hong Kong, Shenzhen +1
From Generic Correlation to Input-Specific Credit in On-Policy Self Distillation
May 12, 2026Guobin Shen, Lei Huang, Xiang Cheng +4Reinforcement Learning On-Policy Self-Distillation
Xiaohongshu Inc. · Institute of Automation, Chinese Academy of Sciences
PRTS: A Primitive Reasoning and Tasking System via Contrastive Representations
Apr 30, 2026Yang Zhang, Jiangyuan Zhao, Chenyou Fan +11Reinforcement Learning Long-Horizon Robotic Manipulation
Institute of Artificial Intelligence (TeleAI), China Telecom · Tsinghua University · Shanghai Jiao Tong University +1
ViSA: Visited-State Augmentation for Generalized Goal-Space Contrastive Reinforcement Learning
Mar 16, 2026Issa Nakamura, Tomoya Yamanokuchi, Yuki Kadokawa +5Reinforcement Learning Goal-Conditioned RL
I. Nakamura, T. Yamanokuchi, Y. Kadokawa, and T. Matsubara are with the Graduate School of Information Science, Nara Institute of Science and Technology (NAIST), Nara, Japan. · J. Qu, S. Otsubo, Y. Susumu, S. Miwa are with the Advanced Technology R&D Center, Mitsubishi Electric Corporation, Hyogo, Japan.
When Right Meets Wrong: Bilateral Context Conditioning with Reward-Confidence Correction for GRPO
Mar 13, 2026Yu Li, Tian Lan, Zhengling QiRL for Language Model Reasoning Group Relative Policy Optimization
Department of Electrical and Computer Engineering, George Washington University · School of Business, George Washington University
CLIPO: Contrastive Learning in Policy Optimization Generalizes RLVR
Mar 10, 2026Sijia Cui, Pengyu Cheng, Jiajun Song +6RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Qwen Large Model Application Team, Alibaba · Institute of Automation, Chinese Academy of Sciences
Let's Reward Step-by-Step: Step-Aware Contrastive Alignment for Vision-Language Navigation in Continuous Environments
Mar 10, 2026Haoyuan Li, Rui Liu, Hehe Fan +1Vision-Language Navigation RL for VLMs
College of Computer Science and Technology, Zhejiang University, Hangzhou, China
Scalable Dexterous Robot Learning with AR-based Remote Human-Robot Interactions
Feb 7, 2026Yicheng Yang, Ruijiao Li, Lifeng Wang +7Reinforcement Learning Robotic RL
Group Contrastive Policy Optimization: Learning Beyond the Self-Exploration Limit with Golden Answers
Oct 9, 2025Hao Wu, Wei Liu, Shizhe Xiang +3RL for Language Model Reasoning LLM Reasoning
CUDA-L1: Improving CUDA Optimization via Contrastive Reinforcement Learning
Jul 18, 2025Xiaoya Li, Albert Wang, Guoyin Wang +2Reinforcement Learning GPU Kernel Optimization
DeepReinforce Team
1000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilities
Mar 19, 2025Kevin Wang, Ishaan Javali, Michał Bortkiewicz +2Reinforcement Learning Self-Supervised RL
Princeton University · Warsaw University of Technology · Warsaw University of Technology, Tooploox, IDEAS Research Institute

Oct 14, 2026J/K move · Enter open · S save

Outperforming 1000 Euclidean Layers With Hyperbolic Goal-Conditioned RL

Timo Klein, Thomas Lang, Andrii Shkabrii, Lukas Miklautz+2

Faculty of Computer Science, University of Vienna, Vienna, Austria · UniVie Doctoral School Computer Science, University of Vienna, Vienna, Austria · Department of Machine Learning and Systems Biology, Max Planck Institute of Biochemistry, Martinsried, Germany+1

Reinforcement Learning Contrastive RL Goal-Conditioned RL Robot Navigation Hyperbolic Representation Learning

Open paper AskPDF arXiv

Scaling self-supervised contrastive reinforcement learning (CRL) to 1000-layer Euclidean neural networks has enabled state-of-the-art goal-reaching performance, yet wall-clock time scales linearly with depth. We find that changing the geometry achieves stronger performance with far shallower networks. This matters in complex navigation environments, where walls can make route distance deviate substantially from Euclidean distance. In a U-shaped maze, we show that the longer the route, the more it deviates from Euclidean distance. Hyperbolic CRL addresses this mismatch by replacing Euclidean distances in CRL's contrastive objective with geodesic distances in hyperbolic space. On hard navigation tasks, inductive bias beats scale: depth 64 hyperbolic agents outperform 1024-layer-deep Euclidean networks while training more than

12 \times

faster and using about

13 \times

fewer parameters. We observe that hyperbolic agents behave qualitatively better in hard humanoid mazes, where they follow corridors to reach goals, whereas Euclidean agents try straight-line shortcuts over walls. Our results show that matching the representation geometry to the task can substitute for substantial network depth. Code is at https://github.com/Probabilistic-and-Interactive-ML/HypGCRL .