Behavioral Diversity in RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

34 papers7 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Not All Entropy Is Equal: Reward-Conditioned Entropy Allocation in Reinforcement Learning with Verifiable Rewards
Nov 6, 2026Song Yu, Li Li, Zhenming WuEntropy Collapse in RL RL for Language Model Reasoning
School of Computer and Information Science, Southwest University, Chongqing 400715, China
Diversity-Aware Experience Replay via Statistical Leverage Scores
Nov 4, 2026Shaifalee Saxena, Alexander ScheinkerOff-Policy RL Prioritized Experience Replay
Instrumentation & Controls Group at Los Alamos National Laboratory, Los Alamos, New Mexico 87545, USA. · Shaifalee Saxena is a PhD student at the University of New Mexico, Albuquerque, New Mexico, 87131, USA.
RIDGE: State-Conditioned Reward Blending for Behavioral Coverage in Deep RL Game Agents
Nov 3, 2026AL Muqshith Mohammed Shifan, Kevin Christopher Chua, Ali Neshati +2Behavioral Diversity in RL Reward Design for RL
Ontario Tech University
Density-Aware Reward Scaling: A Reinforcement Learning Framework for Reward-Proportional Sampling
Nov 3, 2026Ria Shekhawat, Abhijeet Sinha, Yingtao Zhu +1Diversity-Aware Sampling Behavioral Diversity in RL
National University of Singapore
When Rewards Depend on the Group, GRPO Plays a Tournament
Nov 2, 2026Junyi Gao, Yu Shi, Pingzhao Hu +1Group Relative Policy Optimization Reward Design for RL
Centre for Medical Informatics, University of Edinburgh · Health Data Research UK · Biostatistics Division, Dalla Lana School of Public Health, University of Toronto +1
Repulsive Population Policy Gradients for Behaviorally Diverse Reinforcement Learning
Oct 27, 2026Saeed Hedayatian, Harsh Shah, Celine Mneimneh +1Reinforcement Learning Behavioral Diversity in RL
Thomas Lord Department of Computer Science, University of Southern California, Los Angeles, USA · Independent Researcher
Freeze, Don't Clone: Fine-Tuning with Expressive Policies Without Retaining Offline Data
Oct 26, 2026Sami Nur Islam, Johan Obando-Ceron, Doina Precup +2RL Fine-Tuning Offline-To-Online RL
Mila – Québec AI Institute · McGill University · Université de Montréal +1
Intrinsic Goals for Autonomous Agents: Hierarchical Exploration Captures Future Volitional Action in Drosophila
Oct 25, 2026Declan Kutscher, Reece Keller, Alyn Kirsch +2Intrinsic Motivation Intrinsic Reward Methods
Department of Computer Science, University of Maryland, College Park · Machine Learning Department, Carnegie Mellon University · Neuroscience Institute, Carnegie Mellon University +1
MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning
Oct 23, 2026Zhixia Zhang, Binqian Xu, Huaiyang Wang +2Test-Time RL RL for Language Model Reasoning
Beihang University · National University of Singapore
Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay
Oct 16, 2026Wen-Zhi Li, Sainyam Galhotra, Claire CardieReinforcement Learning RL for Language Model Reasoning
Cornell University
EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
Oct 15, 2026Hanane Nour Moussa, Doeun Lee, Sachin KumarLanguage Model Post-Training RL Post-Training
The Ohio State University
Optimizing Team Behavior via Extremum-seeking Control in Multi-agent Reinforcement Learning
Oct 13, 2026Svar Rajankumar Patel, Kshitij JerathBehavioral Diversity in RL Cooperative MARL
Department of Computer Science, University of Massachusetts Lowell, Lowell, Massachusetts, USA · Department of Mechanical and Industrial Engineering, University of Massachusetts Lowell, Lowell, Massachusetts, USA
Behavioral Foundation Models for Quality Diversity
Sep 28, 2026Nazim Bendib, Nicolas Perrin-Gilbert, Olivier SigaudReinforcement Learning Quality-Diversity Optimization
Sorbonne Université, ISIR Paris, France
From Optimal Policies to Individual Differences: Rethinking Reinforcement Learning for Biology
Jul 17, 2026Patrick Govoni, Palina Bartashevich, Clémence Bergerot +3Reinforcement Learning Behavioral Diversity in RL
Institute for Theoretical Biology, Humboldt Universität zu Berlin, Berlin, Germany · Faculty of Life Sciences, Thaer-Institute for Agricultural and Horticultural Sciences, Humboldt Universität zu Berlin, Berlin, Germany · Science of Intelligence, Cluster of Excellence, Berlin, Germany +1
Objective-Behavior Alignment: Diagnostics for MORL Policy Selection
Jun 19, 2026Antonio Mone, Zuzanna Osika, Florian Felten +4Multi-Objective Reinforcement Learning Reinforcement Learning
Delft University of Technology, the Netherlands · ETH Zurich, Switzerland & herve.review, Belgium · ETH Zurich, Switzerland
Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning
Jun 7, 2026Lianrong Zuo, Peilan Xu, Yong Liu +1Reinforcement Learning Policy Optimization
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen 518055, Guangdong, China
Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning
Jun 2, 2026Anthony GX-Chen, Ankit Anand, Gheorghe Comanici +7Reinforcement Learning RL for Language Model Reasoning
New York University · Google DeepMind
Diversifying RLVR Rollouts via First-Token Exploration
May 27, 2026Soeun Kim, Albert NoRL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Department of Artificial Intelligence, Yonsei University
Beyond Mode Collapse: Distribution Matching for Diverse Reasoning
May 19, 2026Xiaozhe Li, Yang Li, Xinyu Fang +10KL-Regularized RL RL for Language Model Reasoning
1Tongji University, China · 6Independent · 2Shanghai AI Laboratory, China +3
Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning
May 12, 2026Hannes Büchi, Manon Flageat, Eduardo Sebastián +1Behavioral Diversity in RL Cooperative MARL
Department of Computer Science and Technology, University of Cambridge, UK
Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies
May 12, 2026Alberta Longhini, David Emukpere, Jean-Michel Renders +1RL for Robotics Reinforcement Learning
Department of Computer Science, Stanford University, CA, USA
Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning
May 6, 2026Shawn RayMulti-Agent Reinforcement Learning Behavioral Diversity in RL
Carnegie Mellon University
Binary Rewards and Reinforcement Learning: Fundamental Challenges
May 4, 2026Marc DymetmanReinforcement Learning KL-Regularized RL
∗Scientific Consultant
Uniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversity
May 1, 2026Anamika Lochab, Bolian Li, Ruqi ZhangRL for Language Model Reasoning Group Relative Policy Optimization
Department of Computer Science Purdue University, West Lafayette
Distributional Value Estimation Without Target Networks for Robust Quality-Diversity
Apr 22, 2026Behrad Koohy, Jamie BayneDistributional RL Reinforcement Learning
Luffy.AI Culham, Oxfordshire, United Kingdom
The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping
Apr 13, 2026Yang Liu, Enxi Wang, Yufei Gao +6RL for Language Models Reinforcement Learning
Fudan University · Shanghai Innovation Institute
Maximum Entropy Behavior Exploration for Sim2Real Zero-Shot Reinforcement Learning
Mar 26, 2026Jiajun Hu, Nuria Armengol Urpi, Jin Cheng +1Zero-Shot RL Reinforcement Learning
Department of Mechanical Engineering, EPFL · Department of Computer Science, ETH Zurich · Max Planck Institute for Intelligent Systems
DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay
Mar 17, 2026Long Li, Zhijian Zhou, Tianyi Wang +7Behavioral Diversity in RL Reinforcement Learning
Griffith University, Australia · Fudan University, China · Shanghai Innovation Institute, China +3
Entropy-Preserving Reinforcement Learning
Mar 12, 2026Aleksei Petrenko, Ben Lipkin, Kevin Chen +4Policy Gradient Methods Entropy Collapse in RL
Apple · MIT
Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling
Jan 29, 2026Abhijeet Sinha, Sundari Elango, Dianbo LiuReinforcement Learning Behavioral Diversity in RL
National University of Singapore, Singapore · IIT Madras, India

Nov 6, 2026J/K move · Enter open · S save

Not All Entropy Is Equal: Reward-Conditioned Entropy Allocation in Reinforcement Learning with Verifiable Rewards

Song Yu, Li Li, Zhenming Wu

School of Computer and Information Science, Southwest University, Chongqing 400715, China

Entropy Collapse in RL RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Behavioral Diversity in RL Maximum Entropy RL

Open paper AskPDF arXiv

Policy entropy is widely used to diagnose exploration and collapse in reinforcement learning with verifiable rewards (RLVR), yet its magnitude does not consistently track reasoning performance across optimization objectives. We argue that this ambiguity arises because global token entropy conflates uncertainty in successful and failed trajectories, diversity across correct solution strategies, and variation among realizations of the same strategy. We introduce Reward-Conditioned Entropy Allocation (RCEA), a hierarchical diagnostic framework that separates complete-trajectory entropy from average token entropy, conditions uncertainty on verifier outcome and process validity, measures correct-solution-mode retention, and connects these population-level quantities to local covariance-driven updates. In controlled sweeps over six RLVR objectives at two model scales, entropy trajectories do not induce a stable ordering of reasoning accuracy. Token entropy is higher for verifier-failed than successful trajectories under both broad separately aggregated estimates and a common-question paired analysis, whose confidence intervals exclude equality at 10 of 12 displayed endpoints. A local covariance approximation predicts the direction of adjacent-checkpoint entropy change in 95.5% of nonfailed transitions; a reward-stratified endpoint proxy further shows that positive chosen-token covariance is more tail-concentrated within failed prefixes. Every reportable trained endpoint exhibits some contraction in independently audited correct-solution-mode breadth, whose positive association with large-

k

coverage remains exploratory. These results show that entropy collapse should be diagnosed by which uncertainty is removed, rather than by global entropy alone.