Behavioral Diversity in RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Latest papers

22 papers4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay
Oct 16, 2026Wen-Zhi Li, Sainyam Galhotra, Claire CardieReinforcement Learning RL for Language Model Reasoning
Cornell University
EliteReplay: Preserving Diverse Solutions in RL Post-Training via Quality-Diversity Search
Oct 15, 2026Hanane Nour Moussa, Doeun Lee, Sachin KumarLanguage Model Post-Training RL Post-Training
The Ohio State University
Optimizing Team Behavior via Extremum-seeking Control in Multi-agent Reinforcement Learning
Oct 13, 2026Svar Rajankumar Patel, Kshitij JerathBehavioral Diversity in RL Cooperative MARL
Department of Computer Science, University of Massachusetts Lowell, Lowell, Massachusetts, USA · Department of Mechanical and Industrial Engineering, University of Massachusetts Lowell, Lowell, Massachusetts, USA
Behavioral Foundation Models for Quality Diversity
Sep 28, 2026Nazim Bendib, Nicolas Perrin-Gilbert, Olivier SigaudReinforcement Learning Quality-Diversity Optimization
Sorbonne Université, ISIR Paris, France
From Optimal Policies to Individual Differences: Rethinking Reinforcement Learning for Biology
Jul 17, 2026Patrick Govoni, Palina Bartashevich, Clémence Bergerot +3Reinforcement Learning Behavioral Diversity in RL
Institute for Theoretical Biology, Humboldt Universität zu Berlin, Berlin, Germany · Faculty of Life Sciences, Thaer-Institute for Agricultural and Horticultural Sciences, Humboldt Universität zu Berlin, Berlin, Germany · Science of Intelligence, Cluster of Excellence, Berlin, Germany +1
Objective-Behavior Alignment: Diagnostics for MORL Policy Selection
Jun 19, 2026Antonio Mone, Zuzanna Osika, Florian Felten +4Multi-Objective Reinforcement Learning Reinforcement Learning
Delft University of Technology, the Netherlands · ETH Zurich, Switzerland & herve.review, Belgium · ETH Zurich, Switzerland
Structure-Conditioned Actor-Critic Branches for Quality-Diversity Reinforcement Learning
Jun 7, 2026Lianrong Zuo, Peilan Xu, Yong Liu +1Reinforcement Learning Policy Optimization
Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Institute of Cyberspace Security, School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen 518055, Guangdong, China
Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning
Jun 2, 2026Anthony GX-Chen, Ankit Anand, Gheorghe Comanici +7Reinforcement Learning RL for Language Model Reasoning
New York University · Google DeepMind
Diversifying RLVR Rollouts via First-Token Exploration
May 27, 2026Soeun Kim, Albert NoRL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Department of Artificial Intelligence, Yonsei University
Beyond Mode Collapse: Distribution Matching for Diverse Reasoning
May 19, 2026Xiaozhe Li, Yang Li, Xinyu Fang +10KL-Regularized RL RL for Language Model Reasoning
1Tongji University, China · 6Independent · 2Shanghai AI Laboratory, China +3
Events as Triggers for Behavioral Diversity in Multi-Agent Reinforcement Learning
May 12, 2026Hannes Büchi, Manon Flageat, Eduardo Sebastián +1Behavioral Diversity in RL Cooperative MARL
Department of Computer Science and Technology, University of Cambridge, UK
Behavioral Mode Discovery for Fine-tuning Multimodal Generative Policies
May 12, 2026Alberta Longhini, David Emukpere, Jean-Michel Renders +1RL for Robotics Reinforcement Learning
Department of Computer Science, Stanford University, CA, USA
Graph-SND: Sparse Aggregation for Behavioral Diversity in Multi-Agent Reinforcement Learning
May 6, 2026Shawn RayMulti-Agent Reinforcement Learning Behavioral Diversity in RL
Carnegie Mellon University
Binary Rewards and Reinforcement Learning: Fundamental Challenges
May 4, 2026Marc DymetmanReinforcement Learning KL-Regularized RL
∗Scientific Consultant
Uniform-Correct Policy Optimization: Breaking RLVR's Indifference to Diversity
May 1, 2026Anamika Lochab, Bolian Li, Ruqi ZhangRL for Language Model Reasoning Group Relative Policy Optimization
Department of Computer Science Purdue University, West Lafayette
Distributional Value Estimation Without Target Networks for Robust Quality-Diversity
Apr 22, 2026Behrad Koohy, Jamie BayneDistributional RL Reinforcement Learning
Luffy.AI Culham, Oxfordshire, United Kingdom
The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping
Apr 13, 2026Yang Liu, Enxi Wang, Yufei Gao +6RL for Language Models Reinforcement Learning
Fudan University · Shanghai Innovation Institute
Maximum Entropy Behavior Exploration for Sim2Real Zero-Shot Reinforcement Learning
Mar 26, 2026Jiajun Hu, Nuria Armengol Urpi, Jin Cheng +1Zero-Shot RL Reinforcement Learning
Department of Mechanical Engineering, EPFL · Department of Computer Science, ETH Zurich · Max Planck Institute for Intelligent Systems
DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replay
Mar 17, 2026Long Li, Zhijian Zhou, Tianyi Wang +7Behavioral Diversity in RL Reinforcement Learning
Griffith University, Australia · Fudan University, China · Shanghai Innovation Institute, China +3
Entropy-Preserving Reinforcement Learning
Mar 12, 2026Aleksei Petrenko, Ben Lipkin, Kevin Chen +4Policy Gradient Methods Behavioral Diversity in RL
Apple · MIT
Policy Improvement with Style-Specific Demonstrations
Jun 20, 2025Lingfeng Li, Yunlong Lu, Yongyi Wang +1Reinforcement Learning Game-Playing Agents
School of Computer Science, Peking University, Beijing 100871, China
Diversity-Enriched Option-Critic
Nov 4, 2020Anand Kamat, Doina PrecupHierarchical RL Intrinsic Motivation
McGill University · Mila · 3Google deepmind

Oct 16, 2026J/K move · Enter open · S save

Preserving Reasoning Diversity in Reinforcement Learning with Verifiable Rewards via Adaptive Replay

Wen-Zhi Li, Sainyam Galhotra, Claire Cardie

Cornell University

Reinforcement Learning RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Behavioral Diversity in RL LLM Reasoning

Open paper AskPDF arXiv

Reinforcement Learning with Verifiable Rewards (RLVR) improves the reasoning capabilities of large language models, but often reduces the diversity of their reasoning behaviors. We study this phenomenon from the perspective of finite-sample learning dynamics and show that limited on-policy rollouts can progressively suppress semantically distinct but equally rewarded reasoning modes. Once a mode becomes sufficiently under-sampled, standard on-policy RLVR rarely recovers it. Motivated by this observation, we propose TAR, a Targeted Adaptive Replay framework that preserves diverse correct reasoning behaviors already present in the base policy. TAR constructs a compact memory bank of diverse base-model solutions and selectively replays them through an adaptive mechanism that activates only when stored behaviors are being suppressed during RLVR. Across six mathematical reasoning benchmarks and two Qwen2.5 model scales, TAR consistently achieves a better balance between reasoning performance and diversity preservation than existing diversity-preserving post-training methods, while substantially reducing the forgetting of correct base-model solutions. Code is available at https://github.com/wenzhilics/tar.