Exploration-Exploitation Tradeoff

Follow this topic Ask about this topic

The field studies how a learner can discover which actions, options, or regions are valuable while still making use of what it has learned, in settings such as bandits, reinforcement learning, recommendation, and robotics. Common approaches include upper confidence bounds and Thompson sampling, randomized perturbations, Bayesian acquisition rules, and structured strategies such as explore-then-commit or coverage-driven planning. Regret guarantees are well developed for many bandit settings, but real tasks often add contextual information, partial feedback, changing environments, or limited trials. A central tension is that strategies that maximize immediate reward may gather too little information, while exploration that improves coverage or final decisions can cost short-term reward and may need to account for uncertainty, safety, and user or task differences.

Latest papers

59 papers10 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Self-organizing reinforcement learning: a learner-free swarm approaches the explore-exploit optimum
Oct 18, 2026Galen J. WilkersonMulti-Agent Resource Allocation Exploration-Exploitation Tradeoff
Komplexity AI LLC
Explore or Confirm? LLM Agents Misallocate Measurements Under Noise
Oct 18, 2026Abdul Karim, Matt KeonAI Agents for Scientific Discovery Exploration-Exploitation Tradeoff
Xstendyl
Clarity-Driven Adaptive Mission Planning for a Solar-Powered Autonomous Surface Vessel
Oct 14, 2026Kavin M. Govindarajan, Kaleb Ben Naveed, David Li +5Informative Path Planning Motion Planning
Kavin M. Govindarajan and Kaleb Ben Naveed are with the Department of Robotics, University of Michigan, Ann Arbor, MI 48109, USA · David Li is with the Department of Mechanical Engineering, University of Michigan, Ann Arbor, MI 48109, USA · Devansh R. Agrawal is a graduate of the Department of Aerospace Engineering, University of Michigan, Ann Arbor, MI 48109, USA +1
Uncertainty-Driven Self-Play for Robust Multi-Agent Driving Policies
Oct 14, 2026Sajad Marvi, Julian Schmidt, Alessandro Canevaro +2Uncertainty-Aware Exploration Autonomous Driving Planning
Mercedes-Benz AG, Germany. · Department of Computer Science, University of Freiburg, Germany.
Follow the Perturbed Leader for Linear Bandits: $\sqrt{T}$ Regret over General Action Sets
Oct 13, 2026Chen-Yu WeiFollow-the-Perturbed-Leader Adversarial Bandits
University of Virginia
MaskSense: Confronting the Visual Exploration Trap in Masked Image Generation
Oct 12, 2026Yawen Shao, Jie Xiao, Kai Zhu +6Generative Modeling RL for Image Generation
University of Science and Technology of China · Tongyi Lab, Alibaba Group · Northeastern University
Exploration strategies for Gaussian process bandit optimization with inference regret guarantees
Oct 12, 2026Hwanwoo Kim, Jungwoo Hong, Kyungjae LeeGaussian Process Bandits Bayesian Optimization
Department of Mathematical Sciences, University of Texas at Dallas · Department of Statistics, Korea University
Understanding Hidden Spatial Preferences in Bayesian Optimization
Oct 10, 2026Maria Bånkestad, Jens SjölundBayesian Optimization Uncertainty-Aware Exploration
Human-inspired, Task-Dimension-Guided Exploration for Efficient Learning in High Dimensions
Sep 29, 2026Fanyu Zhu, Jiahui An, Ni JiExploration-Exploitation Tradeoff RL Exploration
Beijing Institute for Brain Research, Chinese Academy of Medical Sciences & Peking Union Medical College, Chinese Institute for Brain Research, Beijing, Beijing Key Laboratory of Brain Science and Brain-Machine Interface · State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University
Signatures of semantic search in the activations of large language models
Sep 28, 2026Luke Leckie, Peter M. Todd, Jacob G. FosterExploration-Exploitation Tradeoff LLM Interpretability
Department of Informatics, Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN · Cognitive Science Program, Indiana University, Bloomington, IN · Center for Possible Minds, Indiana University, Bloomington, IN +2
LLM sequential decision making under uncertainty in biochemical domains
Sep 27, 2026Mattias Akke, Soojung Yang, Jurgis Ruža +2Large Language Model-Guided Scientific Discovery Large Language Model-Guided Optimization
Lund University · Stanford · MIT
Emergent Intelligence: Resonant Oscillators Produce Proactive Adaptive Behavior
Sep 18, 2026Alex Fedosov, Maxim Yakimenko, Sander StepanovSymmetry Discovery Oscillatory Neural Networks
FoundAItion Inc · FoundAItion Inc.
VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgets
Sep 11, 2026Yu Bai, Yukai Miao, Dawei Wang +8RL Benchmarks Reinforcement Learning
Zhongguancun Laboratory · Tsinghua University · Network Management Center, China Mobile
Learning Adaptive SED for heterogeneous load balancing
Sep 7, 2026Sanne van Kempen, Jaron Sanders, Fiona Sloothaak +1Queueing Theory Online Resource Allocation
Semantic Bandits: In-Context Exploration-Exploitation is Biased by Semantic Priors
Aug 17, 2026David Eric Austin, Kaheer Suleman, Jackie Chi Kit CheungExploration-Exploitation Tradeoff Linguistic Bias in Language Models
McGill University · Mila – Quebec AI Institute · Skyfall AI +1
Situation Aware Frontier Prioritization for Quadruped Search and Rescue
Aug 3, 2026Kevin Farias, Santiago Martin, Barbara Flores +5Exploration-Exploitation Tradeoff Robotics
Robotics and AI Lab, Technological University of Uruguay, Rivera, Uruguay
Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting
Jul 29, 2026Hongqiang Lin, Chao Liu, Xiaofan Bai +4Exploration-Exploitation Tradeoff LLM Agent Skill Learning
Zhejiang University · Alibaba Group
Experimentation and Commitment under Reward Shifts
Jul 26, 2026Puping Jiang, Wei Tang, Renyu +1Multi-Armed Bandits Exploration-Exploitation Tradeoff
Philip
Collaborative Spatial Learning with Multi-LLM Agents in Networked Social Experiments
Jul 16, 2026Hao He, Chris J. Kuhlman, Xinwei DengExploration-Exploitation Tradeoff Multi-Agent LLMs
Department of Statistics, Virginia Tech, Blacksburg, VA, USA · Advanced Research Computing, Virginia Tech, Blacksburg, VA, USA
Learning from Local Walks on Dynamic Graphs with Bandit Feedback
Jul 12, 2026Sourav Chakraborty, Amit Kiran Rege, Claire Monteleoni +1Multi-Armed Bandits Graph Bandits
Department of Computer Science, University of Colorado, Boulder, CO 80309, USA · INRIA, Paris, France
Towards Self-Evolving Agents: A Human-Inspired Adaptive Exploration-Exploitation Framework for Genetic Network Programming
Jul 4, 2026Ali Kohan, Mohamad Roshanzamir, Roohallah Alizadehsani +1Exploration-Exploitation Tradeoff Genetic Programming
Department of Computer Engineering, Faculty of Engineering, Fasa University, Fasa 74617-81189, Iran · Institute for Intelligent Systems Research and Innovation (IISRI), Deakin University, Geelong, Australia · Center of Excellence in Precision Medicine and Digital Health, Department of Physiology, Faculty of Dentistry, Chulalongkorn University, Bangkok 10330, Thailand. +2
Analysis of Parameter Settings for the Bat Algorithm Using Variance Evolution
Jun 26, 2026Xin-She Yang, Mehmet KaramanogluExploration-Exploitation Tradeoff Evolutionary Optimization
Faculty of Science and Technology, Middlesex University, The Burroughs, London NW4 4BT, United Kingdom.
Reward-free Pretraining for Reinforcement Learning via Occupancy Coverage Maximization
Jun 19, 2026Marco Pratticò, Pietro Novelli, Massimiliano Pontil +1Reinforcement Learning Exploration-Exploitation Tradeoff
Computational Statistics and Machine Learning - Istituto Italiano di Tecnologia, 16100 Genova, Italy · 2AI Centre, Computer Science Department, University College London, London, UK.
TreeSeeker: Tree-Structured Trial, Error, and Return in Deep Search
Jun 10, 2026Zhuofan Shi, Mingzhe Ma, Lu Wang +8Inference-Time Search Exploration-Exploitation Tradeoff
1Microsoft · 2East China Normal University
Exploratory Responsiveness and Adaptive Rigidity under AI-Assisted Optimization
Jun 8, 2026Balaraju BattuExploration-Exploitation Tradeoff Adaptive Learning
European University Institute, Florence, Italy · New York University Abu Dhabi, UAE
Algorithm for Contextual Queueing Bandits with Rate-Optimal Queue Length Regret
Jun 8, 2026Seoungbin Bae, Dabeen LeeExploration-Exploitation Tradeoff Regret Minimization
Department of Industrial & Systems Engineering, KAIST · Department of Mathematical Sciences, Seoul National University
Oblivious Learning and Collusive Pricing
Jun 3, 2026Yuhang Wu, Assaf ZeeviExploration-Exploitation Tradeoff Game Theory
Columbia Business School
Annealed Softmax Greedy in Many-Armed Bayesian Bandits
May 29, 2026William Overman, Mohsen BayatiMulti-Armed Bandits Exploration-Exploitation Tradeoff
Stanford University
Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization
May 27, 2026Hao Jiang, Shurui Li, Tianpeng Bu +7RL for Language Models Exploration-Exploitation Tradeoff
Alibaba Cloud Computing, Alibaba Group.
Human Decision-Making with AI Assistance under Correlated Features
May 27, 2026Yanru Guan, Naveen Raman, Fei FangAI-Assisted Decision Making Exploration-Exploitation Tradeoff
Carnegie Mellon University

Oct 18, 2026J/K move · Enter open · S save

Self-organizing reinforcement learning: a learner-free swarm approaches the explore-exploit optimum

Galen J. Wilkerson

Komplexity AI LLC

Multi-Agent Resource Allocation Exploration-Exploitation Tradeoff Multi-Agent Systems Reward-Free RL

Open paper AskPDF arXiv

The explore/exploit trade-off is the core problem of reinforcement learning (RL). We show that a minimal foraging swarm approaches its optimum - the ideal-free / Nash allocation of a multi-agent resource-sharing game (an equilibrium, not a return maximum) - with no learner and no reward signal. Agents explore by heavy-tailed (Levy) flights, recruit neighbours to a discovery through a threshold cascade on their proximity network, and disperse as a patch crowds; this cascade cycle balances exploration and exploitation and drives foragers toward the ideal free distribution. The swarm attains the shape of this optimum (committed foragers track richness with slope 0.94) but not its exact equilibrium - it stays measurably exploitable - reaching it dynamically (~150 steps) through the coupling of recruitment to movement: remove the coupling and matching collapses (0.94 -> 0.08). The RL sits at the level of the problem and the allocation reached, not the micro-rule: no step evaluates payoff (reward-free by construction, though the crowd signal is informationally tied to payoff), and the swarm reaches the allocation without learning it (quality plateaus and re-settles after resources move, no cumulative gain). Against a trained baseline the comparison is two-sided: in an easier, disembodied game a learner is more precise, while in the same embodied environment the reward-free swarm reaches ~65% of a validated oracle ceiling at zero training and leads until trained PPO and centralised-critic (CTDE) learners overtake it (~74-75%, 16 seeds, all above the swarm mean) after ~4,000 of 15,000 training steps - its advantage is in budget, not asymptote. In this one setting we read it not as beating RL but as a strong zero-shot, reward-free operating point: a demonstration that the self-organizing physics of a minimal swarm can carry a collective close to the optimum RL seeks, with none of RL's machinery.