Self-Play RL

RL: Reinforcement Learning

Overview

Self-play reinforcement learning studies how agents can improve by competing or cooperating with other versions of themselves, reducing reliance on fixed opponents or human-labeled examples. Methods include policy-gradient training with search, equilibrium and opponent-population methods, and challenger–solver loops that generate tasks and use verifiers or rubrics for feedback. These approaches are used in games such as Go, poker, and Hanabi, as well as language-model reasoning, tool use, and interactive driving. A central challenge is that agents can overfit to their training partners or collapse into unhelpful strategies, so diverse evaluation, useful task generation, and stable rewards remain important.

Latest papers

CardsList

86 papers7 papers in the last four weeks, down 46% on the four weeks before. 0.1% of all new papers.

  1. When Will You Hand It Over? A Hidden Start Delay Makes Self-Play Handover Robust to Unfamiliar Partners

    Nov 13, 2026Mohammad Khoshnazar, Mike Degany, Deyuan Qu +2Cooperative MARLSelf-Play RL

  2. Toward LLM Self-Play Under Data Scarcity: Effectiveness, Generalizability, and Trustworthiness

    Nov 6, 2026Zhiyin Yu, Yuchen Mou, Bingzhe Wu +3Self-Play for Language ModelsLanguage Model Self-Improvement

  3. EpiAgent: Epistemic Self-Play for LLM Agents

    Oct 29, 2026Xuanfa Jin, Xinyu Cui, Zhijian Ma +4LLM AgentsLLM Agent Training

  4. Tempo-Zero: Recursive Self-Play in Executable Worlds for Video Reasoning

    Oct 23, 2026Jingmin Zhu, Yanyan Zou, Xiuyu Li +10Video ReasoningSelf-Play RL

  5. TerraTransfer++: Demonstration-Free End-to-End Planner

    Oct 21, 2026Zikang Xiong, Weixin Li, Zhouchonghao Wu +6End-to-End Autonomous DrivingAutonomous Driving Planning

  6. The Predictive--Optimization Gap in GuanDan Self-Play

    Oct 21, 2026Qicheng Zou, Biao Yuan, Siyu Song +2Predict-Then-OptimizeSelf-Play RL

  7. Is Anyone Enforcing the Cartel? What Reinforcement Learning Teaches LLM Agents in Markets

    Oct 20, 2026Hieu Tran, Hong YuAlgorithmic CollusionRL for LLM Agents

  8. CasePlay: Self-Play Reinforcement Learning from Case Reports for Medical Reasoning

    Oct 18, 2026Hao Wang, Zihan Wang, Kai Wu +6Self-Play RLRL for Language Model Reasoning

  9. R2^2Orch: Recursive Self-Improvement for Agent Orchestration via Self-Play within a Skill-Internalization Loop

    Oct 15, 2026Mingda Zhang, Yun Huang, Qiang Huang +7LLM Agent OrchestrationSelf-Improving Agents

  10. Stop Training Against Soft Users: Mentalized Adversarial Self-Play for Proactive Dialogue

    Oct 15, 2026Zanwu Liu, Dan Zhang, Xiaodi Wang +6Conversational AgentsProactive Agents

  11. SHRED: Self-Play for Heterogeneous Road-Users without Expert Demonstrations

    Oct 14, 2026Wei-Jer Chang, Zhouchonghao Wu, Akshay Rangesh +3Self-Play RLAutonomous Driving Simulation

  12. Uncertainty-Driven Self-Play for Robust Multi-Agent Driving Policies

    Oct 14, 2026Sajad Marvi, Julian Schmidt, Alessandro Canevaro +2Uncertainty-Aware ExplorationAutonomous Driving Planning

  13. From Hitter to Player: Separating Motor Execution and Strategy in Competitive Humanoid Table Tennis

    Oct 7, 2026Fanrong Dong, Ziyi Sun, Ziqi Han +6Robotic Table TennisRobotics

  14. Faynt: Scaling and Optimizing Policies for Competitive Melee

    Oct 1, 2026Ali Janati, Nikita Kuzmin, Rohit Swamy +1Reinforcement LearningSelf-Play RL

  15. Regularized policy gradient with learned mixtures of Gaussians for games with continuous actions

    Sep 29, 2026Ondřej Kubíček, Viliam Lisý, Tuomas SandholmGame-Playing AgentsSelf-Play RL

  16. Learning to Optimize through Solver-Grounded Self-Play

    Sep 28, 2026Xia Jiang, Yaoxin Wu, Chenyu Zhou +3Self-Play RLNatural Language to Optimization Modeling

  17. CompassPlay: Rewarding the Proposer for Where It Moves the Solver

    Sep 26, 2026Sophia Xiao Pu, Ximeng Sun, Jiang Liu +4Self-Play RLReward Design for RL

  18. CounterPlay: Counterfactual Post-Training for Self-Play Driving Policies

    Sep 18, 2026Jiarong Wei, Yin Wu, Runkai He +1Autonomous DrivingRL Post-Training

  19. The Surprising Effectiveness of Approximate Value Iteration in Self-Play

    Sep 8, 2026Raphael Boige, Amine Boumaza, Bruno ScherrerGame-Playing AgentsSelf-Play RL

  20. SQL-Zero: Self-Evolving Text-to-SQL

    Sep 4, 2026Daniel Machado Pedrozo, Julia Soares Dollis, Bryan Lincoln Marques de Oliveira +3Reinforcement LearningSelf-Play RL

  21. Local Updates, Global Learning (LUGL): Playing Games with non-incremental Learners

    Sep 3, 2026David Milec, Spyridon Samothrakis, Michael Fairbank +1Gradient-Boosted Decision TreesGame-Playing Agents

  22. What Emerges and What Breaks in Self-Play Driving

    Aug 31, 2026Laur Sisask, Ardi Tampuu, Tambet MatiisenReward HackingSelf-Play RL

  23. Beyond Search-Imitation: Prior-Directed Exploration for Searchless Chess

    Aug 27, 2026Szymon Miłosz, Piotr Duch, Szymon GrabowskiChessRL Exploration

  24. ShuttleArena: Interpretable Self-Play in Physics-Based Badminton

    Aug 26, 2026Peize DingGame-Playing AgentsSelf-Play RL

  25. The Chase Is the Curriculum, the Capture Anchors the Credit: Pursuit-Evasion Self-Play for Zero-Data LLM Reasoning

    Aug 22, 2026Jing Yu, Shengchao Chen, Yiyun TanSelf-Play RLRL for Language Model Reasoning