Ad Hoc Teamwork

Follow this topic Ask about this topic

Ad hoc teamwork asks how an agent can cooperate with partners whose policies, goals, or capabilities it does not already know, a problem that arises in human-AI teams and mixed-agent systems. Common approaches model teammates’ actions or capabilities, train against diverse or adaptively generated partners, and use hierarchical skills or policies that adjust as interaction unfolds. Methods are often tested in cooperative environments such as Overcooked, Level-Based Foraging, and Hanabi, with some evaluations involving human partners. A central challenge is generalizing to unseen teammates under partial observability: adaptive training and partner-aware methods can help, but results vary by task, and history-based in-context learning does not reliably solve the problem.

Latest papers

18 papers1 paper in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Centralized Critics for N-Agent Ad Hoc Teamwork
Oct 26, 2026Andrew McAlinden, Caroline Wang, Steven Shi +3Ad Hoc Teamwork Multi-Agent Coordination
ConventionPlay: Capability-Limited Training for Robust Ad-Hoc Collaboration
Oct 8, 2026Abhishek Sriraman, Eleni Vasilaki, Robert LoftinMulti-Agent Collaboration Ad Hoc Teamwork
Department of Computer Science Sheffield, UK · Pittsburgh, PA 15213, USA
JaxAHT: A JAX-Based Library for Ad Hoc Teamwork
Sep 12, 2026Caroline Wang, Rolando Fernandez, Zelal Su Mustafaoglu +9Ad Hoc Teamwork Multi-Agent Coordination
Department of Computer Science, The University of Texas at Austin · Google · Sony AI
Distributed Team Orchestration via Supervisor Networks: Convergence, Optimality, and Resilience
Aug 10, 2026Juntian Zhu, Guanpu Chen, Tongtian Zhu +3Multi-Agent Orchestration Zero-Sum Games
School of Artificial Intelligence and Data Science, University of Science and Technology of China, Hefei 230026, China · School of Automation, Southeast University, Nanjing 210096, China · College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China +2
Partner Capability Estimation for Task-Agnostic Adaptation in Ad-Hoc Teamwork
Jul 29, 2026Peter Tisnikar, Maja Swieczkowska, Benteng Ma +2Bayesian Inference Ad Hoc Teamwork
aDepartment of Informatics, King’s College London, 30 Aldwych, London, WC2B 4BG, England, United Kingdom · bDepartment of Engineering Science, University of Oxford, Parks Road, Oxford, OX1 3PJ, England, United Kingdom
What Types of Human-AI Teams Exist?
Jul 2, 2026Nathan Hughes, Ibrahim HabliAd Hoc Teamwork Human-AI Collaboration
Centre for Assuring Autonomy, University of York, Deramore Lane, York, North Yorkshire, YO10 5DD, UK.
CONCAT: Consensus- and Confidence-Driven Ad Hoc Teaming for Efficient LLM-Based Multi-Agent Systems
May 28, 2026Ziyang Ma, Dingyi Zhang, Sichu Liang +4Multi-Agent System Optimization Multi-Agent Orchestration
Southeast University · Huawei Technologies Ltd
Benchmarking the Limits of In-Context Reinforcement Learning for Ad-Hoc Teamwork
May 23, 2026Yuheng Jing, Kai Li, Ziwen Zhang +8RL Benchmarks Multi-Agent Coordination
C2DL, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · University of Science and Technology of China +5
Partner-Aware Hierarchical Skill Discovery for Robust Human-AI Collaboration
May 23, 2026Adnan Ahmad, Bahareh Nakisa, Mohammad Naim RastgooHierarchical RL Multi-Agent Collaboration
aSchoold of Information Technology, Deakin University, 75 Pidgons Rd, Waurn Ponds, 3216, VIC, Australia · bFaculty of Information Technology, Monash University, Wellington Rd, Clayton, 3800, VIC, Australia
Adaptive Human-AI Coordination via Hierarchical Action Disentanglement
May 23, 2026Adnan Ahmad, Bahareh Nakisa, Mohammad Naim RastgooDisentangled Representation Learning Hierarchical RL
School of Information Technology Deakin University, Waurn Ponds, 3216, VIC, Australia · Faculty of Information Technology, Monash University, 3800, VIC, Australia
Too Many Specialists: Emergent Inefficiencies and Bottlenecks for Multi-agent Ad-hoc Collaboration
May 8, 2026Benjamin Panny, Shashank Mehrotra, Zahra Zahedi +2Multi-Agent Coordination Multi-Agent Collaboration
University of Pittsburgh Pittsburgh, PA, United States · Honda Research Institute USA, Inc. San Jose, CA, United States
ConventionPlay: Capability-Limited Training for Robust Ad-Hoc Collaboration
Apr 20, 2026Abhishek Sriraman, Eleni Vasilaki, Robert LoftinMulti-Agent Coordination Multi-Agent Collaboration
University of Sheffield · Sheffield, United Kingdom
Learning When to Cooperate Under Heterogeneous Goals
Mar 7, 2026Max Taylor-Davies, Neil Bramley, Christopher G. LucasMulti-Agent Collaboration Ad Hoc Teamwork
School of Informatics, The University of Edinburgh · Department of Psychology, The University of Edinburgh
NestRL: A Nested Training Regime for Mutual Adaptation in Human-AI Teaming
Feb 18, 2026Upasana Biswas, Durgesh Kalwar, Subbarao Kambhampati +1Human-AI Collaboration Multi-Agent Coordination
School of Computing and AI, Arizona State University, Tempe, AZ, USA · Department of Computer Science, Colorado State University, Fort Collins, CO, USA
PADiff: Predictive and Adaptive Diffusion Policies for Ad Hoc Teamwork
Nov 10, 2025Hohei Chan, Xinzhi Zhang, Antao Xiang +2Multi-Agent Coordination Multi-Agent Collaboration
School of Software Engineering, South China University of Technology, Guangzhou, China · Shanghai Jiao Tong University, Shanghai, China
Unsupervised Partner Design Enables Robust Ad-hoc Teamwork
Aug 8, 2025Constantin Ruhdorfer, Matteo Bortoletto, Victor Oei +2Multi-Agent Collaboration Ad Hoc Teamwork
Collaborative Artificial Intelligence, University of Stuttgart, Stuttgart, Germany.
ROTATE: Regret-driven Open-ended Training for Ad Hoc Teamwork
May 29, 2025Caroline Wang, Arrasy Rahman, Benjamin Nativi +4Multi-Agent Collaboration Ad Hoc Teamwork
Department of Computer Science The University of Texas at Austin · Visa AI · College of Computing and Data Science Nanyang Technological University +1
A Generalist Hanabi Agent
Mar 17, 2025Arjun V Sudhakar, Hadi Nekoei, Mathieu Reymond +3RL for Game AI Cooperative MARL
Chandar Research Lab · Mila – Quebec AI Institute · Polytechnique Montréal +4

Oct 26, 2026J/K move · Enter open · S save

Centralized Critics for N-Agent Ad Hoc Teamwork

Andrew McAlinden, Caroline Wang, Steven Shi, Arrasy Rahman+2

Ad Hoc Teamwork Multi-Agent Coordination Cooperative MARL Centralized Training with Decentralized Execution

Open paper AskPDF arXiv

Centralized training with decentralized execution (CTDE) has become the dominant paradigm in multi-agent reinforcement learning (MARL), motivating extensive study of what privileged information should be made available to the critic during training. While prior work has examined the role of global state in fully cooperative settings, the design of centralized critics for N-agent ad hoc teamwork - in which dynamically varying set of learning agents must coordinate with an arbitrary set of uncontrolled teammates drawn from an unknown distribution - remains poorly understood. We study this question by comparing two natural classes of advantage estimators: controlled history estimators, which condition only on the observation and action histories of the learner's own agents, and joint history estimators, which additionally condition on the histories of the uncontrolled teammates. We prove that both estimators are unbiased and characterize the conditions under which the joint history estimate attains strictly lower variance than its controlled counterpart. Empirically, joint history critics match or outperform controlled history critics across a range of NAHT tasks, yielding improved sample efficiency and final returns. Together, these results provide both a theoretical justification and an empirical recommendation for incorporating uncontrolled agent histories into centralized critics for N-agent ad hoc teamwork.