DuoCook: An Overcooked-Inspired Benchmark for Visual Planning and Coordination

Ziyi Jiao; Heng Zhang; Drishti Regmi; Xiaokang Ye; Lianhui Qin; Peter Yichen Chen

cs.AIOct 22, 2026

DuoCook: An Overcooked-Inspired Benchmark for Visual Planning and Coordination

Authors: Ziyi Jiao, Heng Zhang, Drishti Regmi, Xiaokang Ye, Lianhui Qin, Peter Yichen Chen

Organizations: Department of Computer Science, University of British Columbia · University of California, San Diego

Abstract

Coordinating two players in a cooking task requires a vision-language agent to track changing scene states, sequence recipe steps, and allocate shared resources under a time budget. We introduce an Overcooked-inspired benchmark for evaluating these capabilities in Unreal Engine environments constructed using coding agents. A single vision-language model controls both players from top-down visual observations, selecting high-level actions while the engine executes navigation. The benchmark contains 20 maps in five categories. These categories support evaluation of centralized coordination under equipment contention, changing terrain, and handoff dependencies. A prop legend and pot ingredient-count indicators provide explicit visual cues, while grid-based destinations support spatial grounding. The evaluation compares four VLMs on all 20 maps, measuring task score, order completion, delivery precision, action acceptance rate, and model API expenditure. A memory ablation on the same 20 maps examines the effect of a shared, self-authored notebook while retaining six-response action history in both conditions. The strongest evaluated model, gpt-6-astra, achieves 26.2% mean order completion. Notebook memory has mixed score effects across models, with positive mean differences for Claude and Qwen and negative differences for GPT and Gemini.

Figures & tables

Figure 1: Benchmark pipeline. Left: coding-agent kitchen construction, manual refinement, and actor-contract verification. Center: joint control of two players from shared multimodal inputs, illustrated by onion-soup preparation. Right: timed-order scoring and evaluation of score, order completion, delivery precision, action acceptance, and API expenditure.

Zone	Elapsed waiting time $w$ (steps)	Base tip
Green	$0\leq w\leq 1{,}350$	8
Yellow	$1{,}350<w\leq 2{,}100$	5
Red	$2{,}100<w\leq 2{,}550$	3
Last call	$2{,}550<w<3{,}000$	0

Table 1: Base tips by elapsed waiting time for a non-expired order. The thresholds correspond to 45%, 70%, and 85% of the 3,000-step waiting limit.

Figure 2: Input components retained in both memory conditions. Left: instructions, live-state JSON, legal operations, six-response action history, and execution feedback. Right: a top-down kitchen image with grid labels and pot indicators, plus a prop legend. The additional notebook used in the memory-enabled condition is not shown.

Controller	Score $\uparrow$	Completion (%) $\uparrow$	Precision (%) $\uparrow$	Acceptance (%) $\uparrow$	USD/episode $\downarrow$
qwen3.8-27b	9.1	8.8	91.7 ( $n=6$ )	39.5	0.27
gemini-3.5-flash-lite	11.9	11.2	81.0 ( $n=7$ )	32.1	0.30
claude-opus-5	8.2	8.8	100.0 ( $n=5$ )	52.7	4.61
gpt-6-astra	25.6	26.2	93.8 ( $n=16$ )	64.6	5.18

Table 2: Main benchmark performance, averaged with equal weight across 20 maps, with one episode per model and map. Arrows give the preferred direction. Delivery precision excludes maps with no deliveries;

n

gives the number of maps entering that mean. Bold values mark the best reported mean in each column.

Figure 3: Mean order completion by map category. Each column averages the four maps in that category with equal weight.

Figure 4: Effect of shared memory, as paired differences (with memory minus without) on the 20 ablation maps. (a) Score difference: grey points are per-map differences, the marker is the equal-weight mean with a 95% percentile bootstrap interval over resampled maps. (b) API cost per episode under each condition, log scale. Intervals resample maps rather than repeated runs, since each model-map-condition result is a single episode; they summarize sensitivity of the mean to resampling the observed maps, not the distribution of individual map effects or run-to-run variability. The constructed maps are not a random sample from a specified population, so these intervals do not establish generalization beyond this suite.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Map category	Source level	Main	Memory
Basic	1-1	Yes	Yes
	1-3	Yes	Yes
	1-4	Yes	Yes
	1-5	Yes	Yes
Shared equipment	3-1	Yes	Yes
	6-4	Yes	Yes

Appendix

Table 3: Source-level correspondence and evaluation membership. Main denotes the full 20-map main evaluation; Memory denotes the 20-map memory ablation.

Multi-Agent Coordination

Multi-agent coordination studies how autonomous agents align actions, share information, and divide work to achieve individual or collective goals.

See all 520 papers

CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning
Aug 28, 2026Yang Chen, Ye-Xin Xie, Lirong Che +8Multi-Agent Coordination AI Agent Benchmarks
Nanjing University · AgiBot · Tsinghua University
MATEO: A Multimodal Benchmark for Temporal Reasoning and Planning in LVLMs
Feb 16, 2026Gabriel Roccabruna, Olha Khomyn, Giuseppe RiccardiTemporal Reasoning in VLMs VLM Reasoning
Signals and Interactive Systems Lab, University of Trento, Italy
vla-eval: A Unified Evaluation Harness for Vision-Language-Action Models
Mar 14, 2026Suhwan Choi, Yunsung Lee, Yubeen Park +4VLM Evaluation Vision-Language-Action Models
Allen Institute for AI (AI2) · Seoul National University

Aug 28, 2026J/K move · Enter open · S save

CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning

Yang Chen, Ye-Xin Xie, Lirong Che, Danyang Peng+7

Nanjing University · AgiBot · Tsinghua University

Multi-Agent Coordination AI Agent Benchmarks Embodied Agent Benchmarks Multi-Agent Planning Multi-Robot Task Allocation

Open paper AskPDF arXiv

Agent systems powered by multimodal large language models (MLLMs) have advanced rapidly in recent years, yet existing embodied-agent benchmarks still lack fine-grained diagnostics for multi-agent coordination. Most benchmarks either focus on single-agent task completion or summarize multi-agent behavior with overall task success rates, which can obscure coordination failures such as duplicated work, violations of ordering constraints, resource contention, and desynchronized handoffs. In this paper, we introduce CoCoBench, a construct-level benchmark for evaluating multi-agent embodied coordination in executable household tasks. CoCoBench contains 897 oracle-validated instances organized around four recurring coordination constructs: task allocation, sequential ordering, mutual exclusion, and handoff coordination. In addition to task success rate, CoCoBench provides construct-level scores that measure whether agents coordinate effectively. We evaluate 11 leading MLLMs across different coordination modes, observation inputs, and numbers of agents. The results show that coordination ability is highly construct-specific: strong overall performance does not imply balanced competence across different coordination types. These findings point to new directions for designing targeted model architectures and improving multi-agent coordination ability.

Abstract

Multi-Agent Coordination

Explore similar work

CoCoBench: A Cooperative Coordination Benchmark for Embodied Multi-Agent Task Planning

MATEO: A Multimodal Benchmark for Temporal Reasoning and Planning in LVLMs

vla-eval: A Unified Evaluation Harness for Vision-Language-Action Models