DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?

Yang Chen; Bin Wen; Danyang Peng; Hong-Jie You; Zi-Qiao Shang; Lan-Zhe Guo

cs.AINov 3, 2026

DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?

Authors: Yang Chen, Bin Wen, Danyang Peng, Hong-Jie You, Zi-Qiao Shang, Lan-Zhe Guo

Organizations: National Key Laboratory for Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · Shanghai Innovation Institute · School of Artificial Intelligence, Nanjing University

Abstract

Recent multimodal agents have shown promising progress in embodied planning and web interaction. However, existing evaluations often rely on simplified tasks with limited combinatorial structure, leaving it unclear whether current agents can robustly integrate perception and reasoning for multi-goal planning under temporal, physical, and multimodal constraints. We introduce DualWorldBench, a unified benchmark for evaluating agent planning under controlled symbolic--temporal and grounded--topological complexity. Built around a shared pickup--delivery interface, DualWorldBench consists of three complementary components: TextWorld, which isolates symbolic planning under varying task compositions and release dynamics; SimWorld, which evaluates grounded path planning under controlled topological variation with text, image, or multimodal inputs; and DualWorld-VQA, a diagnostic suite spanning perception, scene understanding, and planning. Experiments reveal a common gap between local understanding and globally consistent planning. In TextWorld, agents struggle to derive optimal task sequences under interacting temporal constraints; in SimWorld, grounded observations are not reliably converted into feasible action sequences. DualWorld-VQA further confirms that perception and topology understanding do not necessarily translate into constrained planning. Together, these results identify the central challenge as composing symbolic or grounded evidence into globally feasible and efficient multi-goal plans. These findings establish DualWorldBench as a challenging testbed for agents with tighter perception--reasoning--planning integration. All datasets and code are publicly available at our project website https://dualworld.netlify.app/.

Figures & tables

Figure 1: DualWorldBench task overview. The agent maps task specifications and a grounded scene into a topology-aware pickup–delivery action plan under combinatorial constraints.

Figure 2: DualWorld framework. A unified pickup–delivery benchmark with TextWorld for symbolic planning, SimWorld for grounded planning, and DualWorld-VQA for diagnostic evaluation.

Open-source LLMs
Method	Rank	Average		Task Count
		Average		5		8		10		12		15
		R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$
Qwen3.6-27B	9	251.41	5061.46	99.33	1969.57	213.17	5552.39	249.12	5377.14	312.94	5158.18	382.49	7250.02
Llama-2-13B	12	212.33	5515.20	100.28	2396.66	209.73	5471.45	202.32	5403.88	298.18	5912.93	251.15	8391.05
Qwen3-VL-8B	13	68.81	3305.46	19.23	1449.30	121.67	4481.61	35.69	2902.33	135.69	4653.41	31.79	3040.67

Table 1: TextWorld results under varying task counts. Gray shading indicates the best performance.

Panel of Figure 3 — Figure 3: Performance comparison across TextWorld and SimWorld. In TextWorld, reward and completion time are normalized as $R/\max R$ and $\min T/T$ . In SimWorld, scores are averaged across modalities, with $NC$ inversely normalized within each scene.

Method	Modality	Average			Task Count
		Average			5			8			10
		F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$
Qwen3-VL-8B	Multi.	0.989	0.974	1.90	0.984	0.946	1.56	0.984	0.976	1.95	1.000	1.000	2.20
	Image	0.836	0.904	1.90	0.778	0.895	1.58	0.857	0.907	1.95	0.873	0.911	2.17
	Text	1.000	0.544	1.46	1.000	0.527	1.28	1.000	0.619	1.56	1.000	0.486	1.53
Qwen3.7-Plus	Multi.	1.000	1.000	1.27	1.000	1.000	1.13	1.000	1.000	1.30	1.000	1.000	1.38

Table 2: SimWorld scaling results under varying task counts.

Panel of Figure 4 — Figure 4: Failure attribution across the two worlds. (a) TextWorld episode reward and loss composition. (b) SimWorld root-cause shares pooled over the three input modalities for each model.

Figure 5: Evaluation in DualWorld-VQA from three levels: Visual Perception, Spatial Topology, and Logical Planning.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Representative Gazebo BEV previews used in SimWorld.

Panel of Figure 7 — Figure 7: T1 procedural SimWorld layouts. These scenes contain structured corridor systems where local visual proximity often remains a useful but imperfect heuristic.

Panel of Figure 8 — Figure 8: T2 procedural SimWorld layouts. These scenes increase topological ambiguity through loops, disconnected-looking regions, and longer detours, making straight-line proximity less reliable.

Method	Modality	Open			T1			T2			Indoor			Campus
Method	Modality	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$
Qwen3-VL-8B	Multi.	1.000	0.783	1.66	0.973	0.979	1.91	1.000	0.984	1.91	1.000	1.000	1.73	1.000	0.987	2.16
	Image	0.889	0.950	1.92	0.880	0.915	1.91	0.773	0.885	1.91	0.733	0.905	1.76	1.000	0.920	1.97
	Text	1.000	0.853	1.83	1.000	0.462	1.37	1.000	0.533	1.43	1.000	0.808	1.64	1.000	0.557	1.60
Qwen3.7-Plus	Multi.	1.000	1.000	1.24	1.000	1.000	1.23	1.000	1.000	1.30	1.000	1.000	1.29	1.000	1.000	1.29
	Image	0.667	0.894	1.20	1.000	1.000	1.57	1.000	1.000	1.57	1.000	1.000	1.35	1.000	1.000	1.60

Appendix

Table 4: SimWorld scene-wise results across five scene groups: open, T1, T2, indoor, and campus.

Model	Modality	Samples	SR $\uparrow$	CR $\uparrow$	Viol. $\downarrow$
Claude Haiku 4.5	Image	189	0.122	0.707	25.090
Claude Haiku 4.5	Multimodal	189	0.751	0.961	6.492
Claude Haiku 4.5	Text	189	0.831	0.981	3.476
Claude Opus 5	Image	189	0.561	0.865	3.889
Claude Opus 5	Multimodal	189	0.513	0.945	8.788
Claude Opus 5	Text	189	0.698	0.971	3.344

Appendix

Table 5: Aggregate SimWorld performance by model and modality. SR: success rate; CR: average completion rate; Viol.: average violations per sample.

Modality	Samples	SR $\uparrow$	CR $\uparrow$	Viol. $\downarrow$
Image	1134	0.554	0.860	5.822
Multimodal	1134	0.800	0.969	2.786
Text	1134	0.784	0.905	1.148

Appendix

Table 6: Sample-weighted SimWorld summary by input modality.

Modality	Model	Prec. $\downarrow$	State $\downarrow$	Cap. $\downarrow$	Gnd. $\downarrow$	Incomp. $\downarrow$	Succ. $\uparrow$
Text-only	GPT-5.6-Sol	3.2	0.0	0.0	0.0	0.5	96.3
	GPT-5.4-mini	0.0	0.5	0.0	0.0	5.3	94.2
	Claude-Opus-5	16.4	13.2	0.5	0.0	0.0	69.8
	Claude-Haiku-4.5	5.8	3.2	0.0	0.5	7.4	83.1
	Qwen3.7-Plus	0.0	0.0	0.0	0.0	5.3	94.7
	Qwen3-VL-8B	0.0	0.0	0.0	0.0	67.7	32.3

Appendix

Table 7: SimWorld failure attribution by model and input modality (% of runs,

N=189

per cell). Categories follow Section 5.3 : Prec. (delivery before pickup), State (duplicate pickup/delivery), Cap. (capacity exceeded), Gnd. (node outside graph or no parseable plan), Incomp. (legal plan, tasks remaining), Succ. (legal and complete). Each run is assigned a single root cause under the severity order Gnd.

>

Cap.

>

State

>

Prec., so rows sum to

100\%

; columns are ordered for readability, not by severity.

Model	Avg. $R$ $\uparrow$	Earned $\uparrow$	Not deliv. $\downarrow$	Late $\downarrow$	Late L2 $\downarrow$
Gemini-2.5-Flash	265.73	93.4	0.0	6.6	13.5
Gemini-3-Pro	264.36	92.9	0.0	7.1	14.5
GPT-5.6-Sol	264.33	92.9	0.0	7.1	14.5
Claude-Opus-5	264.10	92.8	0.0	7.2	14.7
DeepSeek-v4-Flash	263.60	92.6	2.2	5.1	10.5
Qwen3.7-Plus	263.24	92.5	0.0	7.5	15.3

Appendix

Table 8: TextWorld reward-gap attribution (% of attainable reward

\sum_{i}\rho_{i}

), pooled over

4

scenarios

\times

5

task counts at seed

1

. Earned

+

Not delivered

+

Late

+

Priority

=100\%

by construction. Late (L1) / Late (L2) decompose the lateness term by deadline regime; L1 scenarios have no deadlines. Runs voided by a priority violation are counted in the last column.

Question family	Level	Count	Primary metric
Absolute axis sorting	1	185	Exact match, LCS
Euclidean anchor choice	1	185	Exact match
Euclidean pair choice	1	185	Exact match
Wall-illusion topology	2	555	Exact match
Capacity interleaving	3	555	Distance tolerance
Total	–	1665	–

Appendix

Table 9: DualWorld-VQA dataset composition.

Model	Overall $\uparrow$	Axis $\uparrow$	Anchor $\uparrow$	Pair $\uparrow$	Topology $\uparrow$	Planning $\uparrow$
Qwen3-VL-8B	0.335	0.162	0.622	0.535	0.566	0.000
Qwen3.7-Plus	0.843	1.000	1.000	1.000	0.793	0.737
Claude Haiku 4.5	0.227	0.005	0.232	0.162	0.530	0.018
Claude Opus 5	0.799	1.000	1.000	0.995	0.778	0.622
GPT-5.4 Mini	0.392	0.422	0.589	0.654	0.559	0.063
GPT-5.6-Sol	0.864	0.968	0.989	0.995	0.775	0.832

Appendix

Table 10: DualWorld-VQA accuracy by question family.

Panel of Figure 9 — Figure 9: Annotated BEV observations used in DualWorld-VQA. Red markers indicate pickup nodes, blue markers indicate delivery nodes, and the green marker indicates the start position.

Solver	Task visibility	Optimality	Primary role
DFS	Currently visible	Anytime heuristic	Real-time feasible planning
Online DP	Currently visible	Exact over visible tasks	Strong online baseline
Offline DP	Full task set	Exact with hindsight	Upper bound and label generation

Appendix

Table 11: Roles of deterministic solvers used in the benchmark.

Setting	Inputs	Planning signal	Output
TextWorld	State, task list	Symbolic reward, time, deadline	JSON actions
SimWorld-text	State, task list, node coordinates	Coordinate proxy	JSON node sequence
SimWorld-image	BEV image	Visual topology	JSON node sequence
SimWorld-multimodal	BEV image, task list, node mapping	Visual topology and task grounding	JSON node sequence

Appendix

Table 12: Prompt inputs and output interfaces.

Temporal Planning

See all 8 papers

Jun 4, 2026

AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints

Planning for real-world problems by language models often involves both world and user constraints, which may not be fully specified upfront and are progressively disclosed through interaction. However, existing benchmarks still underexplore adaptive planning under such progressively revealed dual constraints. To address this gap, we introduce AdaPlanBench, a dynamic interactive benchmark for evaluating whether Large Language Model (LLM) agents can adaptively plan and re-plan under progressively revealed world and user constraints. AdaPlanBench is built on 307 household tasks, with a scalable constraint construction pipeline that augments each task with dual constraints. At runtime, agents interact with the environment in a multi-turn protocol where hidden constraints are revealed only when the agent proposes a plan that violates them, requiring iterative plan revision under accumulating feedback. This makes planning challenging, as agents must infer and track constraints from feedback while re-planning effectively. Experiments on ten leading LLMs show that adaptive planning under dual constraints remains challenging, with the best model reaching only 67.75% accuracy. We further observe that performance degrades as more constraints accumulate, with user constraints posing a particularly large challenge and failures often stemming from weaker physical grounding and reduced effectiveness. These results establish AdaPlanBench as a testbed for dual-constrained interactive planning and highlight the challenge of reliable adaptation to dynamically revealed constraints in LLM agents.

Jiayu Liu, Cheng Qian, Zhenhailong Wang +10

University of Illinois Urbana-Champaign

LLM Planning LLM Agent Evaluation AI Agent Benchmarks LLM Agent Planning

Ask PDF

May 27, 2026

VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

Existing benchmarks have laid the foundation for travel planning agents by establishing API-centric paradigms. However, as the capabilities of Autonomous Agents continue to advance, their evaluation must evolve beyond simple tool execution toward handling the inherent complexities of the open web. Current benchmarks bypass core cognitive hurdles: they fail to account for information noise, ignore multi-source factual contradictions, and overlook the necessity of grounding visual perception into logical planning. We introduce VeriTrip, a verifiable benchmark designed to meet the increasing demands for agent robustness and reliability. VeriTrip shifts the evaluation focus to evidence-grounded reasoning over unstructured multimodal web corpora. It establishes a Multimodal Retrieval Base (MRB) derived from real-world sources, forcing agents to autonomously orchestrate queries across heterogeneous data. A synchronized Verifiable Knowledge Base (VKB) enables a cell-wise verification protocol that precisely quantifies factual reliability, distinguishing systematic reasoning failures from parametric hallucinations. Our evaluations across leading MLLMs reveal a critical retrieval-reasoning trade-off: the cognitive load of autonomous retrieval significantly erodes instruction retention. VeriTrip provides the rigorous foundation necessary for the next generation of planning agents capable of operating in unconstrained, multimodal environments.

Yuting Xu, Jiayi Tian, Jian Liang +4

Institute of Information Engineering, CAS · School of Cyber Security, UCAS · Amap, Alibaba Group +2

Multimodal Reasoning AI Agent Benchmarks Multimodal Agents LLM Agent Reliability

Ask PDF

Oct 7, 2026

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments

General-purpose agents increasingly write code, use tools, and complete complex digital tasks, raising the question of how far these capabilities carry into the physical world. To investigate this, we introduce RobotWorld, a challenging simulation testbed for robot use: turning instructions and observations into physical task execution through robot interfaces. Its 84 tasks span manipulation, mobile manipulation, locomotion, driving, and aerial control, with explicit interaction budgets and executable success checks. By analysing task outcomes alongside execution traces, we identify both the capabilities that transfer and the gaps that prevent reliable completion. Furthermore, we find that current agents can construct sophisticated perception and control workflows, including image segmentation, camera calibration, spatial estimation, and dynamics-based computation. These capabilities, however, do not consistently compose into successful behaviour: agents lose task-relevant object states despite reaching commanded poses, fail to correct ineffective actions, recover too late, or mistake unfinished tasks for completion. This uneven transfer also differs across models: Astra succeeds more often on spatial and constrained-contact goals, whereas Opus 5.5 succeeds more often on continuous-balance and timed-interaction goals. By linking these outcomes to execution behaviour, RobotWorld provides both a rigorous proving ground and an empirical account of the remaining capability gaps, thereby establishing concrete targets for training and designing more reliable physical-world agents.

Zhiqin Yang, Chenxin Li, Xiaomeng Hu +30

Robot Manipulation Benchmarks Multimodal Agents Robotics Simulation

Ask PDF

Abstract

Temporal Planning

Explore similar work

AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints

VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments