DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?

Yang Chen; Bin Wen; Danyang Peng; Hong-Jie You; Zi-Qiao Shang; Lan-Zhe Guo

cs.AINov 3, 2026

DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?

Authors: Yang Chen, Bin Wen, Danyang Peng, Hong-Jie You, Zi-Qiao Shang, Lan-Zhe Guo

Organizations: National Key Laboratory for Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · Shanghai Innovation Institute · School of Artificial Intelligence, Nanjing University

Abstract

Recent multimodal agents have shown promising progress in embodied planning and web interaction. However, existing evaluations often rely on simplified tasks with limited combinatorial structure, leaving it unclear whether current agents can robustly integrate perception and reasoning for multi-goal planning under temporal, physical, and multimodal constraints. We introduce DualWorldBench, a unified benchmark for evaluating agent planning under controlled symbolic--temporal and grounded--topological complexity. Built around a shared pickup--delivery interface, DualWorldBench consists of three complementary components: TextWorld, which isolates symbolic planning under varying task compositions and release dynamics; SimWorld, which evaluates grounded path planning under controlled topological variation with text, image, or multimodal inputs; and DualWorld-VQA, a diagnostic suite spanning perception, scene understanding, and planning. Experiments reveal a common gap between local understanding and globally consistent planning. In TextWorld, agents struggle to derive optimal task sequences under interacting temporal constraints; in SimWorld, grounded observations are not reliably converted into feasible action sequences. DualWorld-VQA further confirms that perception and topology understanding do not necessarily translate into constrained planning. Together, these results identify the central challenge as composing symbolic or grounded evidence into globally feasible and efficient multi-goal plans. These findings establish DualWorldBench as a challenging testbed for agents with tighter perception--reasoning--planning integration. All datasets and code are publicly available at our project website https://dualworld.netlify.app/.

Figures & tables

Figure 1: DualWorldBench task overview. The agent maps task specifications and a grounded scene into a topology-aware pickup–delivery action plan under combinatorial constraints.

Figure 2: DualWorld framework. A unified pickup–delivery benchmark with TextWorld for symbolic planning, SimWorld for grounded planning, and DualWorld-VQA for diagnostic evaluation.

Open-source LLMs
Method	Rank	Average		Task Count
		Average		5		8		10		12		15
		R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$	R $\uparrow$	T $\downarrow$
Qwen3.6-27B	9	251.41	5061.46	99.33	1969.57	213.17	5552.39	249.12	5377.14	312.94	5158.18	382.49	7250.02
Llama-2-13B	12	212.33	5515.20	100.28	2396.66	209.73	5471.45	202.32	5403.88	298.18	5912.93	251.15	8391.05
Qwen3-VL-8B	13	68.81	3305.46	19.23	1449.30	121.67	4481.61	35.69	2902.33	135.69	4653.41	31.79	3040.67

Table 1: TextWorld results under varying task counts. Gray shading indicates the best performance.

Panel of Figure 3 — Figure 3: Performance comparison across TextWorld and SimWorld. In TextWorld, reward and completion time are normalized as $R/\max R$ and $\min T/T$ . In SimWorld, scores are averaged across modalities, with $NC$ inversely normalized within each scene.

Method	Modality	Average			Task Count
		Average			5			8			10
		F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$
Qwen3-VL-8B	Multi.	0.989	0.974	1.90	0.984	0.946	1.56	0.984	0.976	1.95	1.000	1.000	2.20
	Image	0.836	0.904	1.90	0.778	0.895	1.58	0.857	0.907	1.95	0.873	0.911	2.17
	Text	1.000	0.544	1.46	1.000	0.527	1.28	1.000	0.619	1.56	1.000	0.486	1.53
Qwen3.7-Plus	Multi.	1.000	1.000	1.27	1.000	1.000	1.13	1.000	1.000	1.30	1.000	1.000	1.38

Table 2: SimWorld scaling results under varying task counts.

Panel of Figure 4 — Figure 4: Failure attribution across the two worlds. (a) TextWorld episode reward and loss composition. (b) SimWorld root-cause shares pooled over the three input modalities for each model.

Figure 5: Evaluation in DualWorld-VQA from three levels: Visual Perception, Spatial Topology, and Logical Planning.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Representative Gazebo BEV previews used in SimWorld.

Panel of Figure 7 — Figure 7: T1 procedural SimWorld layouts. These scenes contain structured corridor systems where local visual proximity often remains a useful but imperfect heuristic.

Panel of Figure 8 — Figure 8: T2 procedural SimWorld layouts. These scenes increase topological ambiguity through loops, disconnected-looking regions, and longer detours, making straight-line proximity less reliable.

Method	Modality	Open			T1			T2			Indoor			Campus
Method	Modality	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$	F $\uparrow$	C $\uparrow$	NC $\downarrow$
Qwen3-VL-8B	Multi.	1.000	0.783	1.66	0.973	0.979	1.91	1.000	0.984	1.91	1.000	1.000	1.73	1.000	0.987	2.16
	Image	0.889	0.950	1.92	0.880	0.915	1.91	0.773	0.885	1.91	0.733	0.905	1.76	1.000	0.920	1.97
	Text	1.000	0.853	1.83	1.000	0.462	1.37	1.000	0.533	1.43	1.000	0.808	1.64	1.000	0.557	1.60
Qwen3.7-Plus	Multi.	1.000	1.000	1.24	1.000	1.000	1.23	1.000	1.000	1.30	1.000	1.000	1.29	1.000	1.000	1.29
	Image	0.667	0.894	1.20	1.000	1.000	1.57	1.000	1.000	1.57	1.000	1.000	1.35	1.000	1.000	1.60

Appendix

Table 4: SimWorld scene-wise results across five scene groups: open, T1, T2, indoor, and campus.

Model	Modality	Samples	SR $\uparrow$	CR $\uparrow$	Viol. $\downarrow$
Claude Haiku 4.5	Image	189	0.122	0.707	25.090
Claude Haiku 4.5	Multimodal	189	0.751	0.961	6.492
Claude Haiku 4.5	Text	189	0.831	0.981	3.476
Claude Opus 5	Image	189	0.561	0.865	3.889
Claude Opus 5	Multimodal	189	0.513	0.945	8.788
Claude Opus 5	Text	189	0.698	0.971	3.344

Appendix

Table 5: Aggregate SimWorld performance by model and modality. SR: success rate; CR: average completion rate; Viol.: average violations per sample.

Modality	Samples	SR $\uparrow$	CR $\uparrow$	Viol. $\downarrow$
Image	1134	0.554	0.860	5.822
Multimodal	1134	0.800	0.969	2.786
Text	1134	0.784	0.905	1.148

Appendix

Table 6: Sample-weighted SimWorld summary by input modality.

Modality	Model	Prec. $\downarrow$	State $\downarrow$	Cap. $\downarrow$	Gnd. $\downarrow$	Incomp. $\downarrow$	Succ. $\uparrow$
Text-only	GPT-5.6-Sol	3.2	0.0	0.0	0.0	0.5	96.3
	GPT-5.4-mini	0.0	0.5	0.0	0.0	5.3	94.2
	Claude-Opus-5	16.4	13.2	0.5	0.0	0.0	69.8
	Claude-Haiku-4.5	5.8	3.2	0.0	0.5	7.4	83.1
	Qwen3.7-Plus	0.0	0.0	0.0	0.0	5.3	94.7
	Qwen3-VL-8B	0.0	0.0	0.0	0.0	67.7	32.3

Appendix

Table 7: SimWorld failure attribution by model and input modality (% of runs,

N=189

per cell). Categories follow Section 5.3 : Prec. (delivery before pickup), State (duplicate pickup/delivery), Cap. (capacity exceeded), Gnd. (node outside graph or no parseable plan), Incomp. (legal plan, tasks remaining), Succ. (legal and complete). Each run is assigned a single root cause under the severity order Gnd.

>

Cap.

>

State

>

Prec., so rows sum to

100\%

; columns are ordered for readability, not by severity.

Model	Avg. $R$ $\uparrow$	Earned $\uparrow$	Not deliv. $\downarrow$	Late $\downarrow$	Late L2 $\downarrow$
Gemini-2.5-Flash	265.73	93.4	0.0	6.6	13.5
Gemini-3-Pro	264.36	92.9	0.0	7.1	14.5
GPT-5.6-Sol	264.33	92.9	0.0	7.1	14.5
Claude-Opus-5	264.10	92.8	0.0	7.2	14.7
DeepSeek-v4-Flash	263.60	92.6	2.2	5.1	10.5
Qwen3.7-Plus	263.24	92.5	0.0	7.5	15.3

Appendix

Table 8: TextWorld reward-gap attribution (% of attainable reward

\sum_{i}\rho_{i}

), pooled over

4

scenarios

\times

5

task counts at seed

1

. Earned

+

Not delivered

+

Late

+

Priority

=100\%

by construction. Late (L1) / Late (L2) decompose the lateness term by deadline regime; L1 scenarios have no deadlines. Runs voided by a priority violation are counted in the last column.

Question family	Level	Count	Primary metric
Absolute axis sorting	1	185	Exact match, LCS
Euclidean anchor choice	1	185	Exact match
Euclidean pair choice	1	185	Exact match
Wall-illusion topology	2	555	Exact match
Capacity interleaving	3	555	Distance tolerance
Total	–	1665	–

Appendix

Table 9: DualWorld-VQA dataset composition.

Model	Overall $\uparrow$	Axis $\uparrow$	Anchor $\uparrow$	Pair $\uparrow$	Topology $\uparrow$	Planning $\uparrow$
Qwen3-VL-8B	0.335	0.162	0.622	0.535	0.566	0.000
Qwen3.7-Plus	0.843	1.000	1.000	1.000	0.793	0.737
Claude Haiku 4.5	0.227	0.005	0.232	0.162	0.530	0.018
Claude Opus 5	0.799	1.000	1.000	0.995	0.778	0.622
GPT-5.4 Mini	0.392	0.422	0.589	0.654	0.559	0.063
GPT-5.6-Sol	0.864	0.968	0.989	0.995	0.775	0.832

Appendix

Table 10: DualWorld-VQA accuracy by question family.

Panel of Figure 9 — Figure 9: Annotated BEV observations used in DualWorld-VQA. Red markers indicate pickup nodes, blue markers indicate delivery nodes, and the green marker indicates the start position.

Solver	Task visibility	Optimality	Primary role
DFS	Currently visible	Anytime heuristic	Real-time feasible planning
Online DP	Currently visible	Exact over visible tasks	Strong online baseline
Offline DP	Full task set	Exact with hindsight	Upper bound and label generation

Appendix

Table 11: Roles of deterministic solvers used in the benchmark.

Setting	Inputs	Planning signal	Output
TextWorld	State, task list	Symbolic reward, time, deadline	JSON actions
SimWorld-text	State, task list, node coordinates	Coordinate proxy	JSON node sequence
SimWorld-image	BEV image	Visual topology	JSON node sequence
SimWorld-multimodal	BEV image, task list, node mapping	Visual topology and task grounding	JSON node sequence

Appendix

Table 12: Prompt inputs and output interfaces.

Temporal Planning

See all 8 papers

AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints
Jun 4, 2026Jiayu Liu, Cheng Qian, Zhenhailong Wang +10LLM Planning LLM Agent Evaluation
University of Illinois Urbana-Champaign
VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora
May 27, 2026Yuting Xu, Jiayi Tian, Jian Liang +4Multimodal Reasoning AI Agent Benchmarks
Institute of Information Engineering, CAS · School of Cyber Security, UCAS · Amap, Alibaba Group +2
RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments
Oct 7, 2026Zhiqin Yang, Chenxin Li, Xiaomeng Hu +30Robot Manipulation Benchmarks Multimodal Agents

Jun 4, 2026J/K move · Enter open · S save

AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints

Jiayu Liu, Cheng Qian, Zhenhailong Wang, Bingxuan Li+9

University of Illinois Urbana-Champaign

LLM Planning LLM Agent Evaluation AI Agent Benchmarks LLM Agent Planning

Open paper AskPDF arXiv

Planning for real-world problems by language models often involves both world and user constraints, which may not be fully specified upfront and are progressively disclosed through interaction. However, existing benchmarks still underexplore adaptive planning under such progressively revealed dual constraints. To address this gap, we introduce AdaPlanBench, a dynamic interactive benchmark for evaluating whether Large Language Model (LLM) agents can adaptively plan and re-plan under progressively revealed world and user constraints. AdaPlanBench is built on 307 household tasks, with a scalable constraint construction pipeline that augments each task with dual constraints. At runtime, agents interact with the environment in a multi-turn protocol where hidden constraints are revealed only when the agent proposes a plan that violates them, requiring iterative plan revision under accumulating feedback. This makes planning challenging, as agents must infer and track constraints from feedback while re-planning effectively. Experiments on ten leading LLMs show that adaptive planning under dual constraints remains challenging, with the best model reaching only 67.75% accuracy. We further observe that performance degrades as more constraints accumulate, with user constraints posing a particularly large challenge and failures often stemming from weaker physical grounding and reduced effectiveness. These results establish AdaPlanBench as a testbed for dual-constrained interactive planning and highlight the challenge of reliable adaptation to dynamically revealed constraints in LLM agents.

Abstract

Temporal Planning

Explore similar work

AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraints

VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments