ForagerSim: Evaluating Embodied Map-Based Reasoning in Large-Scale Dynamic Worlds

Tomáš Musil; Martin Kotík; Michal Vavrečka; Karla Štěpánová; Sebastian Scherer; Martin Saska

cs.ROOct 19, 2026

ForagerSim: Evaluating Embodied Map-Based Reasoning in Large-Scale Dynamic Worlds

Authors: Tomáš Musil, Martin Kotík, Michal Vavrečka, Karla Štěpánová, Sebastian Scherer, Martin Saska

Organizations: Dept. of Cybernetics, FEE, Czech Technical University in Prague · CIIRC, Czech Technical University in Prague · Robotics Institute, Carnegie Mellon University

Abstract

Animals build cognitive maps of vast environments, use them to perform map-based navigation, and might even be reusing the same circuitry used in spatial reasoning for creating and using maps of arbitrary task-spaces. In contrast, current embodied AI systems either rely on hand-engineered mapping pipelines, which break in the presence of unmodeled dynamics and are restricted to metric space, or on learned policies that rarely form map-like representations at all. Developing architectures that could learn to build and use map-like representations as flexibly as animals is, however, difficult because few embodied AI benchmarks actually require map-based reasoning. To address this, we present ForagerSim --- an interdisciplinary research platform and simulation environment focused on embodied tasks that require map-based reasoning, with scenarios ranging from mobile-robot search and rescue to animal foraging in both complex mazes and vast, potentially infinite open areas. ForagerSim is highly modular and allows modifying independent axes of difficulty related to perception (abstract 2D sensing to full RGB or LiDAR), scale (room-scale to multi-km scale), topology (open-fields to complex mazes), dynamics (smoke, debris, sensorimotor failures and others) and tasks (from simple 2D coverage to foraging-inspired well-alternation tasks). We compare reinforcement learning baselines with a simple engineered robotics stack and a human player on 8 example worlds which show how different difficulty axes affect existing methods. Our results show that, when sensing is simplified to 2D, learned policies can perform multi-object search and well-alternation tasks surprisingly well in simple topologies, but they fall short of human performance in larger or more complex-topology environments where map-based baselines perform well, suggesting that map-based reasoning is the bottleneck in these scenarios.

Figures & tables

Panel of Figure 1 — Figure 1: Example scenarios in ForagerSim - from left to right: The 4arm-submazes maze; multi-object search in a dynamic-environment consisting of houses with movable rubble and chaotically walking entities; ratmaze-foraging experiment with rewards alternating between random wells and a home well; massive-scale realistic bat-level foraging scenario

Panel of Figure 2 — Figure 2: Illustration of the agent’s observations. Left: top-down view of the environment and the raycasts. Right: human-UI visualization of the depth, semantics and compass data.

Figure 3: Bar charts of all the eval runs for each of the 8 scenarios described in subsection 4.1

World	PPO	Dreamer	Frontier-Explorer	Human
memorymaze15x15	$309_{\pm 39}$	$320_{\pm 26}$	$\mathbf{339}_{\pm 1}$	$\mathbf{339}_{\pm 1}$
10houses-fenced	$366_{\pm 303}$	$2292_{\pm 481}$	$1947_{\pm 389}$	$\mathbf{3452}_{\pm 134}$
4arm-submazes	$504_{\pm 210}$	$777_{\pm 160}$	$1868_{\pm 44}$	$\mathbf{1895}_{\pm 3}$
5houses-dynamic	$377_{\pm 282}$	$984_{\pm 120}$	$152_{\pm 0}$	$\mathbf{1114}_{\pm 100}$
5houses-sar	$142_{\pm 45}$	$864_{\pm 196}$	$118_{\pm 49}$	$\mathbf{957}_{\pm 99}$
ratmaze-foraging	$470_{\pm 525}$	$2883_{\pm 256}$	–	$\mathbf{3827}_{\pm 515}$

Table 2: Benchmark results on the 8 world configurations. All numbers show mean per-episode return and its standard deviation, computed from 5 eval seeds per scenario, averaged across training seeds

Figure 4: Adaptive difficulty curves for the curriculum experiment described in subsection 4.2 .

Figure 5: Visualization of eval trajectories on the 10houses-fenced and 4arm-submazes topologies. Color of trajectory indicates step index in episode, white circle = start, white square = end, stars = reward object pickups.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: A visualization of 2 random seed for each of the 8 worlds used in the comparisons in subsection 4.1

Figure 7: A visualization of adaptive difficulty scaling of world complexity on the example of the 4arm-submazes world in the experiments described in subsection 4.2

World	Method	Score	OBJ	Coverage [ $10^{2}$ m 2 ]
memorymaze15x15	PPO	$309_{\pm 39}$	$5.5_{\pm 0.7}$	$36.4_{\pm 2.7}$
	Dreamer	$320_{\pm 26}$	$5.7_{\pm 0.5}$	$35.9_{\pm 2.2}$
	Frontier-Explorer	$\mathbf{339}_{\pm 1}$	$6.0_{\pm 0.0}$	$39.2_{\pm 0.7}$
	Human	$\mathbf{339}_{\pm 1}$	$6.0_{\pm 0.0}$	$38.9_{\pm 0.7}$
10houses-fenced	PPO	$366_{\pm 303}$	$3.7_{\pm 3.6}$	$190.0_{\pm 159.8}$
	Dreamer	$2292_{\pm 481}$	$37.9_{\pm 8.4}$	$401.0_{\pm 92.6}$

Appendix

Table 3: Additional metrics for the evaluation described in subsection 4.1 and in Table 2 . OBJ signifies number of collected reward objects, coverage is the area explored by the agent’s sensors in the order of

10^{2}

, since this is equal to the contribution of the coverage to the overall score (since we give a reward of 0.01 for each covered square meter).

Figure 8: Full training curves of for each of the presented worlds. In all figures, the line represents mean return per episode averaged across training seeds. Top 2 rows show the standard deviation of the mean returns between individual training seeds. Bottom 2 rows show the deviation across episodes averaged over training seeds, which shows that the baseline performance varies drastically between individual generated worlds within any scenario, and corresponds to the high eval-world score deviation in the evaluations in subsection 4.1

Figure 9: Visualization of eval trajectories on the forest-foraging unbounded well-alternation scenario. Dreamer walks outside of the main area into the infinite forest in one world, PPO in all of them.

Figure 10: Visualization of eval trajectories on the 4arm-submaze-foraging well-alternation scenario. The human player can perform 3 or more random-home alternations while both PPO and Dreamer struggle with learning to return home after finding the cued random well.

Figure 11: Trajectory visualizations for the memorymaze15x15 world, showing that it can be easily fully covered in the multi-object search task by even a model-free, memoryless PPO agent. This suggests that with enough time (since the human and the frontier explorer were able to cover it sometimes in 50% of the episode steps), this environment is not challenging in terms of map-based reasoning or memory (although the original ordered -search task is, as the authors present, shown to be a good benchmark of memory).

Figure 12: ForagerSim environment architecture overview.

Panel of Figure 13 — Figure 13: Examples of the 3D environments available in ForagerSim in the RGB/LiDAR sensing modes, Top: a 3D variant of search-and-rescue missions in suburb-scale environments with interactible, randomized houses (with ability to swap abstract red boxes as reward objects for human models), Bottom: a recreation of the Hula valley on which the animal experiments documenting the scalability of biological cognitive maps of bats have been recorded Goldshtein et al. (2024)

Parameter	Value (default)
Model size	size12m : RSSM deter 2048, hidden 256, 16 classes, MLP units 256 ( size200m )
Batch size	8 (16)
Sequence length	48 (64)
Replay context	16 (1)
Replay buffer	$10^{6}$ steps ( $5\times 10^{6}$ )
Parallel envs	4 (16)

Appendix

Table 4: DreamerV3 hyperparameters that differ from the official implementation’s defaults (defaults in parentheses). PPO uses the Stable-Baselines3 defaults.

Embodied Navigation

Embodied navigation studies how robots perceive, map, and move through environments to reach goals and follow instructions.

See all 138 papers

LASAR: Towards Spatio-temporal Reasoning with Latent Cognitive Map
May 16, 2026Jinzhou Tang, Sidi Liu, Waikit Xiu +2Spatiotemporal Reasoning State Representation Learning
1Sun Yat-sen University
MAP: A Map-then-Act Paradigm for Long-Horizon Interactive Agent Reasoning
May 13, 2026Yuxin Liu, Ziang Ye, Yueqing Sun +6LLM Agent Training Long-Horizon LLM Agents
University of Science and Technology of China · Meituan · Institution of Automation, Chinese Academy of Sciences +1
Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation
May 11, 2026Zhixuan Shen, Jiawei Du, Ziyu Guo +5RL for Robotics Embodied Navigation
School of Computing and Artificial Intelligence, Southwest Jiaotong University, China · Centre for Frontier AI Research, A*STAR, Singapore · School of Computer Science, University of Leeds, UK

May 16, 2026J/K move · Enter open · S save

LASAR: Towards Spatio-temporal Reasoning with Latent Cognitive Map

Jinzhou Tang, Sidi Liu, Waikit Xiu, Weixing Chen+1

1Sun Yat-sen University

Spatiotemporal Reasoning State Representation Learning Memory-Augmented Agents

Open paper AskPDF arXiv

A fundamental challenge in embodied AI is verifying if agents build internal models of spatial structure or merely learn to mimic task-specific expert trajectories. This is critical as foundational approaches rooted in action-centric tasks (e.g., VLN) and reasoning-centric tasks (e.g., EQA) often share a common limitation: they lack a learning signal that forces them to encode fine-grained spatial relationships (like topology or distance) over long-range, fragmented experiences. To address this, we first propose LASAR, an architecture featuring a dual-memory system designed to maintain both episodic experiences and a semantic cognitive map. We then introduce Spatio-temporal Contextual Representation Learning (ST-CRL), a contrastive objective designed to train this architecture. ST-CRL leverages spatio-temporal cues from cognitive queries generated through annotated spatio-temporal context in simulation to build sample pairs, thereby forming the internal cognitive map from the agent's experiences. Experiments demonstrate that our method achieves 2%-3.5% gains in both zero-shot generalization on standard VLN-CE and VSI-Bench benchmarks. We also demonstrate that our proposed cognitive map has high self-consistency.

Abstract

Embodied Navigation

Explore similar work

LASAR: Towards Spatio-temporal Reasoning with Latent Cognitive Map

MAP: A Map-then-Act Paradigm for Long-Horizon Interactive Agent Reasoning

Plan in Sandbox, Navigate in Open Worlds: Learning Physics-Grounded Abstracted Experience for Embodied Navigation