ForagerSim: Evaluating Embodied Map-Based Reasoning in Large-Scale Dynamic Worlds
Authors: Tomáš Musil, Martin Kotík, Michal Vavrečka, Karla Štěpánová, Sebastian Scherer, Martin Saska
Organizations: Dept. of Cybernetics, FEE, Czech Technical University in Prague · CIIRC, Czech Technical University in Prague · Robotics Institute, Carnegie Mellon University
Animals build cognitive maps of vast environments, use them to perform map-based navigation, and might even be reusing the same circuitry used in spatial reasoning for creating and using maps of arbitrary task-spaces. In contrast, current embodied AI systems either rely on hand-engineered mapping pipelines, which break in the presence of unmodeled dynamics and are restricted to metric space, or on learned policies that rarely form map-like representations at all. Developing architectures that could learn to build and use map-like representations as flexibly as animals is, however, difficult because few embodied AI benchmarks actually require map-based reasoning. To address this, we present ForagerSim --- an interdisciplinary research platform and simulation environment focused on embodied tasks that require map-based reasoning, with scenarios ranging from mobile-robot search and rescue to animal foraging in both complex mazes and vast, potentially infinite open areas. ForagerSim is highly modular and allows modifying independent axes of difficulty related to perception (abstract 2D sensing to full RGB or LiDAR), scale (room-scale to multi-km scale), topology (open-fields to complex mazes), dynamics (smoke, debris, sensorimotor failures and others) and tasks (from simple 2D coverage to foraging-inspired well-alternation tasks). We compare reinforcement learning baselines with a simple engineered robotics stack and a human player on 8 example worlds which show how different difficulty axes affect existing methods. Our results show that, when sensing is simplified to 2D, learned policies can perform multi-object search and well-alternation tasks surprisingly well in simple topologies, but they fall short of human performance in larger or more complex-topology environments where map-based baselines perform well, suggesting that map-based reasoning is the bottleneck in these scenarios.
Figures & tables
Figure 1: Example scenarios in ForagerSim - from left to right: The 4arm-submazes maze; multi-object search in a dynamic-environment consisting of houses with movable rubble and chaotically walking entities; ratmaze-foraging experiment with rewards alternating between random wells and a home well; massive-scale realistic bat-level foraging scenario
Figure 2: Illustration of the agent’s observations. Left: top-down view of the environment and the raycasts. Right: human-UI visualization of the depth, semantics and compass data.
Figure 3: Bar charts of all the eval runs for each of the 8 scenarios described in subsection 4.1
World
PPO
Dreamer
Frontier-Explorer
Human
memorymaze15x15
309±39
320±26
339±1
339±1
10houses-fenced
366±303
2292±481
1947±389
3452±134
4arm-submazes
504±210
777±160
1868±44
1895±3
5houses-dynamic
377±282
984±120
152±0
1114±100
5houses-sar
142±45
864±196
118±49
957±99
ratmaze-foraging
470±525
2883±256
–
3827±515
Table 2: Benchmark results on the 8 world configurations. All numbers show mean per-episode return and its standard deviation, computed from 5 eval seeds per scenario, averaged across training seeds
Figure 4: Adaptive difficulty curves for the curriculum experiment described in subsection 4.2 .
Figure 5: Visualization of eval trajectories on the 10houses-fenced and 4arm-submazes topologies. Color of trajectory indicates step index in episode, white circle = start, white square = end, stars = reward object pickups.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: A visualization of 2 random seed for each of the 8 worlds used in the comparisons in subsection 4.1
Figure 7: A visualization of adaptive difficulty scaling of world complexity on the example of the 4arm-submazes world in the experiments described in subsection 4.2
World
Method
Score
OBJ
Coverage [ 102 m 2 ]
memorymaze15x15
PPO
309±39
5.5±0.7
36.4±2.7
Dreamer
320±26
5.7±0.5
35.9±2.2
Frontier-Explorer
339±1
6.0±0.0
39.2±0.7
Human
339±1
6.0±0.0
38.9±0.7
10houses-fenced
PPO
366±303
3.7±3.6
190.0±159.8
Dreamer
2292±481
37.9±8.4
401.0±92.6
Appendix
Table 3: Additional metrics for the evaluation described in subsection 4.1 and in Table 2 . OBJ signifies number of collected reward objects, coverage is the area explored by the agent’s sensors in the order of 102 , since this is equal to the contribution of the coverage to the overall score (since we give a reward of 0.01 for each covered square meter).
Figure 8: Full training curves of for each of the presented worlds. In all figures, the line represents mean return per episode averaged across training seeds. Top 2 rows show the standard deviation of the mean returns between individual training seeds. Bottom 2 rows show the deviation across episodes averaged over training seeds, which shows that the baseline performance varies drastically between individual generated worlds within any scenario, and corresponds to the high eval-world score deviation in the evaluations in subsection 4.1
Figure 9: Visualization of eval trajectories on the forest-foraging unbounded well-alternation scenario. Dreamer walks outside of the main area into the infinite forest in one world, PPO in all of them.
Figure 10: Visualization of eval trajectories on the 4arm-submaze-foraging well-alternation scenario. The human player can perform 3 or more random-home alternations while both PPO and Dreamer struggle with learning to return home after finding the cued random well.
Figure 11: Trajectory visualizations for the memorymaze15x15 world, showing that it can be easily fully covered in the multi-object search task by even a model-free, memoryless PPO agent. This suggests that with enough time (since the human and the frontier explorer were able to cover it sometimes in 50% of the episode steps), this environment is not challenging in terms of map-based reasoning or memory (although the original ordered -search task is, as the authors present, shown to be a good benchmark of memory).
Figure 13: Examples of the 3D environments available in ForagerSim in the RGB/LiDAR sensing modes, Top: a 3D variant of search-and-rescue missions in suburb-scale environments with interactible, randomized houses (with ability to swap abstract red boxes as reward objects for human models), Bottom: a recreation of the Hula valley on which the animal experiments documenting the scalability of biological cognitive maps of bats have been recorded Goldshtein et al. (2024)
Table 4: DreamerV3 hyperparameters that differ from the official implementation’s defaults (defaults in parentheses). PPO uses the Stable-Baselines3 defaults.
A fundamental challenge in embodied AI is verifying if agents build internal models of spatial structure or merely learn to mimic task-specific expert trajectories. This is critical as foundational approaches rooted in action-centric tasks (e.g., VLN) and reasoning-centric tasks (e.g., EQA) often share a common limitation: they lack a learning signal that forces them to encode fine-grained spatial relationships (like topology or distance) over long-range, fragmented experiences. To address this, we first propose LASAR, an architecture featuring a dual-memory system designed to maintain both episodic experiences and a semantic cognitive map. We then introduce Spatio-temporal Contextual Representation Learning (ST-CRL), a contrastive objective designed to train this architecture. ST-CRL leverages spatio-temporal cues from cognitive queries generated through annotated spatio-temporal context in simulation to build sample pairs, thereby forming the internal cognitive map from the agent's experiences. Experiments demonstrate that our method achieves 2%-3.5% gains in both zero-shot generalization on standard VLN-CE and VSI-Bench benchmarks. We also demonstrate that our proposed cognitive map has high self-consistency.
Current interactive LLM agents rely on goal-conditioned stepwise planning, where environmental understanding is acquired reactively during execution rather than established beforehand. This temporal inversion leads to Delayed Environmental Perception: agents must infer environmental constraints through trial-and-error, resulting in an Epistemic Bottleneck that traps them in inefficient failure cycles. Inspired by human affordance perception and cognitive map theory, we propose the Map-then-Act Paradigm (MAP), a plug-and-play framework that shifts environment understanding before execution. MAP consists of three stages: (1) Global Exploration, acquiring environment-general priors; (2) Task-Specific Mapping, constructing a structured cognitive map; and (3) Knowledge-Augmented Execution, solving tasks grounded on the map. Experiments show consistent gains across benchmarks and LLMs. On ARC-AGI-3, MAP enables frontier models to surpass near-zero baseline performance in 22 of 25 game environments. We further introduce MAP-2K, a dataset of map-then-act trajectories, and show that training on it outperforms expert execution traces, suggesting that understanding environments is more fundamental than imitation.
Yuxin Liu, Ziang Ye, Yueqing Sun +6
University of Science and Technology of China · Meituan · Institution of Automation, Chinese Academy of Sciences +1
Vision-Language Models (VLMs) have demonstrated exceptional general reasoning capabilities. However, their performance in embodied navigation remains hindered by a scarcity of aligned open-world vision and robot control data. Despite simulators providing a cost-effective alternative for data collection, the inherent reliance on photorealistic simulations often limits the transferability of learned policies. To this end, we propose Sandbox-Abstracted Grounded Experience (SAGE), a framework that enables agents to learn within a physics-grounded semantic abstraction rather than a photorealistic simulation, mimicking the human capacity for mental simulation where plans are rehearsed in simplified physics abstractions before execution. SAGE system operates via three synergistic phases: (1) Genesis: constructing diverse, physics-constrained semantic environments to bootstrap experience; (2) Evolution: distilling experiences through Reinforcement Learning (RL), utilizing a novel asymmetric adaptive clipping mechanism to stabilize updates; (3) Navigation: bridging the abstract policy to open-world control. We demonstrate that SAGE significantly improves planner-assisted embodied navigation, achieving a 53.21% LLM-Match Success Rate on A-EQA (+9.7% over baseline), while showing encouraging transfer to physical indoor robot deployment.
Zhixuan Shen, Jiawei Du, Ziyu Guo +5
School of Computing and Artificial Intelligence, Southwest Jiaotong University, China · Centre for Frontier AI Research, A*STAR, Singapore · School of Computer Science, University of Leeds, UK