Organizations: University of Science and Technology of China · Tongji University · The Chinese University of Hong Kong · Anhui University · Nanjing University
In the real world, mobile manipulation robots must perform complex object interactions and make multi-step decisions, often relying on task-relevant information that is no longer observable. However, existing benchmarks provide limited evaluation of such memory-dependent decision-making in mobile manipulation. To address this gap, we introduce RoboCasa-Mem, a new benchmark comprising nine mobile manipulation tasks that require agents to use interaction history at key decision points. Specifically, the tasks are grouped into two categories of memory: One is environment memory, which captures spatial locations, object attributes, and object relationships. The other is process memory, which captures single-task progress and multi-task scheduling. Particularly, we introduce Behavior-Conditioned Evaluation. For tasks with multiple valid execution paths, instead of fixing a single target or trajectory, the evaluator derives later task constraints from the robot's earlier actions. This allows us to evaluate whether later behavior is consistent with the robot's own interaction history while reducing shortcuts based on fixed target assignments. We provide 1,800 successful demonstrations for policy training. We also develop a baseline on our benchmark, namely Hierarchical Memory-Augmented Policy(HMAP). Its high-level planner uses structured entity and process memories to generate subtasks. Its low-level VLA controller uses subgoal-conditioned queries to incorporate VGGT-Omega geometry into action generation. Together, RoboCasa-Mem and HMAP baseline support systematic evaluation of memory-dependent behavior and the development of memory-aware mobile manipulation policies.
Figures & tables
Memory coverage
Benchmark
Mobile
History [-1pt]dependence
Spatial
Attribute
Relation
Progress
Scheduling
Stage [-1pt]eval
MIKASA-Robo [-2pt] ( Cherepanov et al., 2025 )
✗
✓
✓
✗
✓
✓
✗
✗
RMBench [-2pt] ( Chen et al., 2026 )
✗
✓
✓
✗
✓
✓
✗
✗
RoboMME [-2pt] ( Dai et al., 2026 )
✗
✓
✓
✗
✓
✓
✗
✗
RoboTwin-Mem [-2pt] ( Yang et al., 2026 )
✗
✓
✓
✗
✓
✓
✗
✗
RoboMemArena [-2pt] ( Lei et al., 2026 )
✗
✓
✓
✓
✓
✓
✗
✓
Table 1: Comparison of simulated robotic memory benchmarks. Coverage denotes at least one qualifying task, not uniform requirements across tasks or equivalent taxonomies.
Figure 1: Overview of RoboCasa-Mem. Top: a history-dependent cup-selection example illustrates why the current observation alone may not determine the correct action. Middle: five memory categories cover spatial, attribute, and relational Environment Memory, together with single-task progress and multi-task scheduling in Process Memory. Bottom: demonstration construction and semantic evaluation support analysis of final outcomes and stage-level completion. Table 2 lists the nine core tasks.
Task
Memory type
Key challenge
Description
Environment Memory
PutBackCup
Spatial
Location recall
“Place the cup at the previously cleaned spot.”
SweetCup
Attribute
Hidden attribute
“Prepare one sweetened and one unsweetened coffee, then select the sweetened cup.”
TempReport
Attribute
Attribute recall
“Recall whether hot or cold water was dispensed, then press the matching button.”
CanRecall
Relational
Object–container binding
“After closing the cabinets, recall the requested items’ locations and place them back on the tabletop.”
MilkMatch
Relational
Source–target binding
“Transport milk boxes in two trips, placing each on the plate matching its source tray’s color.”
Table 2: Overview of the nine RoboCasa-Mem tasks, organized into five memory categories under Environment Memory and Process Memory. Quoted descriptions summarize the tasks rather than reproduce the exact policy instructions.
Figure 2: Overview of our Hierarchical Memory-Augmented Policy. The high-level planner maintains entity and process memory to select an active subgoal. The low-level subgoal-guided 3D-aware controller conditions on this subgoal and retrieves task-relevant features from current multi-view geometry to modulate action generation.
(a) Environment Memory
Model
PutBackCup
SweetCup
TempReport
CanRecall
MilkMatch
SR
SCR
SR
SCR
SR
SCR
SR
SCR
SR
SCR
ACT
0.00
0.48
0.00
0.07
0.00
0.09
0.00
0.09
0.00
0.03
DP
0.00
0.20
0.00
0.04
0.08
0.21
0.00
0.05
0.00
0.05
π0.5
0.68
0.85
0.04
0.27
0.22
0.57
0.00
0.24
0.12
0.47
MemER-style
0.10
0.22
0.00
0.21
0.00
0.58
0.00
0.21
0.00
0.19
Table 3: Results on the nine RoboCasa-Mem tasks. SR denotes task success rate, and SCR denotes stage completion rate. Avg. is the macro-average across all nine tasks. Best nonzero values are bolded, including ties at the reported precision.
Figure 3: An illustration of our real-world experiments conducted on AgileX Cobot Magic Platform. The task is set as: the robot first observes and picks up an object, then turns to transport it and places it into the corresponding container.
Table 7
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Execution workflows of the nine RoboCasa-Mem tasks.
Method
Clean
Tool
Return
ACT
0.72
0.72
0.00
DP
0.30
0.30
0.00
π0.5
0.94
0.94
0.68
MemER-style
0.28
0.28
0.10
HMAP (Ours)
0.84
0.84
0.70
Appendix
Table 6: Stage-level completion rates for PutBackCup, SweetCup, and TempReport. Each value is the fraction of evaluated episodes satisfying the corresponding stage and its required predecessors.
Method
Close-1
Close-2
Reopen
Success
ACT
0.34
0.00
0.00
0.00
DP
0.18
0.00
0.00
0.00
π0.5
0.76
0.14
0.04
0.00
MemER-style
0.80
0.02
0.00
0.00
HMAP (Ours)
0.80
0.24
0.00
0.00
Appendix
Table 7: Stage-level completion rates for the remaining six RoboCasa-Mem tasks. Abbreviated stage names are explained below each subtable.