Organizations: FNii-Shenzhen · The Chinese University of Hong Kong, Shenzhen · Ising AI · SSE, CUHK-Shenzhen · The University of Hong Kong · National University of Singapore · The Chinese University of Hong Kong
Long-horizon embodied tasks require agents to learn, reason, and generalize robustly across complex real-world environments. However, existing approaches often struggle with partial observability, fragmented spatial reasoning, and inefficient integration of heterogeneous memories, limiting their capacity for long-horizon adaptation. To address this, we introduce RoboMemory, a brain-inspired framework that unifies Temporal, Spatial, Semantic, and Episodic memory for efficient long-horizon planning and interactive learning. Its core innovations are a comprehensive memory system designed for embodied tasks, and a dynamic spatial knowledge graph for scalable, consistent memory updates. Extensive experiments on EmbodiedBench show that RoboMemory improves the average success rate of Qwen2.5-VL-72B-Ins by 26.5 percentage points and achieves performance comparable to strong closed-source VLM agents. Real-world trials further suggest RoboMemory's potential for interactive environmental learning by reusing accumulated experience across repeated task attempts. These results demonstrate RoboMemory as a scalable memory-augmented framework for embodied agents, improving long-horizon planning and cumulative learning in complex interactive environments.
Figures & tables
Figure 2: (a) Left: The loop where the Planner, Critic, and Comprehensive Embodied Memory interact to adjust plans based on real-time visual inputs. (b) Right: The update mechanism of Spatial memory, Semantic memory, and Episodic memory.
Method
Type
EB-ALFRED
EB-Habitat
Average
Base
Long
Base
Long
SR
GC
SR
GC
SR
GC
SR
GC
SR
GC
Single VLM-Agents
GPT-4o
Closed-source
67.0
74.9
64.0
74.0
54.0
62.5
86.0
90.7
64.0
72.2
GPT-4o-mini
30.5
40.9
34.0
47.8
0.0
17.0
74.0
77.5
14.0
21.3
Claude-3.7-Sonnet
68.5
-
68.0
-
70.0
-
90.0
-
46.0
-
Table 1: Comparison of Success Rates (SR) and Goal Condition Success Rates (GC) across difficulty levels (Base/Long) on EB-ALFRED and EB-Habitat benchmarks. Values are reported in percentages (%).
Figure 3Figure 4
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Multimodal
Episodic
Semantic
Spatial
Temporal
Procedural
Memory Implementation
Real Robot
NeSyC [ 9 ]
✓
✓
✓
Symbolic logic rules
✓
Reflexion [ 27 ]
✓
✓
Buffer
Voyager [ 36 ]
✓
RAG
MSI-Agent [ 11 ]
✓
✓
Database, RAG
CoELA [ 42 ]
✓
✓
✓
✓
Top-down semantic map
Cradle [ 31 ]
✓
✓
✓
✓
RAG
Appendix
Table 3: Comparison of Memory-Related Methods in Embodied Agent
Figure 6: The reason why RoboMemory failed to complete the task.
Figure 7: Average relationships related to update in spatial memory in each step.
Condition
Avg.
Base
Long
Planner–Critic, no RoboMemory
38
62
14
Flat Spatial RAG
48
56
40
RoboMemory (Dynamic KG)
67
68
66
Appendix
Table 4: Matched attribution controls on EB-ALFRED (SR, %).
Figure 8: Visualization of Spatial Memory’s dynamic update process.
Action Type
EB-ALFRED
EB-Habitat
Real World
Navigation
find(obj)
navigate(point)
navigate_to(point)
Pick Up Object
pick_up(obj)
pick(obj)
pick_up(obj)
Drop to Ground
drop()
–
–
Place to Receptacle
put_down()
place(rec)
put_down_to(rec)
Open Object
open(obj)
open(obj)
open(obj)
Close Object
close(obj)
close(obj)
close(obj)
Appendix
Table 5: Robot Action Command For different environments
Dataset Statistics
Training Configuration
Action Type
#Episodes
Parameter
Value
Turning on/off faucet
142
Optimizer
AdamW
Picking up & Placing basket on counter
63
Batch size
32×6
Picking up & Placing basket in sink
72
Training steps
10,000
Picking up & Placing banana into basket
114
Learning rate
6.12×10−5
Throwing bottle into trash bin
132
warm up step
500
Appendix
Table 6: Dataset statistics and training hyperparameters for robotic manipulation tasks.
Figure 9: Case that a task is failed, but the experience can help RoboMemory to succeed in the next try.
Figure 10: Case that a task is successful.
Figure 11: Case that a task is successful with the help of the critic and spatial memory modules.
Figure 12: Case that a task fails in an infinite loop because the critic module failed to stop the agent when its planned action is no longer suitable.
Figure 13: Case that a task fails in an infinite loop because of inaccurate action planning.