RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

Yinpei Dai; Hongze Fu; Jayjun Lee; Yuejiang Liu; Haoran Zhang; Jianing Yang; Chelsea Finn; Nima Fazeli

cs.ROMar 4, 2026

RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies

Authors: Yinpei Dai, Hongze Fu, Jayjun Lee, Yuejiang Liu, Haoran Zhang, Jianing Yang, Chelsea Finn, Nima Fazeli, +1 more

Organizations: University of Michigan · Stanford University · Figure AI

Abstract

Memory is critical for long-horizon and history-dependent robotic manipulation. Such tasks often involve counting repeated actions or manipulating objects that become temporarily occluded. Recent vision-language-action (VLA) models have begun to incorporate memory mechanisms; however, their evaluations remain confined to narrow, non-standardized settings. This limits systematic understanding, comparison, and progress measurement. To address these challenges, we introduce RoboMME: a large-scale standardized benchmark for evaluating and advancing VLA models in long-horizon, history-dependent scenarios. Our benchmark comprises 16 manipulation tasks constructed under a carefully designed taxonomy that evaluates temporal, spatial, object, and procedural memory. We further develop a suite of 14 memory-augmented VLA variants built on the π0.5 backbone to systematically explore different memory representations across multiple integration strategies. Experimental results show that the effectiveness of memory representations is highly task-dependent, with each design offering distinct advantages and limitations across different tasks. Videos and code can be found at our website https://robomme.github.io.

Figures & tables

Task Name	Memory Type	Avg. #Steps	Task Challenge	Brief Description
Task Name		Avg. #Steps
Task Suite: Counting
PickXTimes	T	538	count	Pick and place a cube of a given color for a specified number of repetitions.
BinFill	T	604	count	Place a specified number of cubes of a given color into the bin as the cubes appear over time.
SwingXTimes	T	435	count, swing-motion	Swing a cube back and forth between two targets for a specified number of cycles.
StopCube	T	317	count, time-critical	Press a button exactly when a moving cube reaches the target at a specified occurrence.

Table 1: Task Summary . Overview of all tasks, their corresponding memory types, average total timesteps, and key challenges.

Dataset	Environment Complexity			Condition		Annotation		Memory Type	Total #Tasks	Total #Demos	Avg. #Steps
Dataset	Non-Markov	Partial Obs.	Dynamic	Vid.	Lang.	Subgoal	Keyframe	Memory Type	Total #Tasks	Total #Demos	Avg. #Steps
RLBench18 [ 21 , 39 ]	✗	✗	✗	✗	✓	✗	✓	T	18	1,800	137
CALVIN [ 34 ]	✗	✗	✗	✗	✓	✓	✓	T	34	1,000	584
LIBERO [ 31 ]	✗	✗	✗	✗	✓	✗	✗	T	130	6,500	162
VLABench [ 50 ]	✗	✗	✗	✗	✓	✗	✗	T	100	1,600	115
RoboCerebra [ 19 ]	✗	✗	✓	✗	✓	✓	✗	T	100	1,000	2,972

Table 2: Benchmark Comparison. The upper section lists general-purpose benchmarks and the lower section lists memory-based benchmarks. Memory types include temporal (T), spatial (S), object (O), and procedural (P). Environment complexity captures non-Markovian structure, partial observability, and dynamic scene change. Condition denotes initial inputs (Vid. for video observations; Lang. for language instructions). Annotations indicate the availability of language-based subgoals or keyframe timesteps for auxiliary supervision.

Figure 2: Framework of MME-VLA Suite. The top part illustrates three memory representations, each with two instantiations: (1) Symbolic Memory summarizes past interactions as high-level abstractions via language-based subgoals, optionally grounded to image pixels; (2) Perceptual Memory encodes history as raw visual tokens, using either token dropping to remove redundancy or uniform frame sampling to preserve essential context; (3) Recurrent Memory compresses history into fixed-size latent states through test-time training or recurrent memory transformers. The bottom part shows three integration strategies consistent with $\pi_{0.5}$ : (1) Memory-as-Context concatenates memory tokens with observation tokens (e.g., images, task instructions, and proprioception); (2) Memory-as-Modulator applies adaptive LayerNorm to modulate the action expert, where action features cross-attend to memory tokens via multi-head attention; (3) Memory-as-Expert adds a separate memory expert that interacts with other experts through block-wise causal attention.

Method		Counting				Permanence				Reference				Imitation				AVG
Method		Bin Fill	Pick Xtimes	Swing Xtimes	Stop Cube	Video Umsk	Button Umsk	Video UmskS	Button UmskS	Pick HighL	Video Repick	Video PlcBtn	Video PlcOrd	Move Cube	Insert Peg	Pattern Lock	Route Stick	AVG
Human Performance		96.00	100.0	80.00	78.00	90.00	92.00	92.00	90.00	92.00	92.00	98.00	90.00	90.00	98.00	84.00	86.00	90.50
MME-VLA w/ Symbolic Memory
SimpleSG	Oracle	85.78	99.78	100.0	44.67	33.11	22.00	15.56	15.56	44.00	27.78	31.33	26.00	87.33	10.00	95.33	55.11	49.58
GroundSG	Oracle	85.78	100.0	100.0	49.67	98.78	95.00	99.22	80.22	83.33	97.33	100.0	100.0	87.78	15.56	97.00	55.56	84.08
	Gemini	46.00	63.00	45.00	2.00	29.00	9.00	14.00	2.00	20.00	15.00	26.00	29.00	61.00	4.00	7.00	0.00	23.25

Table 3: Main Results. Our MME-VLA suite integrates

\pi_{0.5}

with three memory representations: (1) symbolic memory, implemented as simple or grounded subgoals ( SimpleSG , GroundSG ); (2) perceptual memory, using token dropping ( TokenDrop ) or frame sampling ( FrameSamp ); and (3) recurrent memory, using test-time training ( TTT ) or a recurrent memory transformer ( RMT ). Symbolic subgoals can be generated by Gemini-2.5-Pro ( Gemini ), a fine-tuned Qwen3-VL-4B ( QwenVL ), or simulator ground truth ( Oracle ). Perceptual and recurrent memory are integrated end-to-end via three mechanisms: memory-as-context ( Context ), memory-as-modulator ( Modul ), and memory-as-expert ( Expert ). Red marks the best per section; marks the overall best for non-oracle models.

Figure 3: Performance comparison across task characteristics.

Figure 4: Efficiency-performance comparison among models.

Method	Put Fruits	Track Cube	Repick Block	Draw Pattern	Total
$\pi_{0.5}$	2/10	1/10	1/10	0/10	4/40
GroundSG + QwenVL	9/10	3/10	5/10	2/10	19/40
FrameSamp + Modul	6/10	5/10	6/10	8/10	25/40

Table 4: Real-world experiment results.

Appendix figures & tables51 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
Batch Size	48
Warmup Ratio	0.05
LR	$1\times 10^{-4}$
Train Epochs	2
LoRA	rank 16, alpha 32
DeepSpeed	ZeRO-2

Appendix

Table 5: Training configuration for VLM subgoal predictor.

Panel of Figure 5 — Figure 5: Example Prompt for VLM Simple Subgoal Prediction. This is used to generate simple subgoals for the SimpleSG VLA policy.

Panel of Figure 6 — Figure 6: Example Prompt for VLM Grounded Subgoal Prediction. This is used to generate grounded subgoals for the GroundSG VLA policy.

Panel of Figure 7 — Figure 7: Example Prompt for MemER Baseline. We use grounded subgoals for MemER because we found that grounded subgoals perform better.

Hyperparameter	Value
Base model	$\pi_{0.5}$ base
Batch size	64 for symbolic/perceptual, 16 for recurrent
Action horizon	20
Action space	Joint-space
Proprioception	False
Total steps	80,000

Appendix

Table 6: Training hyperparameters for MME-VLA models.

Hyperparameters	SAM2Act Training	SAM2Act+ Training
Batch size	48	80
LR	$6.0\times 10^{-4}$	$1.0\times 10^{-4}$
Optimizer	LAMB	LAMB
LR schedule	warmup + cosine	warmup + cosine
Weight decay	$1\times 10^{-4}$	$1\times 10^{-4}$
Warmup steps	2000	2000

Appendix

Table 7: Training hyperparameters for SAM2Act and SAM2Act+.

Hyperparameter	Value
Base model	OpenVLA-7B (LoRA, rank $32$ )
Batch size	32 (symbolic), 24 (perceptual)
Action head	L1 regression, parallel decoding
Action horizon	20
Action space	Joint-space
Proprioception	True

Appendix

Table 8: Training hyperparameters for OpenVLA-OFT.

Hyperparameter	Value
Base VLM	Prismatic VLM
Batch size	$64$
Total steps	$160{,}000$
Action head	DiT-L diffusion (depth $24$ , width $1024$ , $16$ heads)
Action horizon	$16$ ( $1$ current $+\,15$ future)
Action space	EEF

Appendix

Table 9: Training hyperparameters for MemoryVLA.

Hyperparameter	Value
Model	1D U-Net (down dims $[256,512,1024]$ )
Visual encoder	ResNet-18 + SpatialSoftmax
Batch size	$128$ , single GPU
Total steps	$200{,}000$
Conditioning	2 cameras + $8$ -D state + CLIP ViT-B/32 text (frozen)
Diffusion	DDPM, $100$ steps, cosine, $\epsilon$ -pred, clip sample

Appendix

Table 10: Training hyperparameters for Diffusion Policy on RoboMME.

Method	Counting				Permanence				Reference				Imitation				AVG
Method	Bin Fill	Pick Xtimes	Swing Xtimes	Stop Cube	Video Umsk	Button Umsk	Video UmskS	Button UmskS	Pick HighL	Video Repick	Video PlcBtn	Video PlcOrd	Move Cube	Insert Peg	Draw Pattern	Route Stick	AVG
Human	96.00	100.0	80.00	78.00	90.00	92.00	92.00	90.00	92.00	92.00	98.00	90.00	90.00	98.00	84.00	86.00	90.50
Gemini-2.5-Pro	78.00	80.00	58.00	32.00	86.00	82.00	18.00	12.00	78.00	28.00	50.00	60.00	66.00	32.00	4.00	2.00	47.88
GPT4o	50.00	58.00	32.00	18.00	32.00	60.00	28.00	34.00	30.00	14.00	18.00	20.00	70.00	18.00	6.00	2.00	30.63
Qwen3-VL-32B	54.00	60.00	54.00	32.00	26.00	28.00	22.00	14.00	14.00	32.00	26.00	14.00	66.00	2.00	2.00	0.00	27.88

Appendix

Table 11: Performance using the oracle planner for both foundation models and human participants.

Representation	Policy	Integration/VLM	Counting	Permanence	Reference	Imitation	AVG
No Memory ( $\pi_{0.5}$ )	–	–	28.78	17.00	17.17	8.78	17.93 ± 0.61
Symbolic	SimpleSG	Oracle	82.56	21.56	32.28	61.94	49.58 ± 1.33
	GroundSG	Oracle	83.86	93.31	95.17	63.98	84.08 ± 1.86
		Gemini	39.00	13.50	22.50	18.00	23.25 ± 0.35
	SimpleSG	QwenVL	44.61	19.61	25.22	26.56	29.00 ± 0.39
		Gemini	12.75	15.75	11.25	6.5	11.56 ± 0.97

Appendix

Table 12: Comparison over MME-VLA suite and prior methods per task suite. Red marks the best per section; marks the overall best for non-oracle models.

Category	Functional Requirement	Tasks
Motion-Centric	Sustained trajectory tracking and structured continuous motions (e.g., linear, circular, insertion, swinging)	PatternLock , RouteStick , SwingXtimes , InsertPeg
Time-Sensitive	Precise temporal coordination; success depends critically on acting at the correct moment	StopCube
Short-Horizon Video Reasoning	Grounding objects from short video history in mostly static or mildly dynamic scenes	VideoUnmask , VideoRepick , VideoUnmaskSwap
Long-Horizon Video Reasoning	Integrating information across extended temporal sequences (often $>1000$ steps) before acting	VideoPlaceButton , VideoPlaceOrder
Dynamic Online Scene-Change	Continual belief updates and online adaptation under concurrent environmental changes during execution	ButtonUnmask , ButtonUnmaskSwap , PickHighlight
Event-Salient	progress is marked by visually salient, discrete events that clearly signal subtask completion or manipulation intent	PickXtimes , BinFill , MoveCube

Appendix

Table 13: Task categorization based on functional characteristics.

Representation	Policy	Integration/ VLM Type	Motion- Centric	Time- Sensitive	Short- Horizon	Long- Horizon	Scene- Change	Event- Salient
No Memory ( $\pi_{0}$ )			11.17	6.67	16.39	28.45	13.41	32.96
Symbolic	SimpleSG	Gemini	14.00	2.00	19.33	27.50	10.33	56.67
	SimpleSG	QwenVL	7.34	0.44	24.96	29.22	15.33	84.96
	GroundSG	Gemini	3.25	3.00	22.00	9.50	7.67	20.33
	GroundSG	QwenVL	5.83	0.00	48.22	42.89	17.70	72.08
Perceptual	TokenDrop	Context	32.84	3.11	25.93	28.22	22.74	71.70

Appendix

Table 14: Performance grouped by task functional requirements. Results are averaged within each category.

Representation	Policy	Integration/ VLM Type	Counting				Permanence
Representation	Policy	Integration/ VLM Type	BinFill	PickXTimes	SwingXTimes	StopCube	VideoUnmask	ButtonUnmask	VideoUnmaskSwap	ButtonUnmaskSwap
Symbolic	SimpleSG	Oracle	85.78 ± 2.91	99.78 ± 0.67	100.0 ± 0.00	44.67 ± 22.25	33.11 ± 2.67	22.00 ± 3.06	15.56 ± 2.60	15.56 ± 3.84
Symbolic	GroundSG	Oracle	85.78 ± 1.67	100.00 ± 0.00	100.0 ± 0.00	49.67 ± 20.02	98.78 ± 1.04	95.00 ± 2.83	99.22 ± 1.04	80.22 ± 15.69
Symbolic	SimpleSG	Gemini	46.00 ± 7.14	63.00 ± 7.07	45.00 ± 4.24	2.00 ± 0.00	29.00 ± 1.41	9.00 ± 7.07	14.00 ± 2.83	2.00 ± 2.83
	SimpleSG	QwenVL	77.56 ± 2.19	95.33 ± 2.00	5.11 ± 1.45	0.44 ± 0.88	34.22 ± 3.53	19.33 ± 8.31	15.33 ± 2.83	9.56 ± 4.56
	GroundSG	Gemini	26.00 ± 8.49	18.00 ± 0.00	4.00 ± 0.00	3.00 ± 1.41	36.00 ± 0.00	14.00 ± 2.83	13.00 ± 1.41	0.00 ± 0.00

Appendix

Table 15: Full results for different models. Red marks the best per section; marks the overall best for non-oracle models.

Memory Tokens	FrameSamp + Modul	TokenDrop + Modul
64	30.42	18.11
128	36.54	32.76
256	42.90	36.36
512	44.51	38.02
1024	45.87	40.17

Appendix

Table 16: Effect of memory token budget on perceptual memory methods. Increasing the memory budget generally improves performance.

Method	Memory	Observation	Language	Success
FrameSamp + Context	1.5 / 8.9	60.2 / 84.9	38.3 / 63.9	30.68
TokenDrop + Context	4.8 / 17.0	45.4 / 72.9	49.2 / 94.8	34.50
TokenDrop + Expert	0.7 / 12.7	47.5 / 71.9	51.8 / 97.8	34.86
FrameSamp + Expert	1.3 / 21.8	46.0 / 68.2	52.7 / 96.4	36.25

Appendix

Table 17: Attention allocation among memory, observation, and language tokens. We report mean / max attention probability mass (%) aggregated across layers.

Method	Scale	Bias	Success
TokenDrop + Modul	$5.99\pm 38.67$	$0.024\pm 14.19$	38.04
FrameSamp + Modul	$3.74\pm 22.55$	$0.012\pm 9.45$	44.51

Appendix

Table 18: Statistics of memory-conditioned modulation. We report mean

\pm

standard deviation of the predicted scale and bias values.

Figure 9: We design four history-dependent real-world tasks for policy evaluation. The upper dashed box indicates selected frames from previous video history. The bottom large box indicates current observation for execution.

Figure 10: Real Robot Experiment Platform Setup.

Figure 11: Overview of the main assets used in RoboMME.

Difficulty	#Colors	#Total Cubes	#Goal Cubes
Easy	1	4–6	1
Medium	2	8–10	1–2
Hard	3	10–12	2–3

Appendix

Table 19: BinFill Task Configuration.

Figure 12: BinFill Task Example. In this instance, the goal is to place one blue cube into the bin and then press the button to stop.

Difficulty	#Total Cubes	#Repetitions
Easy	1	1–3
Medium	3 (2 distractors)	1–3
Hard	3 (2 distractors)	4–5

Appendix

Table 20: PickXtimes Task Configuration.

Figure 13: PickXtimes Task Example. In this instance, the goal is to pick up the green cube and place it on the target twice, then press the button to stop.

Difficulty	#Total Cubes	#Repetitions
Easy	1	1–3
Medium	3 (2 distractors)	1–2
Hard	3 (2 distractors)	3

Appendix

Table 21: SwingXtimes Task Configuration.

Figure 14: SwingXtimes Task Example. In this instance, the goal is pick up the green cube, first move it to the right-side target, then put down the cube on the left-side target (i.e., swing between targets one time), finally press the button to stop. Spatial directions (e.g., left/right) follow the robot base coordinate frame rather than the front camera frame.

Difficulty	#Passes	Move Speed
Easy	2–3	slow/medium/fast
Medium	3–4	slow/medium/fast
Hard	4–5	slow/medium/fast

Appendix

Table 22: StopCube Task Configuration.

Figure 15: StopCube Task Example. In this instance, the goal is press the button to stop the cube exactly at the target on its second visit.

Difficulty	#Total Containers	#Goal Containers	#Total Cubes
Easy	3	1	3 (red, green, blue)
Medium	5	1	3 (red, green, blue)
Hard	10	2	3 (red, green, blue)

Appendix

Table 23: VideoUnmask Task Configuration.

Figure 16: VideoUnmask Task Example. In this instance, after watching a video in which all cubes are masked, the robot must pick up the container hiding the green cube. Red-bordered frames denote the video-based observation prior to execution.

Difficulty	#Total Containers	#Goal Containers	#Total Cubes
Easy	3	1	3 (red, green, blue)
Medium	5	1	3 (red, green, blue)
Hard	15	2	3 (red, green, blue)

Appendix

Table 24: ButtonUnmask Task Configuration.

Figure 17: ButtonUnmask Task Example. In this instance, the robot first presses the button on the table and then selects the container hiding the green cube. During the button press, all cubes are masked.

Difficulty	#Total Containers	#Swap Times	#Goal Containers	#Total Cubes
Easy	3	1–2	1–2	3 (red, green, blue)
Medium	4	1–2	1	3 (red, green, blue)
Hard	4	2–3	2	3 (red, green, blue)

Appendix

Table 25: VideoUnmaskSwap Task Configuration.

Figure 18: VideoUnmaskSwap Task Example. In this instance, the robot first watches the video, then picks up the container hiding the blue cube, followed by the one hiding the green cube. The video shows that all cubes are being masked by containers, with some containers swapped between locations. Red-bordered frames denote the video-based initial observations provided before execution.

Difficulty	#Total Containers	#Goal Containers	#Swap Times	#Total Cubes
Easy	3	1–2	1–2	3 (red, green, blue)
Medium	4	1	1–2	3 (red, green, blue)
Hard	4	2	2–3	3 (red, green, blue)

Appendix

Table 26: ButtonUnmaskSwap Task Configuration.

Figure 19: ButtonUnmaskSwap Task Example. In this instance, the robot first presses both buttons on the table, then picks up the container hiding the green cube, followed by the one hiding the blue cube. During the button presses, all cubes are masked and their locations are swapped.

Difficulty	#Total Cubes	#Goal Cubes
Easy	3	1
Medium	4	2
Hard	6 (cluster)	3

Appendix

Table 27: PickHighlight Task Configuration.

Figure 20: PickHighlight Task Example. In this instance, the goal is to first press the button, then pick up all cubes highlighted by a white area , finally press the button again to stop. During the button pressing, the green cube was highlighted with a white area on the table, the robot needs to capture this information, and pick up the correct cubes.

Difficulty	#Cubes	#Swap Times	#Repetition
Easy	3 (same color)	1–2	1–3
Medium	3 (same color)	2–3	1–3
Hard	15 (different colors, clutter scene)	0	1–3

Appendix

Table 28: VideoRepick Task Configuration.

Figure 21: VideoRepick Task Example. In this instance, the robot first watches a video, then picks up the same previously picked cube twice, and finally presses the button to stop. Red-bordered frames denote the video-based observations shown prior to execution.

Difficulty	#Cubes	#Targets	Target Swapping
Easy	1	3	No
Medium	3	4	No
Hard	3	4	Yes

Appendix

Table 29: VideoPlaceButton Task Configuration.

Figure 22: VideoPlaceButton Task Example. In this example, the robot first observes a video depicting an interleaved sequence of cube placements and button presses, and then places the cube onto the correct target corresponding to its location prior to the button press. Red-bordered frames indicate the video-based observations provided before execution.

Difficulty	#Cubes	#Targets	Target Swapping
Easy	1	4	No
Medium	3	4	No
Hard	3	4	Yes

Appendix

Table 30: VideoPlaceOrder Task Configuration.

Figure 23: VideoPlaceOrder Task Example. In this instance, the goal is to place the red cube on the third target where it was previously placed in the given video. Red-bordered frames denote the video-based observation prior to execution.

Figure 24: MoveCube Task Example. In this instance, the robot must watch the video carefully, then move the cube to the target in the same manner as demonstrated. Red-bordered frames denote the video-based observation prior to execution.

Figure 25: InsertPeg Task Example. In this instance, the robot must watch the video carefully, then grasp the same peg at the same end and insert it into the same side of the box as demonstrated. Red-bordered frames denote the video-based observation prior to execution.

Difficulty	Grid Size	Pattern Length
Easy	3 $\times$ 3	2–4
Medium	4 $\times$ 4	3–5
Hard	5 $\times$ 5	4–8

Appendix

Table 31: PatternLock Task Configuration

Figure 26: PatternLock Task Example. In this instance, the goal is to watch the video carefully and then use the stick attached to the robot to retrace the same pattern shown in the video. Red-bordered frames denote the video-based observations prior to execution.

Difficulty	Path Length	Backtracking
Easy	2–3	No
Medium	4–5	No
Hard	4–7	Yes

Appendix

Table 32: RouteStick Task Configuration

Figure 27: RouteStick Task Example. In this instance, the robot watches the video and uses the stick attached to the end-effector to navigate around obstacles on the table, reproducing the demonstrated path.

Explore similar work

MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation
Sep 7, 2026Haiyang Sun, Haoxiao Wang, Junming Chen +6Memory Agent Benchmarks Robot Manipulation Benchmarks
Zhejiang University, Hangzhou, China
MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation
Sep 30, 2026Egor Cherepanov, Nikita Kachaev, Aleksandr I. Panov +1Memory-Augmented VLMs Long-Horizon Robotic Manipulation
AXXX, Moscow, Russia · MIRIAI, Moscow, Russia · HSE University, Moscow, Russia
Where Memory Belongs: Ledger, an Object Ledger for Memory-Augmented VLAs
Sep 28, 2026Tanguy Dieudonné, Jack B. Jedlicki, Heng YangMemory-Augmented VLMs Long-Horizon Robotic Manipulation
Harvard University · ETH Zürich

Sep 7, 2026J/K move · Enter open · S save

MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation

Haiyang Sun, Haoxiao Wang, Junming Chen, Weicheng Fang+5

Zhejiang University, Hangzhou, China

Memory Agent Benchmarks Robot Manipulation Benchmarks Robotic Manipulation

Open paper AskPDF arXiv

Robotic manipulation often requires acting on information that is no longer visible, yet Vision-Language-Action policies are usually evaluated when the current observation largely determines the next action. Existing robotic memory benchmarks expose this gap, but they still rely mainly on final task success and therefore conflate forgetting with manipulation failure. We present MEMOBench, a benchmark for process level memory evaluation in robotic manipulation. MEMOBench includes 30 history dependent tasks, 1{,}500 expert demonstrations, and 4{,}200 executable checkpoint instances from 84 templates. Each checkpoint pairs coarse to fine language with a simulator predicate and labels one memory operation: Storage, Update, or Compression. These annotations define Memory Storage Rate, Memory Update Rate, and Memory Compression Rate, which measure memory fidelity alongside task success. Across standard and memory augmented VLA policies, the strongest memory module baseline reaches only 31.9% average success rate, and high storage often coexists with weak update and compression. Checkpoint language also supervises semantic, contrastive, and framewise memory alignment objectives, yielding modest gains across different memory operations. MEMOBench provides a diagnostic evaluation suite and training supervision for memory grounded robotic policies. The project page is available at https://github.com/Collab-Gen/MEMOBench.

Abstract

Explore similar work

MEMOBench: A Process Level Memory Benchmark for Robotic Manipulation

MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation

Where Memory Belongs: Ledger, an Object Ledger for Memory-Augmented VLAs