EMBER-Bench: Benchmarking Cross-Event Causal Memory in Long-Horizon Embodied Tasks

Aoyang Cai; Boning Zhao; Shaoxuan Xie; Dahui Gao; Huan Yang; Zhongyuan Wang; Zhiwei Yu; Guocai Yao

cs.CVOct 4, 2026

EMBER-Bench: Benchmarking Cross-Event Causal Memory in Long-Horizon Embodied Tasks

Authors: Aoyang Cai, Boning Zhao, Shaoxuan Xie, Dahui Gao, Huan Yang, Zhongyuan Wang, Zhiwei Yu, Guocai Yao

Organizations: Tsinghua University · Beijing Academy of Artificial Intelligence · The University of Hong Kong

Abstract

Lifelong physical agents must reason over extended interactions where past events continue to shape the world long after they disappear from view. Beyond recalling what happened, agents must infer how history changes the current state and constrains future actions. Yet existing embodied and video-memory benchmarks largely focus on historical retrieval and summary, leaving such history-dependent causal reasoning underexplored. We introduce EMBER-Bench, an egocentric benchmark for cross-event causal reasoning in long-horizon embodied tasks, for which we newly created the task design, video recording, and data annotation. It contains 189 household tasks and 699 QA pairs, spanning task progress, failure recovery, external interventions, and compound long-horizon tasks with distant dependencies and prerequisites, with fine-grained event and causal-chain annotations. EMBER-Bench evaluates reasoning in both directions: next-action prediction selects the next action from history, and causal traceback, given that action, identifies the historical event that makes it necessary. Input ablations that add action logs or privileged cause-and-consequence annotations to the video indicate which kind of historical information models fail to use. Among the 16 evaluated models, the highest overall accuracy is 61.2%, compared with a mean of 98.3% across two human evaluators. At paired decision points, correct traceback is not associated with correct next-action prediction. Adding action logs yields a gain of 1.6 points, whereas cause-and-consequence annotations yield an additional gain of 13.0 points on top of that. These results suggest that extracting causal information from past events and converting it into constraints on current actions remains a key difficulty for long-horizon embodied agents. Project Page: https://zhaoalexgoat.github.io/EMBER-Bench/

Figures & tables

Figure 1: Overview of EMBER-Bench. (a) Four types of memory required for long-horizon task scenarios. (b) Paired next-action prediction and causal traceback assess history-grounded action selection and identification of the historical event supporting a given action.

Benchmark	Data	Hist. ctrl.	L1	L2	L3	L4	Focus
MEMENTO ( Kwon et al., 2026 )	Sim.	Task const.	–	–	–	–	Personalized object semantics and user routines.
FindingDory ( Yadav et al., 2026 )	Sim.	Task const.	$\checkmark$	–	–	–	Conditional, ordered, multi-goal memory control.
MIKASA-Robo ( Cherepanov et al., 2026 )	Sim.+real	Task const.	$\checkmark$	–	$\triangle$	–	Object, spatial, sequence, and capacity memory.
RoboMME ( Dai et al., 2026 )	Sim.+real	Task const.	$\checkmark$	$\triangle$	$\triangle$	$\triangle$	Temporal, spatial, object, and procedural memory.
RMBench ( Chen et al., 2026a )	Sim.+real	Task const.	$\checkmark$	–	–	–	Manipulation tasks organized by memory complexity.
RoboMemArena ( Lei et al., 2026a )	Sim.+real	Task const.	$\checkmark$	–	$\triangle$	–	Memory-driven robotic subtasks with keyframes.

Table 1: Comparison with embodied-memory and long-horizon embodied-reasoning benchmarks.

Figure 2: The process of EMBER-Bench construction. A human-led, model-assisted workflow covers recording, filtering, annotation, and bidirectional QA construction. Iterative quality control checks memory dependence and answer uniqueness, producing traceable benchmark samples.

Figure 3: Data statistics and distribution of EMBER-Bench. (a) The hierarchical distribution of scenes and activities; (b) The distribution of video input duration based on all QA pairs; (c) The frequency of the 16 most commonly used types of action words; (d) The number of the two types of problems in L1 to L4; (e) The distribution of task length of L1 to L4 (segments).

Table 2: Accuracy (%) under the video-history setting

V

P

denotes next-action prediction and

C

causal traceback. Models are grouped into closed-source and open-source models and sorted by overall accuracy within each group. The Human row averages two evaluators ( Appendix E.7 ) and is excluded from ranking. The best model result in each column is shaded and bold; the second best is underlined. Unanswered items after retries are scored incorrect. Random-choice accuracy is 25%.

Figure 4: Temporal span, information ablation, and paired prediction–traceback outcomes. (a–b)

P

and

C

accuracy across tertiles of total history duration and event-to-decision interval. (c–d)

P

accuracy of six models under

V

V{+}L

, and

V{+}L{+}E

inputs, and gains by decision type. (e–f) Joint

P

C

outcomes and

P

accuracy conditioned on

C

correctness. Details: Appendix E.2 .

Model	Direct (%)	Causal CoT (%)	Net gain, correct- $C$ group (pp)
Gemini 3.8 Flash	47.0	55.0	+13.0
Qwen3.8 Flash	31.1	34.4	+4.5
Qwen3.8 27B	31.8	27.8	0.0

Table 3: Decision gains from causal CoT guidance. The final column is the net gain within the correct-

C

group (fixed by main-evaluation results),

100\times(\text{corrected}-\text{newly wrong})/(\text{group size})

, comparing CoT against direct answering on the same 151 questions; it is not the overall direct-to-CoT change.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Field	Guiding question	When to populate	Example values
execution_stage	Where does the action occur in the accident timeline?	Every action.	before , during , repaying , after .
obligation_related	Which accident does the action help remedy?	Only in the repaying stage; false otherwise.	“Replenish the milk” or false .
accident_cause	Which action caused the accident?	Only in the during stage; false otherwise.	“Turning around caused the insulated cup to hit the counter edge.”
accident_consequence	What consequences did the accident cause?	Only in the during stage; false otherwise.	“Cup damage”; “Raw materials contaminated”, etc.

Appendix

Table 4: Action-level annotation fields. Each action receives an execution-stage label. The other fields are populated only at the specified stage and are set to the Boolean value false otherwise.

Action	execution_stage	Explanation
Cut cheese normally.	before	No accident has occurred yet.
Slice a cucumber (accidentally knocking some slices onto the floor).	during	The first accident occurs.
Pick up the fallen cucumber slices.	repaying	Attempt to address the first accident.
Throw the cucumber slices into the trash bin.	repaying	—
Get a cloth, wipe up the water, and put the cloth back.	repaying	Fulfill the floor-cleaning obligation.
Go to the refrigerator, get a fresh cucumber, and slice it again.	repaying	Fulfill the obligation to redo the preparation.

Appendix

Table 5: Example execution-stage annotations for a sequence of food-preparation actions. The after stage begins only when all remedial actions have been completed.

Model	Correct / 548	Accuracy (%)
Gemini 3.8 Flash	166	30.29
Gemini 3.1 Pro	147	26.82
Qwen3.8 Omni Flash	146	26.64
Qwen3.8 Max	146	26.64
Qwen3-VL 32B	145	26.46
Qwen3.8 Flash	143	26.09

Appendix

Table 6: Model accuracy under the setting with goal, question, options, and the current frame.

Analysis	Models	Questions / responses
Main leaderboard (Table 2 )	16	699 per model (548 $P$ , 151 $C$ )
Temporal analysis (Figure 4 (a–b))	16	151 matched $P$ – $C$ pairs
Information ablation (Figure 4 (c–d))	6	151 $P$ questions $\times$ 3 inputs
Joint analysis (Figure 4 (e–f))	16	151 matched pairs; 2,416 paired responses
Causal CoT (Table 3 )	3	The same 151 $P$ questions

Appendix

Table 7: Samples used in each analysis.

Variable	Group	Range (s)
Total history duration	Short	21.17–83.50
	Medium	84.57–134.63
	Long	136.60–594.80
Event-to-decision interval	Short	0–7.97
	Medium	8.03–30.37
	Long	30.57–163.03

Appendix

Table 8: Observed ranges of the temporal tertiles.

Outcome	Predictor (per doubling)	Coefficient (pp)	95% CI
$P$	Total history duration	$+11.7$	[ $+5.5$ , $+17.5$ ]
$P$	Interval $+$ 1 s	$+1.7$	[ $-0.6$ , $+4.4$ ]
$C$	Total history duration	$+3.9$	[ $-4.0$ , $+11.9$ ]
$C$	Interval $+$ 1 s	$-3.7$	[ $-6.7$ , $-0.9$ ]

Appendix

Table 9: Temporal associations after controlling for level.

Condition	Appended input (excerpt)
$V{+}L$	{"subtask": 18, "action": "The arm hits the clean apple on the table", "start_sec": 80.6, "stop_sec": 82.067, "left_hand": false, "right_hand": false, "navigation": false}
$V{+}L{+}E$	The same log, plus {"subtask": 18, "action": "The arm hits the clean apple on the table", "accident_cause": "The arm touches a clean apple on the tabletop.", "accident_consequence": "The apple rolled onto the ground."}

Appendix

Table 10: Appended input for one question ( Long-Horizon_L2_task3_q1 ; excerpt of subtask 18).

Model	$V$ (%)	$V{+}L$ (%)	$V{+}L{+}E$ (%)	$\Delta L$ (pp, 95% CI)	$\Delta E$ (pp, 95% CI)
Gemini 3.8 Flash	49.7	53.0	72.8	$+3.3$ [ $-4.6$ , $+11.3$ ]	$+19.9$ [ $+12.9$ , $+27.2$ ]
Qwen3.5 397B-A17B	37.7	36.4	53.6	$-1.3$ [ $-7.2$ , $+4.3$ ]	$+17.2$ [ $+10.2$ , $+24.5$ ]
Qwen3.8 Omni Flash	35.8	38.4	55.6	$+2.6$ [ $-4.0$ , $+9.4$ ]	$+17.2$ [ $+11.0$ , $+23.7$ ]
Qwen3.8 27B	31.8	33.1	43.0	$+1.3$ [ $-4.2$ , $+7.0$ ]	$+9.9$ [ $+3.9$ , $+16.1$ ]
Qwen3.8 Flash	31.1	32.5	42.4	$+1.3$ [ $-4.7$ , $+7.6$ ]	$+9.9$ [ $+2.7$ , $+17.2$ ]
Qwen3.5 122B-A10B	30.5	33.1	37.1	$+2.6$ [ $-2.1$ , $+7.4$ ]	$+4.0$ [ $-1.2$ , $+9.3$ ]

Appendix

Table 11:

P

accuracy under the three inputs and paired gains.

Model	Corrected / newly wrong	Net gain (pp, 95% CI)	Questions	Corrected / newly wrong
	All 151 questions		$C$ -correct group
Gemini 3.8 Flash	21 / 9	$+7.9$ [ $+1.4$ , $+14.7$ ]	108	19 / 5
Qwen3.8 Flash	18 / 13	$+3.3$ [ $-3.6$ , $+10.6$ ]	67	7 / 4
Qwen3.8 27B	13 / 19	$-4.0$ [ $-12.1$ , $+3.6$ ]	65	8 / 8

Appendix

Table 12: Answer changes under causal CoT.

Model	Task and key event	Direct answer (incorrect)	CoT output (correct)
Gemini 3.8 Flash	Wipe a tablet; the wipe packet slips while being torn open and the wipes scatter on the floor (L2).	Move over to the front of the table.	“Assessment: When opening the packet of wipes, it tore aggressively and the wipes scattered onto the floor by the shelf. The user just discarded the wrapper into the trash can and now needs to collect the wipes. Final Answer: [B]” (Go over to where the wipes are scattered.)
Gemini 3.8 Flash	Prepare a fruit plate; another person takes some grapes, leaving fewer than 12 (L3).	Carry the fruit plate onward into the living room.	“Assessment: Another person took grapes from the plate while the user was wiping their hands, leaving fewer than the required 12 grapes. More grapes must be added to fulfill the instruction. Final Answer: D” (Add grapes onto the fruit plate to fill it.)
Qwen3.8 Flash	Spray anti-mite spray on bedding; the robot arm knocks over the balcony trash can and garbage scatters (L2).	Carry on straightening the quilt out for drying.	“Assessment: The task of spraying the bedding is complete, and the person has moved to hang the quilt on a rack. A black trash can has fallen over with garbage scattered on the floor, which is an obstacle in the current path that needs to be addressed before continuing. Final Answer: C” (Step over to where the trash can fell.)
Qwen3.8 Flash	Boil a kettle of water; the kettle is switched off and moved onto the counter before the water boils (L3).	Raise the kettle lid without lifting the kettle.	“Assessment: The kettle was just moved off the heating base to the right side of the counter, so it is no longer positioned for boiling. To resume the task, the next step is to pick up the kettle from its current location on the right. Final Answer: C” (Pick up the kettle from the counter’s right.)
Qwen3.8 27B	Get a bottle of soda; the bottle had fallen earlier, and foam overflows after the cap is loosened (L2).	Turn the bottle cap to release trapped gas.	“Assessment: The bottle was opened and is now foaming over the top, with foam visible on the neck. The most logical next step to clean up this mess before proceeding is to wipe the foam off the bottle and counter. Final Answer: C” (Wipe the foam off the bottle and counter.)
Qwen3.8 27B	Clear standing water in a bathroom; another person walks in and out of the shower area, leaving wet footprints on the mopped floor (L3).	Mop the damp patch beside the bathroom cabinet.	“Assessment: A second person entered the bathroom and left new wet footprints on the floor, which disrupted the cleaning. The most logical next step is to mop up these new wet footprints one by one. Final Answer: B” (Mop up the new wet footprints one by one.)

Appendix

Table 13: Wrong-to-right examples under causal CoT.

Evaluator	$P$ errors (L1/L2/L3/L4)	L1	L2	L3	L4	All	$C$ errors	$C$ (%)	Overall (%)
		$P$ accuracy (%)
Evaluator 1	13 (0/4/3/6)	100.0	96.5	97.7	94.1	97.6	0	100.0	98.1
Evaluator 2	11 (0/3/3/5)	100.0	97.3	97.7	95.1	98.0	0	100.0	98.4
Mean	12	100.0	96.9	97.7	94.6	97.8	0	100.0	98.3

Appendix

Table 14: Human evaluation results.

Explore similar work

Sep 23, 2026cs.CV

EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks

Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounter change. Yet current agents struggle to maintain such memory reliably. Our analysis traces this limitation to four key deficiencies: weak fine-grained visual memory, unreliable dynamic world-state tracking, failing to record world state revealed by interaction outcomes, and limited generalization from prior experience. However, existing benchmarks do not directly assess these memory capabilities during long-horizon embodied interaction. To address this gap, we introduce EmbodiedMemory-Bench (EMem-Bench), comprising 2,554 interactive episodes across four task families. EMem-Bench requires agents to build and update memory from interaction history, then use it to complete a later task by acting in the environment. We further present Embodied-Memorizer (EMem), an external memory system that organizes embodied experience into spatial, event, and scene memories. We also train EMem-8B, an 8B policy that manages and uses these memories. We evaluate a diverse range of open-source and proprietary MLLMs and representative multimodal memory systems. Results show that current models remain weak and uneven across the four challenges. Under matched backbones, EMem achieves the best overall performance among the evaluated memory systems and improves both open-source and proprietary models, while EMem-8B further improves over its backbone. Project page: https://zju-omniai.github.io/EmbodiedMemoryBench/

Lizhou Liang, Xinyu Zhong, Miao Pan +7

Zhejiang University · Central South University · Institute of Software, Chinese Academy of Sciences

Embodied Agents Interaction History Visual Memory

Ask PDF

Jun 1, 2026cs.AI

Planning Takes More Than Token Prediction: Causal Plan for Benchmarking and Building Physically Grounded Embodied Reasoners

Current benchmarks for embodied vision-language planning inadvertently favor linguistic next-token prediction over physically grounded next-state reasoning. This rewards models that mimic statistical language priors rather than track true causal dependencies, reducing complex physical planning to shallow sequence modeling. Hence, achieving genuine physical autonomy requires a fundamental shift from linguistically grounded token prediction toward physically grounded causal reasoning. To this end, we introduce Causal-Plan-Bench, a high-fidelity diagnostic suite spanning four causal dimensions, curated via multi-stage verification. To endow models with this capability, a four-stage annotation pipeline extracts structured interaction records from egocentric videos to construct Causal-Plan-1M, a dense million-scale corpus of explicit causal reasoning traces. Extensive evaluation reveals a striking gap: leading models struggle to demonstrate genuine physical agency -- even GPT-6-astra scores only 43.04. In contrast, our tailored training recipe enables Causal Planner to internalize the complex physical logic required for accurate next-state estimation. Built upon Qwen3-VL-8B, Causal Planner raises its backbone's score from 33.23 to 45.28, a 36.3% relative gain, and improves on three external benchmarks without benchmark-specific adaptation. We further observe an empirical Causal-Supervision Scaling Trend. Paired no-vision controls also reveal substantial visual dependence, while cross-judge comparisons and human scoring assess the reliability of automated evaluation. More importantly, we initiate the first effort to turn agents from superficial token predictors into physically grounded causal reasoners, bridging language modeling and world modeling.

Zheng Lu, Mingqi Gao, Qinlei Xie +7

Tsinghua University · Microsoft Research Asia · MBZUAI

Topic pending

Ask PDF

Sep 19, 2026cs.CV

CausalWM: Causal Chain-of-Thought Reasoning for Embodied World Model

Embodied world models learn to predict future physical dynamics from visual observations and control signals, where physical knowledge is implicitly entangled within latent representations. We introduce CausalWM, a 16B embodied world model that performs explicit causal chain-of-thought reasoning before future video prediction. CausalWM organizes useful variables into a reasoning trajectory, allowing the model to progressively capture causal dependencies underlying physical evolution. To train CausalWM, we collect 31K hours embodied data and develop a three-stage paradigm consisting of large-scale video pre-training, causal CoT mid-training, and multi-objective RL post-training. Despite using only a limited set of supervised CoT variables, CausalWM exhibits emergent in-context learning capabilities, enabling contextual visual feature guidance and efficient few-step generation. CausalWM achieves state-of-the-art performance across language-conditioned, action-conditioned, single-view and multi-view benchmarks, including Top-1 performance on TriWorldBench leaderboard.

Ziming Xu, Shuang Liang, Ruobing Han +10

Aether AI · University of California, San Diego · Vanderbilt University

Efficient World-Action Model Future Video Prediction Causal Chain-of-Thought Reasoning Reinforcement Learning Post-Training

Ask PDF