A-MBER: Affective Memory Benchmark for Emotion Recognition

Deliang Wen; Ke Sun; Yu Wang

cs.AIApr 8, 2026

A-MBER: Affective Memory Benchmark for Emotion Recognition

Authors: Deliang Wen, Ke Sun, Yu Wang

Abstract

AI assistants that interact with users over time need to interpret the user's current emotional state in order to respond appropriately and personally. However, this capability remains insufficiently evaluated. Existing emotion datasets mainly assess local or instantaneous affect, while long-term memory benchmarks focus largely on factual recall, temporal consistency, or knowledge updating. As a result, current resources provide limited support for testing whether a model can use remembered interaction history to interpret a user's present affective state. We introduce A-MBER, an Affective Memory Benchmark for Emotion Recognition, to evaluate this capability. A-MBER focuses on present affective interpretation grounded in remembered multi-session interaction history. Given an interaction trajectory and a designated anchor turn, a model must infer the user's current affective state, identify historically relevant evidence, and justify its interpretation in a grounded way. The benchmark is constructed through a staged pipeline with explicit intermediate representations, including long-horizon planning, conversation generation, annotation, question construction, and final packaging. It supports judgment, retrieval, and explanation tasks, together with robustness settings such as modality degradation and insufficient-evidence conditions. Experiments compare local-context, long-context, retrieved-memory, structured-memory, and gold-evidence conditions within a unified framework. Results show that A-MBER is especially discriminative on the subsets it is designed to stress, including long-range implicit affect, high-dependency memory levels, trajectory-based reasoning, and adversarial settings. These findings suggest that memory supports affective interpretation not simply by providing more history, but by enabling more selective, grounded, and context-sensitive use of past interaction

Figures & tables

Figure 1.1: Motivating contrast between local reading and history-grounded present affect interpretation. The figure illustrates why the same anchor turn may appear neutral or weakly emotional in isolation, yet become interpretable once earlier events, recurring triggers, and interaction trajectory are taken into account.

Figure 2.1: Positioning of A-MBER relative to existing evaluation spaces. The figure visualizes the gap between resources that mainly test local or instantaneous emotion understanding and benchmarks that emphasize long-term memory for factual or temporal reasoning, locating A-MBER at their intersection around history-grounded present affect interpretation.

Dimension	LongMemEval	LoCoMo	LoCoMo-Plus	A-MBER
Target	Interactive memory	Conversational memory	Cognitive memory	Affective memory
Present-state interpretation	Limited	Limited	Yes	Yes
History beyond local cues	Partial	Partial	Yes	Yes
Affective trajectory	No	No	Partial	Yes
Main difficulty	Memory operations	Long-span retention	Implicit constraints	Affective event logic

Table 1: Comparison with representative long-term memory benchmarks.

Figure 3.1: Anchor-turn-centered benchmark unit schema in A-MBER. Each evaluation item is organized around a designated anchor turn together with its accessible interaction history, task-specific question, gold evidence, and expected output, so that present-state interpretation remains explicitly grounded in observable dialogue context.

Figure 3.2: Primary benchmark construction pipeline of A-MBER. The pipeline moves from persona and long-horizon planning to conversation generation, annotation, question construction, post-generation curation, and final benchmark-unit packaging, making the connection between scenario design, observable interaction, and evaluable supervision explicit.

Figure 4.1: Three primary task families in A-MBER. Judgment evaluates present-state interpretation at the anchor turn, retrieval evaluates whether the model can locate the historically relevant supporting turns, and explanation evaluates whether the interpretation can be justified in a grounded way.

Figure 4.2: Layered benchmark composition of A-MBER. The figure separates the core layer, diagnostic layer, and stress-test layer so that the main target of long-horizon affective interpretation remains distinct from auxiliary analyses of reasoning difficulty and robustness.

Figure 4.3: Overview of memory levels and reasoning structures in A-MBER. The figure summarizes how items vary in historical dependency and in reasoning form, from relatively local or direct cases to strongly history-dependent, multi-hop, and trajectory-based interpretations.

Task family	Answer format	Scoring
Judgment	Multiple choice / label	Exact match
Retrieval	Evidence turns	Set match / turn-level F1
Explanation	Open-form text	LLM judge
Insufficient-evidence	Label / open-form	Exact match or calibration-sensitive judgment

Table 2: Task-specific scoring rules in A-MBER.

Figure 5.1: Evaluation settings used for system comparison in A-MBER. The figure contrasts no-memory, long-context, retrieved-memory, structured-memory, and gold-evidence conditions, making explicit how each system is given different forms of historical access while keeping the benchmark units themselves fixed.

System	Judgment	Retrieval	Explanation
No-Memory Baseline	0.34	0.29	0.31
Long-Context Baseline	0.47	0.41	0.44
Retrieved-Memory Baseline	0.58	0.54	0.53
Red Bear AI Memory	0.69	0.66	0.65
Gold-Evidence	0.81	0.79	0.77

Table 3: Results across compared systems on A-MBER.

System	Implicit	Explicit	Instant	Long Fact	Near Fact
No-Memory	0.18	0.31	0.45	0.27	0.49
Long-Context	0.34	0.46	0.57	0.41	0.60
Retrieved-Memory	0.49	0.58	0.66	0.54	0.68
Red Bear AI Memory	0.65	0.69	0.74	0.67	0.76
Gold-Evidence	0.79	0.81	0.84	0.78	0.85

Table 4: Results by content type. “Implicit” denotes long-range implicit affect; “Explicit” denotes long-range explicit or semi-explicit affect.

System	Level 0	Level 1	Level 2	Level 3
No-Memory	0.52	0.39	0.28	0.16
Long-Context	0.62	0.50	0.40	0.30
Retrieved-Memory	0.70	0.61	0.53	0.43
Red Bear AI Memory	0.78	0.71	0.64	0.58
Gold-Evidence	0.84	0.80	0.76	0.72

Table 5: Results by memory dependency level.

System	Direct / Single-hop	Multi-hop	Trajectory	Conflict / Complex
No-Memory	0.49	0.27	0.21	0.24
Long-Context	0.60	0.39	0.33	0.36
Retrieved-Memory	0.69	0.53	0.47	0.50
Red Bear AI Memory	0.77	0.68	0.63	0.60
Gold-Evidence	0.84	0.79	0.75	0.72

Table 6: Results by reasoning structure.

System	Standard	Modality-Missing	Modality-Ambiguous	Adversarial
No-Memory	0.39	0.30	0.28	0.22
Long-Context	0.52	0.41	0.38	0.32
Retrieved-Memory	0.62	0.52	0.49	0.43
Red Bear AI Memory	0.73	0.64	0.60	0.54
Gold-Evidence	0.82	0.75	0.72	0.66

Table 7: Results by robustness condition.

System	Insufficient Evidence	Pseudo-Relevant History	Other Adversarial
No-Memory	0.19	0.24	0.28
Long-Context	0.31	0.38	0.42
Retrieved-Memory	0.43	0.51	0.55
Red Bear AI Memory	0.56	0.63	0.66
Gold-Evidence	0.68	0.75	0.77

Table 8: Results on adversarial and insufficient-evidence subsets.

Dimension	Single-Agent	Multi-Agent
Schema stability	High	Moderate
Long-horizon consistency	High	Moderate
Character differentiation	Moderate	High
Interaction spontaneity	Moderate	High
Question derivation reliability	High	Moderate
Overall production suitability	Primary	Auxiliary

Table 9: Comparison of single-agent and multi-agent construction regimes.

Explore similar work

Memory Bear AI Memory Science Engine for Multimodal Affective Intelligence: A Technical Report
Mar 18, 2026Deliang Wen, Ke Sun, Yu WangAffective Computing Emotion Recognition
MemEmo: Evaluating Emotion in Memory Systems of Agents
Feb 27, 2026Peng Liu, Zhen Tao, Jihao Zhao +5Memory-Augmented Agents Memory Agent Benchmarks
School of Information, Renmin University of China, Beijing, China · China Telecom Research Institute · MemTensor (Shanghai) Technology
ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents
May 26, 2026Xing Fu, Yulin Hu, Mengtong Ji +5Emotional Support Conversation AI Agent Benchmarks
Research Center for Social Computing and Interactive Robotics Harbin Institute of Technology, Heilongjiang, China

Mar 18, 2026J/K move · Enter open · S save

Memory Bear AI Memory Science Engine for Multimodal Affective Intelligence: A Technical Report

Deliang Wen, Ke Sun, Yu Wang

Affective Computing Emotion Recognition Multimodal Emotion Recognition Multimodal Memory

Open paper AskPDF arXiv

Affective judgment in real interaction is rarely a purely local prediction problem. Emotional meaning often depends on prior trajectory, accumulated context, and multimodal evidence that may be weak, noisy, or incomplete at the current moment. Although multimodal emotion recognition (MER) has improved the integration of text, speech, and visual signals, many existing systems remain optimized for short-range inference and provide limited support for persistent affective memory, long-horizon dependency modeling, and robust interpretation under imperfect input. This technical report presents the Memory Bear AI Memory Science Engine, a memory-centered framework for multimodal affective intelligence. Instead of treating emotion as a transient output label, the framework models affective information as a structured and evolving variable within a memory system. It organizes processing through structured memory formation, working-memory aggregation, long-term consolidation, memory-driven retrieval, dynamic fusion calibration, and continuous memory updating. At its core, multimodal signals are transformed into structured Emotion Memory Units (EMUs), enabling affective information to be preserved, reactivated, and revised across interaction horizons. Experimental results show consistent gains over comparison systems across benchmark and business-grounded settings, with stronger accuracy and robustness, especially under noisy or missing-modality conditions. The framework offers a practical step from local emotion recognition toward more continuous, robust, and deployment-relevant affective intelligence.

Abstract

Explore similar work

Memory Bear AI Memory Science Engine for Multimodal Affective Intelligence: A Technical Report

MemEmo: Evaluating Emotion in Memory Systems of Agents

ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents