A-MBER: Affective Memory Benchmark for Emotion Recognition

Deliang Wen; Ke Sun; Yu Wang

cs.AIApr 8, 2026

A-MBER: Affective Memory Benchmark for Emotion Recognition

Authors: Deliang Wen, Ke Sun, Yu Wang

Abstract

AI assistants that interact with users over time need to interpret the user's current emotional state in order to respond appropriately and personally. However, this capability remains insufficiently evaluated. Existing emotion datasets mainly assess local or instantaneous affect, while long-term memory benchmarks focus largely on factual recall, temporal consistency, or knowledge updating. As a result, current resources provide limited support for testing whether a model can use remembered interaction history to interpret a user's present affective state. We introduce A-MBER, an Affective Memory Benchmark for Emotion Recognition, to evaluate this capability. A-MBER focuses on present affective interpretation grounded in remembered multi-session interaction history. Given an interaction trajectory and a designated anchor turn, a model must infer the user's current affective state, identify historically relevant evidence, and justify its interpretation in a grounded way. The benchmark is constructed through a staged pipeline with explicit intermediate representations, including long-horizon planning, conversation generation, annotation, question construction, and final packaging. It supports judgment, retrieval, and explanation tasks, together with robustness settings such as modality degradation and insufficient-evidence conditions. Experiments compare local-context, long-context, retrieved-memory, structured-memory, and gold-evidence conditions within a unified framework. Results show that A-MBER is especially discriminative on the subsets it is designed to stress, including long-range implicit affect, high-dependency memory levels, trajectory-based reasoning, and adversarial settings. These findings suggest that memory supports affective interpretation not simply by providing more history, but by enabling more selective, grounded, and context-sensitive use of past interaction

Figures & tables

Figure 1.1: Motivating contrast between local reading and history-grounded present affect interpretation. The figure illustrates why the same anchor turn may appear neutral or weakly emotional in isolation, yet become interpretable once earlier events, recurring triggers, and interaction trajectory are taken into account.

Figure 2.1: Positioning of A-MBER relative to existing evaluation spaces. The figure visualizes the gap between resources that mainly test local or instantaneous emotion understanding and benchmarks that emphasize long-term memory for factual or temporal reasoning, locating A-MBER at their intersection around history-grounded present affect interpretation.

Dimension	LongMemEval	LoCoMo	LoCoMo-Plus	A-MBER
Target	Interactive memory	Conversational memory	Cognitive memory	Affective memory
Present-state interpretation	Limited	Limited	Yes	Yes
History beyond local cues	Partial	Partial	Yes	Yes
Affective trajectory	No	No	Partial	Yes
Main difficulty	Memory operations	Long-span retention	Implicit constraints	Affective event logic

Table 1: Comparison with representative long-term memory benchmarks.

Figure 3.1: Anchor-turn-centered benchmark unit schema in A-MBER. Each evaluation item is organized around a designated anchor turn together with its accessible interaction history, task-specific question, gold evidence, and expected output, so that present-state interpretation remains explicitly grounded in observable dialogue context.

Figure 3.2: Primary benchmark construction pipeline of A-MBER. The pipeline moves from persona and long-horizon planning to conversation generation, annotation, question construction, post-generation curation, and final benchmark-unit packaging, making the connection between scenario design, observable interaction, and evaluable supervision explicit.

Figure 4.1: Three primary task families in A-MBER. Judgment evaluates present-state interpretation at the anchor turn, retrieval evaluates whether the model can locate the historically relevant supporting turns, and explanation evaluates whether the interpretation can be justified in a grounded way.

Figure 4.2: Layered benchmark composition of A-MBER. The figure separates the core layer, diagnostic layer, and stress-test layer so that the main target of long-horizon affective interpretation remains distinct from auxiliary analyses of reasoning difficulty and robustness.

Figure 4.3: Overview of memory levels and reasoning structures in A-MBER. The figure summarizes how items vary in historical dependency and in reasoning form, from relatively local or direct cases to strongly history-dependent, multi-hop, and trajectory-based interpretations.

Task family	Answer format	Scoring
Judgment	Multiple choice / label	Exact match
Retrieval	Evidence turns	Set match / turn-level F1
Explanation	Open-form text	LLM judge
Insufficient-evidence	Label / open-form	Exact match or calibration-sensitive judgment

Table 2: Task-specific scoring rules in A-MBER.

Figure 5.1: Evaluation settings used for system comparison in A-MBER. The figure contrasts no-memory, long-context, retrieved-memory, structured-memory, and gold-evidence conditions, making explicit how each system is given different forms of historical access while keeping the benchmark units themselves fixed.

System	Judgment	Retrieval	Explanation
No-Memory Baseline	0.34	0.29	0.31
Long-Context Baseline	0.47	0.41	0.44
Retrieved-Memory Baseline	0.58	0.54	0.53
Red Bear AI Memory	0.69	0.66	0.65
Gold-Evidence	0.81	0.79	0.77

Table 3: Results across compared systems on A-MBER.

System	Implicit	Explicit	Instant	Long Fact	Near Fact
No-Memory	0.18	0.31	0.45	0.27	0.49
Long-Context	0.34	0.46	0.57	0.41	0.60
Retrieved-Memory	0.49	0.58	0.66	0.54	0.68
Red Bear AI Memory	0.65	0.69	0.74	0.67	0.76
Gold-Evidence	0.79	0.81	0.84	0.78	0.85

Table 4: Results by content type. “Implicit” denotes long-range implicit affect; “Explicit” denotes long-range explicit or semi-explicit affect.

System	Level 0	Level 1	Level 2	Level 3
No-Memory	0.52	0.39	0.28	0.16
Long-Context	0.62	0.50	0.40	0.30
Retrieved-Memory	0.70	0.61	0.53	0.43
Red Bear AI Memory	0.78	0.71	0.64	0.58
Gold-Evidence	0.84	0.80	0.76	0.72

Table 5: Results by memory dependency level.

System	Direct / Single-hop	Multi-hop	Trajectory	Conflict / Complex
No-Memory	0.49	0.27	0.21	0.24
Long-Context	0.60	0.39	0.33	0.36
Retrieved-Memory	0.69	0.53	0.47	0.50
Red Bear AI Memory	0.77	0.68	0.63	0.60
Gold-Evidence	0.84	0.79	0.75	0.72

Table 6: Results by reasoning structure.

System	Standard	Modality-Missing	Modality-Ambiguous	Adversarial
No-Memory	0.39	0.30	0.28	0.22
Long-Context	0.52	0.41	0.38	0.32
Retrieved-Memory	0.62	0.52	0.49	0.43
Red Bear AI Memory	0.73	0.64	0.60	0.54
Gold-Evidence	0.82	0.75	0.72	0.66

Table 7: Results by robustness condition.

System	Insufficient Evidence	Pseudo-Relevant History	Other Adversarial
No-Memory	0.19	0.24	0.28
Long-Context	0.31	0.38	0.42
Retrieved-Memory	0.43	0.51	0.55
Red Bear AI Memory	0.56	0.63	0.66
Gold-Evidence	0.68	0.75	0.77

Table 8: Results on adversarial and insufficient-evidence subsets.

Dimension	Single-Agent	Multi-Agent
Schema stability	High	Moderate
Long-horizon consistency	High	Moderate
Character differentiation	Moderate	High
Interaction spontaneity	Moderate	High
Question derivation reliability	High	Moderate
Overall production suitability	Primary	Auxiliary

Table 9: Comparison of single-agent and multi-agent construction regimes.

Explore similar work

Mar 18, 2026

Memory Bear AI Memory Science Engine for Multimodal Affective Intelligence: A Technical Report

Affective judgment in real interaction is rarely a purely local prediction problem. Emotional meaning often depends on prior trajectory, accumulated context, and multimodal evidence that may be weak, noisy, or incomplete at the current moment. Although multimodal emotion recognition (MER) has improved the integration of text, speech, and visual signals, many existing systems remain optimized for short-range inference and provide limited support for persistent affective memory, long-horizon dependency modeling, and robust interpretation under imperfect input. This technical report presents the Memory Bear AI Memory Science Engine, a memory-centered framework for multimodal affective intelligence. Instead of treating emotion as a transient output label, the framework models affective information as a structured and evolving variable within a memory system. It organizes processing through structured memory formation, working-memory aggregation, long-term consolidation, memory-driven retrieval, dynamic fusion calibration, and continuous memory updating. At its core, multimodal signals are transformed into structured Emotion Memory Units (EMUs), enabling affective information to be preserved, reactivated, and revised across interaction horizons. Experimental results show consistent gains over comparison systems across benchmark and business-grounded settings, with stronger accuracy and robustness, especially under noisy or missing-modality conditions. The framework offers a practical step from local emotion recognition toward more continuous, robust, and deployment-relevant affective intelligence.

Deliang Wen, Ke Sun, Yu Wang

Affective Computing Emotion Recognition Multimodal Emotion Recognition Multimodal Memory

Ask PDF

Feb 27, 2026

MemEmo: Evaluating Emotion in Memory Systems of Agents

Memory systems address the challenge of context loss in Large Language Model during prolonged interactions. However, compared to human cognition, the efficacy of these systems in processing emotion-related information remains inconclusive. To address this gap, we propose an emotion-enhanced memory evaluation benchmark to assess the performance of mainstream and state-of-the-art memory systems in handling affective information. We developed the Human-Like Memory Emotion (HLME) dataset, which evaluates memory systems across three dimensions: emotional information extraction, emotional memory updating, and emotional memory question answering. Experimental results indicate that none of the evaluated systems achieve robust performance across all three tasks. Our findings provide an objective perspective on the current deficiencies of memory systems in processing emotional memories and suggest a new trajectory for future research and system optimization.

Peng Liu, Zhen Tao, Jihao Zhao +5

School of Information, Renmin University of China, Beijing, China · China Telecom Research Institute · MemTensor (Shanghai) Technology

Memory-Augmented Agents Memory Agent Benchmarks Emotion Representation in Language Models

Ask PDF

May 26, 2026

ENPMR-Bench: Benchmarking Proactive Memory Retrieval for Emotional Support Agents

Memory-augmented language agents are increasingly deployed in affective applications such as emotional support, where understanding and responding to users' latent emotional needs is critical. However, existing research often treats memory as a tool for factual retrieval, overlooking its role in shaping users' emotional experiences. In this work, we introduce ENPMR-Bench, a benchmark for evaluating Emotional Need-aware Proactive Memory Retrieval (ENPMR), a core capability that enables agents to infer users' latent emotional needs and proactively retrieve appropriate memories to support empathetic interaction. Grounded in Maslow's hierarchy of needs, ENPMR-Bench includes over 1,800 memory-augmented dialogues and defines structured mappings between emotional needs and supportive memory types. Experimental results demonstrate that current retrieval paradigms, including both embedding-based and LLM-driven approaches, exhibit substantial deficiencies, with empathy scores significantly lagging behind golden memory conditions. While chain-of-thought prompting improves the alignment between inferred emotional needs and retrieved memories to some extent, a notable performance gap remains. Together, these findings reveal critical limitations in current agents and outline directions for advancing personalized emotional support through need-sensitive memory retrieval.

Xing Fu, Yulin Hu, Mengtong Ji +5

Research Center for Social Computing and Interactive Robotics Harbin Institute of Technology, Heilongjiang, China

Emotional Support Conversation AI Agent Benchmarks Memory-Augmented Language Model Agents

Ask PDF