Image memorability, i.e., how likely an image is to be remembered, has traditionally been studied in computer vision either as a passive prediction task, with models regressing a scalar score, or with generative methods altering the visual input to boost the image likelihood of being remembered. Yet, none of these paradigms supports users at capture time, when the crucial question is how to improve a photo memorability. We introduce the task of Memorability Feedback (MemFeed), where an automated model should provide actionable, human-interpretable guidance to users with the goal to enhance an image future recall. We also present MemCoach, the first approach designed to provide concrete suggestions in natural language for memorability improvement (e.g., "emphasize facial expression," "bring the subject forward"). Our method, based on Multimodal Large Language Models (MLLMs), is training-free and employs a teacher-student steering strategy, aligning the model internal activations toward more memorable patterns learned from a teacher model progressing along least-to-most memorable samples. To enable systematic evaluation on this novel task, we further introduce MemBench, a new benchmark featuring sequence-aligned photoshoots with annotated memorability scores. Our experiments, considering multiple MLLMs, demonstrate the effectiveness of MemCoach, showing consistently improved performance over several zero-shot models. The results indicate that memorability can not only be predicted but also taught and instructed, shifting the focus from mere prediction to actionable feedback for human creators.
Figures & tables
Figure 1: Given an input photo, memorability feedback aims to generate natural-language suggestions to guide users toward a more memorable shot. MemCoach provides memorability-aware feedback, effectively assisting users to capture memorable images.
Figure 2: Overview of MemBench generation and evaluation . Top: Data pipeline for constructing MemBench, including scene grouping, memorability regression, image ranking, and generation of actionable memorability-aware feedback. Bottom: Evaluation pipeline assessing feedback quality through editing-based memorability improvement and perplexity scoring.
Figure 3: MemBench statistics. Data analysis in terms of (a) most frequent words; (b) distribution of memorability scores for the least and most memorable images within each scene; (c) feedback length as measured by content words; and (d) categorization of atomic sub-actions, where the width of each chord indicates the frequency of co-occurrence between categories.
Model
Spearman Rank ( ↑ )
Inter-annotator ∗
0.68
Qwen2.5VL [ 5 ]
-0.06
InternVL3.5 [ 67 ]
-0.01
Idefics3 [ 39 ]
-0.07
LLaVA-OV [ 3 ]
0.08
Table 1: MLLMs lack memorability understanding. Left: Memorability prediction performance in terms of Spearman’s Rank Correlation (↑) . (∗) is reported from [ 32 ] . Right: Improvement ratio of zero-shot models with respect to the Editing baseline , marginal improvement is observed.
Figure 5: Overview of the proposed method. Ay (a) Contrasting data generation : paired samples are built by coupling the memorability-aware guidance of a teacher MLLM with the neutral responses of a student MLLM on the same scene; Ay (b) Steering vector extraction : activation differences between memorability-aware and neutral feedback are averaged to obtain a memorability steering vector capturing the latent shift toward effective suggestions for memorability; Ay (c) Inference with MLLM steering : the student activations are shifted using the memorability steering vector to produce improved, memorability-oriented feedback without additional training.
Model
Editing
Perplexity ( ↓ )
IR ( ↑ )
RM% ( ↑ )
Edit model
0.68
3.72
n.d.
Teacher oracles
LLaVA-OV [ 3 ]
0.74
5.93
5.73
Idefics3 [ 39 ]
0.80
9.84
29.21
Qwen2.5VL [ 5 ]
0.83
10.16
2.34
Table 2: Comparison with state-of-the-art models. MemFeed performance of Ay MemCoach when comparing to several Ay teacher oracle , Ay zero-shot and Ay aesthetics specialized MLLMs. MemCoach achieves the best results in the considered metrics. Best results in bold .
Model
Editing
Perplexity ( ↓ )
IR ( ↑ )
RM% ( ↑ )
LLaVA-OV [ 3 ]
0.70
5.87
7.58
MemCoach- LLaVA
0.73 (+4.29%)
5.04 (-14.14%)
14.05 (+85.36%)
Idefics3 [ 39 ]
0.73
6.64
20.19
MemCoach- Idefics
0.75 (+2.74%)
6.69 (+0.75%)
19.81 (-1.88%)
Qwen2.5VL [ 5 ]
0.68
4.26
10.23
Table 3: Generalization to different MLLMs. MemFeed performance of Ay MemCoach when applied to different architectures. MemCoach generalizes to different models, enhancing their ability to produce memorability feedback.
Figure 6: Common feedback patterns on source images. MemCoach favors symmetric and socially connected compositions, reflecting principles of human photography.
Figure 7: Qualitative feedback from MemCoach. For each source image (left), the model provides natural-language feedback (bottom) that is applied to produce the destination image (right). Each score represents the Relative Memorability (RM), indicating how suggested feedback affects memorability. MemCoach provides human-interpretable and actionable feedback that translates into semantic changes for overall improved memorability. Observed failure cases propose to remove out-of-context elements.
Model
Editing
Perplexity ( ↓ )
IR ( ↑ )
RM% ( ↑ )
Qwen-contrasting
0.73
5.68
5.13
Diff(Mean)
0.78
6.64
4.39
MemCoach (Ours)
0.80
7.21
4.99
Table 4: Ablation analysis. MemFeed performance of Ay MemCoach when ablating on the contrasting data teacher and steering vector computation.
Figure 8: Data efficiency. Top: performance vs number of training/contrasting samples. Bottom: performance vs coefficient α .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Fts dim
Normalization
Spearman Rank ( ↑ )
Human*
n.d.
n.d.
0.68
ViT-L-14-quickgelu
768
✓
0.73
ViT-L-14-quickgelu
768
×
0.81
ViT-SO400M-14-SigLIP-384
1152
✓
0.76
MemBench M
1152
×
0.82
Appendix
Table 5: Model M performance. Comparison of memorability predictor models trained on CLIP-like embeddings. We report feature dimensionality, whether feature normalization is applied, and Spearman’s rank correlation. The Ay used predictor model achieves the highest correlation among all evaluated variants. (*) is reported from [ 74 ] .
Model
Pretrained
Spearman Rank ( ↑ )
MemNet* [ 63 ]
LaMem
0.64
AMNet* [ 17 ]
LaMem
0.67
Human*
n.d.
0.68
ViTMem* [ 21 ]
LaMem
0.71
Henry [ 58 ]
LaMem + MemCat + SUN
0.72
Henry [ 58 ]
LaMem
0.74
Appendix
Table 6: Comparison with state-of-the-art memorability predictors. Spearman Rank correlation on the LaMem test set. Our model, trained on LaMem, MemCat, and SUN, achieves the highest correlation among all methods. (*) is reported from [ 74 ] .
Model
Layer
Coefficient
LM Depth
IR
MemCoach- Idefics
13
30
32
0.75
MemCoach- LLaVA
20
143
28
0.73
MemCoach- Qwen
12
26
28
0.74
MemCoach- InternVL
12
55
36
0.80
Appendix
Table 7: Optimal steering configuration across models. Best-performing layer index l and steering coefficient α obtained via hyperparameter tuning for each evaluated open-source multimodal model. We also report the language-model depth (LM Depth) and the corresponding IR score.
Figure 9: Human memory performance across three different image settings.
Figure 16
Model
Clearness
Relevance
Feasibility
Oracle
4.19
4.30
4.11
MemCoach
3.96
3.76
4.32
Appendix
Table 8: Feedback quality results.
Model
Edit Model
Mem Predictor
Qwen-IE
FLUX.2-k
VM
VM → MB
Edit model
0.69
0.68
0.64
0.69
Zero-shot baselines
GPT-5-Mini
0.78
0.80
0.73
0.76
LLaVA-OV
0.59
0.68
0.76
0.71
Idefics3
0.69
0.68
0.73
0.73
Appendix
Table 9: Performance of our framework on different settings . We report the results using different editing models for the evaluation and different memorability predictors.
Figure 10: A set of qualitative examples from MemBench.
Figure 11: A set of qualitative examples from MemBench.
Long-term memory is increasingly important for personalized AI agents, yet existing benchmarks and methods remain largely text-centric. Even when images are included, the user-specific information needed for later questions is typically recoverable from text alone, and most memory systems reduce image turns to generic captions. Yet images often carry personal information that text rarely states -- both explicit evidence, such as recurring user-associated entities, and implicit evidence, such as latent user facts inferred from visual or multimodal cues. We introduce a benchmark for personal visual memory that targets both forms of evidence, and propose VisualMem, a hybrid visual--text architecture that augments a text-memory backend with a structured personal visual memory module. Rather than collapsing images into captions, VisualMem uses conversational context to resolve identity, ownership, and durable user facts. Experiments show that VisualMem substantially outperforms prior memory systems on our benchmark while remaining competitive on standard text-memory benchmarks, indicating that personal visual memory is a distinct and important component of long-term memory for personalized AI agents.
Viet Nguyen, Thao Nguyen, Vishal M. Patel +1
Johns Hopkins University · University of Wisconsin-Madison · Adobe Research
Multimodal assistants answer questions from long-term memories that contain images. After retrieval, each retrieved image reaches the answering model either as pixels, at about a thousand visual tokens per image, or as a stored text proxy that often misses the detail the question asks about. We find that the benefit of pixels usually comes from one or two retrieved memories, and that it can be predicted before the answering model runs, without reading any full-resolution image. In PixelTriage, a plug-in placed after retrieval, a small model that does not generate text reads the dialogue, a short note and a thumbnail of each retrieved memory and predicts how much its pixels would add. It is trained on synthetic memory episodes labeled by a frozen 27B model that answers each question with and without each memory's pixels. With a 7B answering model, PixelTriage lies on the accuracy--cost frontier of M3Exam, DMV and MemEye and uses 11--23% of the visual tokens without a significant loss of accuracy. On DMV it answers 2.9 times faster than opening all images. It outperforms retrieval order and uniform down-sizing at equal budgets and transfers to other memory systems and to a 397B answering model.
Long-term memory is essential for multimodal agents that interact with users across sustained conversations. However, user memories are not always explicitly stated: they may also be implied by recurring background objects in images, ambient sounds in audio, or other peripheral multimodal cues. Existing benchmarks largely focus on text-only memory or explicit multimodal evidence, leaving implicit multimodal cues underexplored. We introduce CUE-Mem, a text-image-audio benchmark for evaluating long-term user memory from implicit cues. CUE-Mem contains 2,674 questions across explicit and implicit evidence settings and covers four tasks: Entity Recall, Long Pattern, Personalized Recommendation, and Answer Refusal. Across textualized memory systems, implicit performance remains far below oracle evidence, locating the main bottleneck in preserving and retrieving subtle cues rather than question answerability. Increasing caption detail recovers more of this evidence, but brings uneven gains and rapidly growing token costs, motivating native multimodal access. Yet native access does not uniformly resolve the bottleneck: evidence use depends strongly on the backbone, while multimodal indexing introduces substantial retrieval noise. CUE-Mem provides a testbed for memory systems that selectively retain, retrieve, and use subtle multimodal evidence.
Yulin Hu, Yanyan Zhao, Zimo Long +6
Harbin Institute of Technology · Huawei Technologies Co., Ltd.