People often make decisions by holistically reasoning over heterogeneous forms of personal information. In this paper, we study the relatively underexplored capability of multimodal personalized reasoning in multimodal large language models (MLLMs). To this end, we introduce MPRBench, a novel and comprehensive benchmark specifically designed to evaluate personalized reasoning across a wide range of tasks and real-world challenges. MPRBench consists of 12 sub-tasks and additionally supports interpretable analysis of representative error cases. Through extensive experiments, we find that current personalized MLLMs still struggle with personalized reasoning due to several key challenges, including inaccurate recognition of personalized concepts and sensitivity to irrelevant personal information. We hope that MPRBench will stimulate further research on personalized reasoning in MLLMs. Code and data are available at https://github.com/kkdh15/MPRBench
Figures & tables
Figure 1: Conceptual Overview and Problem Setup for Multimodal Personalized Reasoning.
Figure 2: Illustrative Examples and Statistics of MPRBench.
Figure 3: Overview of the Data Construction Pipeline for MPRBench.
Vision-specific Reasoning
Text-specific Reasoning
Multimodal
Method
predict
spatial
numeric
commsense
change
V-Avg
predict
numeric
multi-hop
T-Avg
predict
sentiment
numeric
history
M-Avg
Avg
Random Choice
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
25.0
Proprietary+Prompting
GPT-4o [ 1 ] +P
91.6
64.3
63.5
71.6
65.0
70.6
97.5
70.1
40.3
62.5
88.1
79.4
50.0
49.4
73.7
69.4
GPT-5-mini [ 42 ] +P
90.3
57.4
83.0
66.7
91.9
79.9
95.8
90.6
48.4
70.7
77.8
69.0
58.0
92.4
74.4
75.9
GPT-5 [ 42 ] +P
88.4
66.1
81.2
72.6
87.8
80.3
98.3
95.8
77.4
87.2
88.1
79.4
64.0
94.9
84.1
83.1
Table 1: Evaluation on proposed MPRBench.
Figure 4: Diagnostic Analysis of Multimodal Reasoning Paths
Figure 5: Distractor Injection Points in the Input Structure.
Figure 6: Analysis on Failure Cases.
Figure 7: Analysis on Cross-modal Distractor.
Appendix figures & tables32 assets
Supplementary material from the paper’s appendix.
Appendix
Category
Source
# Concepts
Person
Yo’LLaVA [ 30 ]
7
MC-LLaVA [ 3 ]
9
MMPB [ 19 ]
6
CustomConcept101 [ 22 ]
3
SoccerAgent [ 37 ]
22
Web
22
Appendix
Table 2: Distribution of personalized concepts in MPRBench across concept categories and source datasets.
Task Category
Sub-task
# Questions
Visual-Specific
Prediction
155
Spatial
115
Numerical
271
Commonsense
102
Visual Appearance Change
123
Sub-total (Visual)
766
Appendix
Table 3: Detailed statistics of MPRBench across different task categories.
Category
Attribute
Definitions
Visually-grounded
gender
Gender of Person Concept which is strongly correlated with the concept’s appearance in reference images.
height
Height of Person Concept which inferred from visual context and diversity-oriented rules.
Diversity-oriented
name
Name of Person Concept by using unique name list to avoid duplication.
birthday
Birthday of Person Concept randomly generated based on estimated age to ensure temporal diversity.
family
Family Information about Person Concept which includes Family Relation with its birthday each. Randomly generated based on Person Concept’s birthday and Replationship.
Context-aware
personality
Personality of Person Concept generated based on visual queries and fixed attributes.
Appendix
Table 4: Full List and Definitions of Attributes in the Textual Reference for Person Concepts.
Category
Attribute
Definitions
Identification
name
Name of Object Concept assigned by a unique name list to avoid duplication.
Diversity-oriented
acquisition_date
Acquisition date of Object Concept.
manufacturing_date
Manufacturing date of Object Concept which is always before acquisition date.
owner
Name of Object Concept’s Owner which assigned Person Concept Name for further extensibility .
Context-aware
object_type
Type of Object Concept.
acquisition_process
Acquisition process of Object Concept.
Appendix
Table 5: Full List and Definitions of Attributes in the Textual Reference for Object Concepts.
Attribute
Validation criterion
N
Agr.
Gender
match w/ annotators
50
100.0
Age
within ± 10-year range
50
96.0
Preference
free of contradiction
150
99.3
Event
consistent & grounded
150
94.0
Appendix
Table 6: Validation of the generated textual references. N denotes the number of inspected items, and Agr. denotes the agreement rate (%) between two annotators.
Stage
Inspected
Retained
Rate
VCR query selection
4,000
120
3.0%
Type-3 image filtering
3,117
579
18.6%
Appendix
Table 7: Retention rates of the visual query filtering stages in MPRBench .
Method
Natural
Synthetic
Δ
GPT-4o+P
70.4
79.3
+8.9
GPT-5-mini+P
67.9
77.2
+9.3
GPT-5+P
76.5
81.2
+4.7
Gemini-2.5 Pro+P
73.5
76.2
+2.7
LLaVA+P
46.9
50.7
+ 3.8
Qwen2.5-VL-7B+P
60.5
54.3
− 6.2
Appendix
Table 8: Accuracy on natural versus synthetically generated visual queries, averaged over the sub-tasks that contain both image sources. Δ denotes synthetic minus natural.
Method
w/ distractors
w/o distractors
Δ
GPT-4o+P
92.2
93.7
+1.5
GPT-5-mini+P
89.3
97.9
+8.6
GPT-5+P
88.4
95.8
+7.4
Gemini-2.5 Pro+P
83.5
96.8
+13.3
LLaVA+P
68.9
82.1
+13.2
Qwen2.5-VL-7B+P
75.7
90.5
+14.8
Appendix
Table 9: Effect of removing visually similar distractors from the query images of the predictive reasoning sub-task. Δ denotes w/o distractors minus w distractors.
Figure 8: Analysis on Text-Specific Distractor
Method
w/ distractors
w/o distractors
Δ
GPT-4o+P
60.5
69.4
+8.9
GPT-5-mini+P
71.9
75.9
+4
GPT-5+P
79.9
83.1
+3.2
Gemini-2.5 Pro+P
78.5
83.1
+4.6
Qwen2.5-VL-7B+P
45.4
51.0
+5.6
RAP
33.0
33.4
+0.4
Appendix
Table 10: Effect of inter-concept distractors. Scores are the overall Avg. on MPRBench. Δ denotes w/o distractors minus w distractors.
Figure 9: Comparison between General Multi-modal Capabilities and Personalized Reasoning Performance