Organizations: Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · Shanghai University of Electric Power · East China Normal University
The rapid progress of Multimodal Large Language Models (MLLMs) marks a significant step toward artificial general intelligence, offering great potential for augmenting human capabilities. However, their ability to provide effective assistance in dynamic, real-world environments remains largely underexplored. Existing video benchmarks predominantly assess passive understanding through retrospective analysis or isolated perception tasks, failing to capture the interactive and adaptive nature of real-time user assistance. To bridge this gap, we introduce LifeEval, a multimodal benchmark designed to evaluate real-time, task-oriented human-AI collaboration in daily life from an egocentric perspective. LifeEval emphasizes three key aspects: task-oriented holistic evaluation, egocentric real-time perception from continuous first-person streams, and human-assistant collaborative interaction through natural dialogues. Constructed via a rigorous annotation pipeline, the benchmark comprises 4,075 high-quality question-answer pairs across 6 core capability dimensions. Extensive evaluations of 26 state-of-the-art MLLMs on LifeEval reveal substantial challenges in achieving timely, effective and adaptive interaction, highlighting essential directions for advancing human-centered interactive intelligence.
Figures & tables
Figure 1: Overview of LifeEval. The benchmark emphasizes real-life interactive scenarios, requiring models to deliver precise, context-aware, and adaptive assistance.
Benchmark
#QAs
#Clips
Egocentric
Interactive
Real-Time
Task-Assist.
Multi-Turn
Rationale
Question Type
Annotation
MVBench [ 19 ]
4000
3641
✗
✗
✗
✗
✗
✗
Close
Auto
VideoMME [ 12 ]
2700
900
✗
✗
✗
✗
✗
✗
Close
Manual
Video-TT [ 49 ]
5000
1000
✗
✗
✗
✗
✗
✓
Open&Close
Auto&Manual
OmniMMI [ 34 ]
2290
1121
❍
✓
✓
✗
❍
✗
Open
Manual
OvO-Bench [ 25 ]
2814
644
❍
✗
❍
✗
✗
✗
Close
Auto&Manual
EgoVQA [ 10 ]
520
520
✓
✗
✗
✗
✗
✗
Close
Manual
Table 1: Comparison of LifeEval with existing video benchmarks. Columns summarize key properties, including dataset size (#QAs, #Clips), egocentric perspective, interactive and real-time settings, task-oriented assistance, multi-turn dialogue, availability of reasoning rationales for each answer, and annotation method. Symbols: ✓ (Yes), ❍ (Partial), ✗ (No).
Figure 2: Capability dimensions and examples in LifeEval. LifeEval defines six core capability dimensions to comprehensively evaluate a model’s interactive assistance abilities in daily life. The benchmark features both multiple-choice and open-ended questions, each accompanied by detailed reasoning explanations.
Figure 3: Task goal distribution and QA vocabulary statistics in LifeEval.
Statistics
Number
Percentage
Capacities
- Static Environment Perception (SEP)
666
16.34%
- Dynamic Task Reasoning (DTR)
708
17.37%
- Contextual Knowledge Retrieval (CKR)
775
19.02%
- Goal-oriented Planning (GP)
744
18.26%
- Safety and Feasibility Assessment (SFA)
646
15.85%
Table 2: Question statistics of LifeEval.
Model
Multiple-Choice Questions
Open-Ended Questions
SEP
DTR
CKR
GP
SFA
MIC
Overall
SEP
DTR
CKR
GP
SFA
MIC
Overall
Proprietary MLLMs
GPT-5 [ 26 ]
7 9.10
79.73
8 8.21
7 6.66
8 4.45
8 7.92
8 2.48
63.67
66.05
80.65
69.89
79.80
73.92
72.39
GPT-4o [ 15 ]
68.06
70.00
81.54
67.11
77.74
82.87
74.23
46.15
48.08
61.23
52.72
64.23
59.29
55.19
GPT-5-mini [ 26 ]
77.31
75.68
84.36
75.07
82.01
86.25
79.85
6 4.95
58.21
73.18
63.69
75.79
69.12
67.44
Gemini-2.5-Flash [ 7 ]
71.04
70.00
85.38
72.41
82.62
81.32
76.98
56.12
57.91
71.30
59.54
72.56
67.23
64.05
Table 3: Main evaluation results of representative MLLMs on LifeEval across different capability dimensions and question types. Bold and u nderline denote the best and second-best results, respectively.
Figure 7
# Frames
MCQ
OEQ
1f
8f
32f
1f
8f
32f
GPT-5
80.03
82.48
82.51
67.94
72.39
70.32
(+2.45)
(+2.48)
(+4.45)
(+2.38)
InternVL3.5-8B
56.72
57.98
57.26
24.38
25.40
26.31
(+1.26)
(+0.54)
(+1.02)
(+1.93)
LLaVA-OV-7B
56.65
56.17
55.39
29.11
29.81
29.22
Table 4: Performance comparison with varying input frame. Numbers in the second row of each model denote performance gain relative to 1-frame.
School of Data Science, Fudan University · College of Computer Science and Artificial Intelligence, Fudan University · Shanghai Innovation Institute +2