This report summarizes the Mobile Unified Multimodal Understanding (MUMU) track of the 8th Large-scale Video Object Segmentation (LSVOS) Challenge, held in conjunction with ECCV 2026. MUMU asks for a single unified model that jointly performs multi-concept image tagging over quality, scene, and event vocabularies, open-vocabulary object detection, and concise image captioning, within a budget of 0.5B parameters and 8 GB of peak inference memory. Submissions are ranked by a weighted task score scaled by an efficiency factor. We describe the tasks, rules, and evaluation protocol, report the final results, and review the methods of the top three teams. All three leading solutions build on the compact Florence-2-base model, but extend it in different ways: lightweight tagging heads with cross-task inference refinement, a shared low-rank adapter trained on coverage-aware multi-source data with task-dependent decoding, and a training-free pipeline that maps captions to tags and applies detection test-time augmentation. The results show that a compact prompt-based vision-language model can serve all three outputs when supervision, decoding, and cross-task cues are designed for each task, while open-vocabulary detection on novel categories remains the main bottleneck.
Table 1: Recommended training datasets of the MUMU track.
Rank
Team
A
B
C
E
Final
1
SSUfficient
27.18
8.22
47.34
0.676
17.3409
2
Team Spicy Gradient Descent
38.63
4.36
34.89
0.702
16.7172
3
DDUN
26.49
5.45
44.05
0.706
16.4815
–
Baseline (Florence-2-base) †
21.15
4.77
44.05
0.706
15.1568
Table 2: MUMU final test-set results. A : tagging macro-F1; B : novel-aware detection mAP@[0.5:0.95]; C : caption score (Eq. 1 ); E : efficiency factor; Final=(0.3A+0.4B+0.3C)×E . Bold marks the best score among ranked entries. † Florence-2-base evaluated with the official baseline inference script.
Team
Adaptation
Params (M)
Mem. (GB)
SSUfficient
Tagging heads
239.17
4.5
Team Spicy Gradient Descent
Fused LoRA
232.25
7.5
DDUN
None (training-free)
231.41
0.6
Table 3: Model complexity of the top three entries. All three build on Florence-2-base; Adaptation lists what each entry adds to it. Mem.: peak inference memory.
Figure 1: Overview of EUMU (SSUfficient). One Florence-2-base model is shared across tasks; lightweight heads predict tags from shared visual features, and task outputs are reused as cross-task cues for inference refinement.
Figure 2: Overview of the Team Spicy Gradient Descent system. Coverage-aware sampling and namespace-specific supervision build the training mixture; one Florence-2 model with a shared LoRA is decoded with task-dependent policies.
Figure 3: Overview of CapMap-MS-TTA (DDUN). One Florence-2-base checkpoint serves all tasks: Tasks A and C share the detailed-caption pathway, and Task B uses multi-scale and flip detection TTA followed by NMS.
The Mobile Unified Multimodal Understanding (MUMU) Challenge requires a single efficient model to jointly perform multi-concept image tagging, open-vocabulary object detection, and image captioning. We present Efficient Unified Multimodal Understanding (EUMU), the winning solution for the MUMU Track of the 8th LSVOS Challenge. EUMU builds on a shared pretrained multimodal model, using its prompt-based capabilities for detection and captioning and training lightweight heads on shared visual features to predict quality, scene, and event tags. Rather than treating the three tasks independently, EUMU applies task-aware inference refinement by reusing task outputs as cross-task cues. For detection, caption cues help recover objects missed by the initial detection. For captioning, detection cues help refine the caption to better reflect the detected objects. For tagging, image statistics refine quality predictions, while caption and detection cues refine scene and event predictions. This design unifies all three tasks within a single model while satisfying the challenge's resource constraints. EUMU contains 239.169M parameters, requires 23.947 GFLOPs, uses 4.5 GB of peak inference memory, and achieves a final challenge score of 17.3409. Code and models are available at https://github.com/Dayoung-Kil/EUMU.
Dayoung Kil, Seong-heum Kim
Department of Intelligent Semiconductors, Soongsil University, Seoul, Republic of Korea
The MUMU track of the 8th Large-scale Video Object Segmentation (LSVOS) Challenge requires a single unified multimodal model to jointly solve image tagging (Task A), open-vocabulary object detection (Task B), and English captioning (Task C) under strict resource constraints (<=0.5B parameters and <=8 GB peak GPU memory). We present CapMap-MS-TTA, a training-free submission built on Microsoft Florence-2-base (~231M parameters), combining caption keyword mapping with multi-scale flip test-time augmentation. Task C uses the native <DETAILED_CAPTION> pathway with length/token sanitization. Task A maps the same detailed caption into the official quality/scene/event vocabularies via an expanded keyword lexicon with whole-word matching and a lightweight expand-hints stage. Task B runs Florence-2 open detection (<OD>) with multi-scale and horizontal-flip test-time augmentation (TTA), followed by label-aware non-maximum suppression (NMS). Without fine-tuning, the system improves our reproduced Florence-2 baseline from 15.16 to a best public score of 16.4815, and ranks 3rd on the final MUMU leaderboard.
This report summarizes the 8th Large-scale Video Object Segmentation (LSVOS) Challenge, held in conjunction with ECCV 2026. The challenge evaluates video segmentation in three complementary settings: complex semi-supervised video object segmentation on MOSEv2, text-guided referring video object segmentation on MeViSv2-Text, and audio-guided referring video object segmentation on MeViSv2-Audio. We describe the tasks and evaluation protocols and review the methods of the top three teams in each track. Across the nine leading solutions, foundation segmentation models are combined with target-aware memory, multimodal reasoning, explicit target-existence verification, agentic interaction, and corrective tracking. These systems illustrate a broader transition from single-model mask propagation toward modular pipelines that reason about object identity, query validity, and temporal reliability.
Chang Liu, Henghui Ding, Lingyi Hong +36
Institute of Big Data, Fudan University, Shanghai, China