MaRiO: Multi-agent Collaborative Reasoning via Shared Observations in MLLMs
Organizations: Institute of Artificial Intelligence, University of Central Florida
Abstract
Multimodal Large Language Models (MLLMs) show strong visual perception capabilities, but existing evaluations largely focus on single-view or multi-view settings with shared camera parameters, leaving the challenge of multi-agent collaborative scenarios underexplored. In such settings, multiple agents observe a shared environment from independent viewpoints, and reliable decisions require reasoning across these independent observations of the same environment. To study this problem, we introduce MaRiO, a benchmark for evaluating multi-agent collaborative reasoning, and evaluate 23 state-of-the-art MLLMs on it. Our analysis reveals three key findings: (1) tasks governed by static perception are generally easier than tasks that require transformational event sequences; (2) most models struggle to resolve cross-agent correspondences, with errors increasing with scene complexity; and (3) models show limited ability to leverage geometric cues present in the scene. As an initial step toward addressing these challenges, we explore geometry-guided synthetic view augmentation that generates intermediate transition frames between agent observations, providing auxiliary spatial context that improves performance in certain categories. Overall, our results highlight that multi-agent collaborative reasoning remains an open challenge and an important direction for multimodal embodied systems. The code is available on the project page: https://omarftt.github.io/MaRiO-page/.
Figures & tables
| Benchmark | Setup | Reasoning Requirement | Reasoning Challenge | ||||
|---|---|---|---|---|---|---|---|
| MV | MA | Perc. | Comp. | Cam. Geom. | Aux. Info. | ||
| VSR [ 48 ] | × | × | ✓ | × | - | None | Spatial relation |
| Winoground [ 10 ] | × | × | ✓ | × | - | Attribute tag | Compositionality |
| GeoMeter [ 11 ] | × | × | × | ✓ | - | None | Depth perception |
| OmniSpatial [ 22 ] | × | × | ✓ | ✓ | - | None | Cognitive spatial reasoning |
| ScanQA [ 52 ] | × | × | ✓ | × | - | Point Clouds | 3D scene understanding |
| Model | Comp. | Static Perc. | Overall | Avg. | |||
| Ego. | Nav. | Dist. | Occ. | Rel. | |||
| Human Evaluators | 64.7 | 68.8 | 86.7 | 77.8 | 68.8 | 73.2 | 73.3 |
| Random Choice | 21.7 | 24.9 | 22.1 | 26.5 | 25.3 | 23.9 | 24.1 |
| Proprietary Models | |||||||
| GPT 4o [ 70 ] | 26.4 | 41.8 | 43.7 | 51.1 | 34.1 | 38.3 | 39.4 |
| GPT 5.1 [ 71 ] | 32.3 | 43.9 | 47.4 | 53.6 | 55.9 | 45.2 | 46.6 |
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | RGB only | +Depth | ||
|---|---|---|---|---|
| Gemini-2.5-Pro | 61.9 | 48.3 | ||
| Qwen2.5-VL-7B | 53.2 | 39.8 | ||
| LLaVA-OneVision-7B | 49.7 | 47.1 | 0.25 | |
| LLaVA-NeXT-Video-7B | 46.2 | 46.8 | 0.82 |
| Model | Input | Ego. | Nav. | Dist. | Occ. | Rel. | Overall |
|---|---|---|---|---|---|---|---|
| LLaVA-OneVision-7B | RGB | 30.4 | 36.4 | 49.8 | 47.8 | 43.2 | 40.5 |
| Gemini-2.5-Flash | RGB | 31.3 | 44.6 | 51.5 | 50.3 | 46.3 | 44.1 |
| SpatialBot | RGB + depth | 27.1 | 24.6 | 37.1 | 42.0 | 41.3 | 33.2 |
| Model | Comp. | Static Perception | Overall | Avg. | |||
|---|---|---|---|---|---|---|---|
| Ego. | Nav. | Dist. | Occ. | Rel. | |||
| General Models | |||||||
| Phi-3.5-Vision-Instruct-4B | 27.3 | 31.9 | 36.7 | 37.3 | 31.4 | 32.3 | 32.9 |
| Syn. View | 28.4 | 34.4 | 38.7 | 39.2 | 34.1 | 34.3 | 35.0 |
| +1.1 | +2.5 | +2.0 | +1.9 | +2.7 | +2.0 | +2.1 | |
| VILA1.5-7B | 28.1 | 32.5 | 41.3 | 39.8 | 37.5 | 34.9 | 35.8 |
| Type | Category | 2-Agents | 3-Agents | Total |
|---|---|---|---|---|
| Comp. | Egocentric Motion | 576 | 189 | 765 |
| Navigation | 659 | 214 | 873 | |
| Static Perc. | Distance Awareness | 472 | 149 | 621 |
| Occlusion | 366 | 41 | 407 | |
| Relative Position | 517 | 72 | 589 | |
| All | 2,590 | 665 | 3,255 |
| Criterion | A1 | A2 | A3 | Pooled |
|---|---|---|---|---|
| (i) Category-style fit | 99.2 | 92.8 | 89.6 | 93.9 |
| (ii) Answer-option consistency | 100.0 | 92.8 | 99.2 | 97.3 |
| (iii) Entity concreteness | 98.4 | 96.8 | 96.0 | 97.1 |
| (iv) Overall style match | 97.6 | 89.6 | 86.4 | 91.2 |
| Overall | 98.8 | 93.0 | 92.8 | 94.9 |
| Task | 4 Choices | 6 Choices | 8 Choices |
|---|---|---|---|
| Distance Awareness | 35.90% (14/39) | 38.46% (15/39) | 33.33% (13/39) |
| Egocentric Motion | 40.57% (43/106) | 23.58% (25/106) | 20.75% (22/106) |
| Navigation | 39.11% (79/202) | 31.68% (64/202) | 28.22% (57/202) |
| Occlusion | 44.44% (48/108) | 32.41% (35/108) | 25.00% (27/108) |
| Relative Position | 44.14% (64/145) | 30.34% (44/145) | 19.31% (28/145) |
| Overall | 41.33% (248/600) | 30.50% (183/600) | 24.50% (147/600) |
| Task | 4 Choices | 6 Choices | 8 Choices |
|---|---|---|---|
| Distance Awareness | 20.51% (8/39) | 28.21% (11/39) | 17.95% (7/39) |
| Egocentric Motion | 30.19% (32/106) | 16.98% (18/106) | 20.75% (22/106) |
| Navigation | 36.14% (73/202) | 29.70% (60/202) | 19.80% (40/202) |
| Occlusion | 51.85% (56/108) | 34.26% (37/108) | 26.85% (29/108) |
| Relative Position | 50.34% (73/145) | 24.83% (36/145) | 20.69% (30/145) |
| Overall | 40.33% (242/600) | 27.00% (162/600) | 21.33% (128/600) |
| Task | 4 Choices | 6 Choices | 8 Choices |
|---|---|---|---|
| Distance Awareness | 35.90% (14/39) | 41.03% (16/39) | 41.03% (16/39) |
| Egocentric Motion | 33.96% (36/106) | 26.42% (28/106) | 21.70% (23/106) |
| Navigation | 33.17% (67/202) | 26.24% (53/202) | 25.74% (52/202) |
| Occlusion | 50.93% (55/108) | 31.48% (34/108) | 39.81% (43/108) |
| Relative Position | 45.52% (66/145) | 19.31% (28/145) | 15.17% (22/145) |
| Overall | 39.67% (238/600) | 26.50% (159/600) | 26.00% (156/600) |
| Compositional | Static Perception | ||||
| Ego. | Nav. | Dist. | Occ. | Rel. | |
| Ego. | 1.000 | 0.407 | 0.399 | 0.530 | 0.634 |
| Nav. | 0.407 | 1.000 | 0.426 | 0.612 | 0.542 |
| Dist. | 0.399 | 0.426 | 1.000 | 0.933 | 0.855 |
| Occ. | 0.530 | 0.612 | 0.933 | 1.000 | 0.856 |
| Rel. | 0.634 | 0.542 | 0.855 | 0.856 | 1.000 |
| Model | Synthetic | Real-World |
|---|---|---|
| LLaVA-NeXT-Video-7B-HF | 31.97 | 32.61 |
| mPLUG-Owl3-7B-240728 | 34.77 | 37.90 |
| Qwen2-VL-7B-Instruct | 39.86 | 42.97 |
| Qwen2.5-VL-3B-Instruct | 40.76 | 40.81 |
| Qwen2.5-VL-7B-Instruct | 37.26 | 39.71 |
Multi-Agent Reasoning
Multi-Agent Reasoning studies how multiple AI agents share evidence, model one another, and coordinate decisions on complex tasks.
See all 242 papers