Organizations: School of Computer Science, Shanghai Jiao Tong University · Zhongguancun Academy · Ant Group · Fudan University · Shanghai Innovation Institute
Multimodal perception forms the foundation of multimodal intelligence. In particular, spatial perception and multimodal long-context understanding are two pivotal tasks that underpin embodied AI and multimodal agents. However, synthesizing targeted, high-quality training data for these tasks remains difficult and costly. Our key insight is that these tasks share an underlying capability, which we call connecting evidence: identifying relevant visual facts, binding them to the correct entities, recovering their relations, and composing them into an answer. Crucially, targeted training data are far easier to synthesize for this capability than for these tasks themselves. To strengthen this capability, we propose EvidenceConnect, an easy-to-implement data synthesis framework that uses zoomed views of related regions within a single image to generate questions requiring comparison, aggregation, or chaining of visual evidence. To ensure quality, a reference answer is retained only when independent VLMs agree on it, and region-ablation tests discard questions whose answer survives the removal of selected regions. We then train base VLMs on the resulting data with RLVR, obtaining Zooming without Zooming v2 (ZwZ2). Although the synthesized data are not tailored to any downstream benchmark, ZwZ2 improves fine-grained and spatial perception, with further gains on embodied spatial reasoning, multimodal long-context understanding, multimodal agent tasks, and general visual reasoning. These results establish EvidenceConnect as an effective framework for supervising evidence connection and suggest that strengthening this shared capability leads to more generalizable multimodal perception.
Figures & tables
Figure 1: Performance comparison of ZwZ2 and other advanced models.
Figure 2: Overview of our multi-region question synthesis pipeline. We select related regions from a single image and construct a privileged crop view to generate compositional questions. The illustrative example connects a flight number on a boarding pass to its departure gate and then to a service icon (T218 → C6 → plug). Cross-model answer anchoring and drop-one region tests assess answer reliability and evidence dependence, followed by multiple-choice conversion and difficulty filtering. During training, the policy receives the full image with region outlines, while privileged crops are used only for data construction.
Domains
Benchmarks
Base models
ZwZ2
Advanced models
4B
9B
27B
4B
9B
27B
GPT
GLM
DeepSeek
VStar
82.20
83.77
87.96
89.00
90.58
91.62
75.92
83.77
78.53
HR-4K
82.37
84.88
84.25
88.62
89.75
89.88
83.87
84.25
79.13
HR-8K
78.37
82.50
80.00
83.63
87.62
88.00
79.13
80.63
75.00
MME-en
70.60
71.00
72.62
75.48
75.33
77.41
58.50
67.88
50.75
MME-cn
69.33
69.53
68.68
73.89
73.45
74.97
56.90
68.31
50.04
Table 1: Main results on 25 benchmarks (accuracy, %). Domain averages and Overall are unweighted means within each domain and over all 25 benchmarks, respectively.
Models
Fine-grained perception
Spatial perception
Embodied spatial reasoning
Long-context understanding
Avg.
HR-4K
HR-8K
Chart
O3
CountQA
MMSI
MindCube
ERQA
ViewSpatial
MileBench
MemLens
Base
82.37
78.37
5.00
53.91
36.91
30.70
47.62
45.75
37.43
60.08
40.62
47.16
MMHops (31K)
77.75
73.12
3.00
45.80
30.82
30.50
42.10
43.75
40.90
66.29
15.97
42.73
TVG (19.4K)
78.25
73.62
13.00
53.04
33.97
29.50
28.86
46.00
42.16
64.56
37.29
45.48
F 2 RVLM (37K)
80.13
77.62
10.00
54.20
32.72
31.90
34.38
44.75
40.56
64.58
39.80
46.42
CRIT (30K)
82.25
77.88
6.00
45.22
30.56
29.90
43.43
45.00
43.98
68.13
40.40
46.61
Table 2: Training Qwen3.5-4B on different synthetic datasets (accuracy, %). Parentheses give the number of training examples. HR denotes HR-Bench; Chart and O3 denote Chartography and O3-Bench. MMSI, MindCube, and ViewSpatial denote MMSI-Bench, MindCube-Tiny, and ViewSpatial-Bench, respectively.
Settings
Fine-grained perception
Spatial perception
Embodied spatial reasoning
Long-context understanding
Avg.
HR-4K
HR-8K
Chart
O3
CountQA
MMSI
MindCube
ERQA
ViewSpatial
MileBench
MemLens
Base
82.37
78.37
5.00
53.91
36.91
30.70
47.62
45.75
37.43
60.08
40.62
47.16
Without privileged views
85.12
82.50
13.00
53.62
40.51
33.00
50.19
52.25
43.87
70.22
40.87
51.38
Without drop-one
85.75
79.75
8.00
58.26
41.30
34.10
47.14
51.50
43.84
70.16
41.48
51.03
Ours (7K)
86.62
82.87
9.00
62.90
45.09
34.60
52.29
47.50
45.68
70.47
42.14
52.65
Table 3: Data-synthesis ablations on Qwen3.5-4B (accuracy, %). Each trained variant uses 7K examples. The 7K reference model is shared by both comparisons and is reported only once; it uses privileged views and drop-one verification. Avg. is the unweighted mean over 11 benchmarks.
Figure 6
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Stage
Model
Visual input
Calls
Temp.
Evidence discovery
Qwen3.5-397B-A17B
source image
1 per image
0.4
Evidence grouping
Qwen3.5-397B-A17B
none (text only)
1 per image
0.3
Question realization
Qwen3.5-397B-A17B
VG
1 per group
0.5
Answer anchoring
Qwen3.5-397B-A17B, Kimi-K2.6
VG
up to 4 + 4 per candidate
0.4
Equivalence judging
GLM-5.2
none (text only)
when needed
0.0
Drop-one testing
Qwen3.5-397B-A17B
VG−i
1–3 per region
0.4
Appendix
Table 4: Model calls in the EvidenceConnect pipeline. VG denotes the privileged layout, VG−i the layout with region ri masked, and VI the policy view.
Figure 7: Composition of the 20.4K training examples by the number of selected regions per example and by difficulty level.
4B
9B
Layer
Qwen3.5
ZwZ2
Δ
Qwen3.5
ZwZ2
Δ
4
11.625
11.877
+0.252
10.622
10.699
+0.077
8
10.355
10.402
+0.047
10.069
10.051
-0.018
12
11.568
11.692
+0.124
11.073
11.050
-0.023
16
10.400
10.650
+0.251
11.080
11.256
+0.175
20
11.132
11.386
+0.255
11.493
11.777
+0.284
Appendix
Table 5: Top- k evidence IoU on TreeBench. Mean IoU is reported as a percentage; Δ is ZwZ2 minus Qwen3.5 in percentage points. Each entry averages the same 405 paired examples. Differences are computed before rounding.
Figure 8: Qualitative comparisons between Qwen3.5-4B and ZwZ2-4B. Three selected examples show correct predictions by ZwZ2-4B where the base model fails: (a) identifying a category present in every image; (b) associating recognized text with the queried location; and (c) counting all instances in a structured arrangement. Responses are excerpted from the original outputs, with omissions marked by [...] and answer fields reformatted. The enlarged image region in (b) is provided for visualization only.
Long-term agent memory is increasingly multimodal, yet existing evaluations rarely test whether agents preserve the visual evidence needed for later reasoning. In prior work, many visually grounded questions can be answered using only captions or textual traces, allowing answers to be inferred without preserving the fine-grained visual evidence. Meanwhile, harder cases that require reasoning over changing visual states are largely absent. Therefore, we introduce MemEye, a framework that evaluates memory capabilities from two dimensions: one measures the granularity of decisive visual evidence (from scene-level to pixel-level evidence), and the other measures how retrieved evidence must be used (from single evidence to evolutionary synthesis). Under this framework, we construct a new benchmark across 8 life-scenario tasks, with ablation-driven validation gates for assessing answerability, shortcut resistance, visual necessity, and reasoning structure. By evaluating 13 memory methods across 4 VLM backbones, we show that current architectures still struggle to preserve fine-grained visual details and reason about state changes over time. Our findings show that long-term multimodal memory depends on evidence routing, temporal tracking, and detail extraction.
Vision language models (VLMs) excel at many tasks but still struggle with spatial reasoning when critical information is not directly observable. Many such problems require imaginative perception: inferring what would be seen from an unseen viewpoint, tracing paths through occluded spaces, or integrating partial observations into a coherent spatial representation. We introduce Imaginative Perception Tokens (IPT), intermediate perceptual representations that externalize what a VLM would perceive under alternative spatial configurations while remaining consistent with the observed input. To study this capability, we formulate three tasks, Perspective Taking (PET), Path Tracing (PT), and Multiview Counting (MVC), and construct datasets of approximately 20K examples with ground truth imaginations, answers, and evaluation benchmarks. Using the unified VLM BAGEL as the backbone, IPT supervision consistently improves spatial reasoning and often outperforms textual chain of thought training, even without generating images at inference time. On MVC, IPT improves accuracy by 3.4% and achieves competitive performance with strong closed-source models on PT. We further find that combining IPT and label-only supervision yields additional gains, whereas textual chain of thought can substantially degrade performance, suggesting a modality mismatch when spatial computation is forced through language. Overall, IPT provides a principled supervision signal for reasoning about unobserved spatial structure, improving generalization while producing interpretable intermediate representations.
Mahtab Bigverdi, Linjie Li, Weikai Huang +8
University of Washington · 2Allen Institute for AI · 3Microsoft +1
Reasoning VLMs can become more accurate while progressively losing visual grounding as they think. This creates task-conditional danger zones where low-entropy predictions are confident but ungrounded, a failure mode text-only monitoring cannot detect. Evaluating three reasoning VLMs on MathVista, HallusionBench, and MMMU_Pro, we find a pervasive evidence-collapse phenomenon: attention to annotated evidence regions drops substantially, often losing over half of evidence mass, as reasoning unfolds. Full-response entropy is the most reliable text-only uncertainty signal under cross-dataset transfer, yet adding vision features with a single global linear rule is brittle and often degrades transfer. An entropy-vision interaction model reveals a task-conditional regime: lowentropy, visually disengaged predictions are hazardous on sustained visual-reference tasks but benign on symbolic tasks. Using this structure, a targeted vision veto reduces selective risk by up to 1.9 percentage points at 90% coverage, while avoiding degradations where disengagement is expected. The results support task-aware multimodal monitoring for safe deployment under distribution shift.