A robot that returns to a space must tell which objects were added, removed or moved since its last visit, while its trajectory, viewpoints and lighting differ between the two visits. We find that multi-image vision-language models (VLMs) fail at this task mainly because they cannot find the place where an object was. A VLM recognises what each frame shows but cannot check whether an object is still at its place in the other capture. It therefore reports objects that only one capture shows and misses most removed and moved objects. We propose STILL, which reformulates change detection as object verification at a registered place. 3D geometry registers the place of each change hypothesis in the other capture, and one VLM answers, from a scene view and a local place view, whether the object is still at that place, trained with supervision of the same form. Under the official SceneDiff protocol, STILL raises the main metric obj/sc AP from 24.0 to 37.8, and from 11.6 to 20.6 on the egocentric kitchen subset without any kitchen training data. It improves F1 by more than 21 points over a VLM fine-tuned on the same annotations and adds only about 11% to the runtime.
Figures & tables
Figure 1: Change is a question of place. (a) Two captures of a room in which four objects were moved, under a new viewpoint and new lighting. A multi-image VLM fine-tuned on the same data reports objects that only one capture shows and finds none of the four. (b) STILL registers each change hypothesis in a joint 3D reconstruction of both captures, which links the object to the place where it was. (c) At that place, one VLM answers whether the object is still there. A moved chair leaves its place empty, while a plant seen from a new side is still in place. (d) obj/sc AP under the official SceneDiff protocol for the VLM baseline of [ 1 ] (Qwen2.5-VL + SAM3), reproduced SceneDiff and STILL.
Figure 2: VLMs see what is present and miss what is gone (SD-V test split, all VLMs are Qwen3.5-9B). (a) Recall by change type at precision ≥ 0.34. The fine-tuned holistic VLM misses removed and moved objects, which the untrained VLM asked at the registered place (dashed) recovers. (b) Share of no-change controls, cut from one video into interleaved frames or into two halves, with a reported change.
Figure 3: Overview of STILL. (1) A high-recall proposer, SceneDiff by default, gives change hypotheses hi , each with a frame, a box and a geometric cost ci . (2) π3 reconstructs both captures jointly. Each hypothesis is lifted to 3D and projected into the other capture, and a depth occlusion test selects the frame in which its place is best visible, which gives the registered place pi . This step relies only on the static scene, so it stays correct where the object is gone. (3) One fine-tuned VLM answers two questions. The scene view (5+5 frames) asks whether the object is still at its place in the other capture, and the place view (object crop next to place crop) asks whether it is still at this place. The fused score si is thresholded at the operating point chosen on dev. The verifier is trained on per-object questions of the same form, built from real captures with same-capture hard controls and from synthetic two-view pairs.
Figure 4: Qualitative comparison on three SD-V scenes and one SD-K scene. Each row shows two of the 5+5 frames of an item, and the counts are over all frames. Gray boxes: ground-truth changed objects. Green: correct detections. Red: false alarms. Over the four items, SceneDiff finds 6 of 17 changes with 4 false alarms, the fine-tuned VLM finds 2 with 2 false alarms, and STILL finds 15 with none.
Figure 5: The registration paradox. Places found (same frame as the π3 place, IoU ≥ 0.5) for unchanged (open) and changed (filled) objects by the box left in place, SIFT matching and successive versions of learned VLM localisation. Appearance-based registration finds unchanged places about twice as often as changed ones, where the answer is decided. Right: system AP change against π3 places.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1: Official protocol, every run. Horizontal bars: reproduced SceneDiff. Dots: the three Qwen3.5-9B runs of STILL. Diamonds: the Qwen3-VL-8B run. obj/sc+type also requires the correct change type.
Figure S2: Every paired comparison of the main text as the gain with its 95% scene-bootstrap interval on a 0 to 1 scale (three runs, operating points chosen on dev). Groups: the main system, the second backbone, and the place-view ablations on SD-V F1.
Figure S3: All metrics at a glance , each axis scaled by its best value. Both backbones of STILL enclose SceneDiff on every axis.
Figure S4: Precision-recall curves under the VLM-output protocol (IoU 0.3) on SD-V (a) and SD-K (b). The three runs of STILL lie above all baselines over the whole curve. Dots: operating points chosen on SD-V dev.
Figure S5: Breakdown on the SD-V test split. (a) F1 by scene type, with the largest gain in cluttered public indoor scenes such as stores, offices and gyms. (b) Recall by object size, where small objects (29 of 1,016) remain hard.
Figure S6: F1 per scene category on the SD-V test split for each method (number of changed objects in brackets, best per row in bold).
Figure S7: Per-pair F1 of STILL against SceneDiff (a) and the fine-tuned VLM (b) on the SD-V test split. Marker size: number of changed objects. Colour: scene type. Points above the diagonal are pairs where STILL is better.
Figure S8: The two views complement each other. Scene-view against place-view probability of changed for SceneDiff candidates on the SD-V test split, for unchanged (a) and changed (b) objects, with the share of candidates in each quadrant at 0.5.
Figure S9: Score distributions of unchanged and changed SceneDiff candidates under the SceneDiff cost (a) and the fused score of STILL (b), with the AUC of separating the two.
Figure S10: Threshold sensitivity on the SD-V test split for STILL (a, three runs) and SceneDiff (b). Dots: thresholds chosen on dev. Shaded: F1 below the best value of SceneDiff.
Version
places found (%)
SD-V AP
SD-K AP
Δ SD-V
Δ SD-K
locate in the 10-frame layout
6.8 / 32.2
56.4
41.9
-3.5
-5.1
+ zoomed refinement
12.0 / 48.5
54.5
41.9
-2.9
-4.9
object masked (context only)
13.1 / 33.2
54.9
39.4
-3.4
-6.9
+ changed × 4, KC-3D localisation
19.6 / 50.9
53.6
39.9
-3.4
-4.2
+ wide context, second pass
22.5 / 50.6
53.8
42.6
-1.3
-4.5
+ RL, verification reward (v2b)
21.7 / 51.1
53.0
42.6
-2.1
-4.5
Appendix
Table S5: Learned registration (one run each). The VLM predicts the place itself as a frame and a box in the other capture, optionally refined on a zoomed crop. Places found: agreement with the π3 place in the frame the model chose (IoU ≥ 0.5), for changed / unchanged objects. Δ : system AP with the learned place view minus that with the π3 place view under the same verifier. The main system with π3 places reaches 57.6 / 46.9 AP. † SFT on half of the training scenes and RL on the other half, compared with its own π3 reference of 51.4 / 43.2 AP.
Figure S11: Learned registration by change type. Share of places found (%) for unchanged, moved, removed and added objects, for the unmapped box, SIFT matching and each learned version.
Figure S12: RL for learned localisation (50-step moving average) in four settings: items seen in SFT (v1), unseen synthetic items (v2), twice as many unseen synthetic items (v2b), and cross-fitting on unseen real scenes. With seen or synthetic items the verifier reward saturates near 1, while on unseen real scenes it stays informative and the hit rate on changed places rises during training. No setting improves the test split ( Tab. S5 ).
Figure S13: Place-view cases on the SD-V test split (four random cases per row). Each case shows the object crop and the place crop with the scene-view and place-view probabilities of changed ( Ps , Pp ) and the SceneDiff cost. Top: SceneDiff false alarms rejected by STILL, mostly on dense shelves and motion-blurred frames. Middle: true changes that SceneDiff ranks low and STILL keeps. Bottom: errors of STILL, three of four on store shelves where an identical product stands at the registered place.
Zero-shot Qwen3-VL-8B
Qwen3.5-9B fine-tuned on the same data
STILL
✗ added: mirror (v2 f1)
✗ removed: yellow bottle (v1 f2)
✓ added: gray chair (v2 f2, score 0.140)
✗ added: light stand (v2 f1)
✗ removed: tissue (v1 f3)
✓ added: white chair (v2 f2, score 0.062)
✗ added: laptop (v2 f3)
✗ removed: cardboard box (v1 f4)
✓ added: laptop (v2 f5, score 0.046)
✗ added: power strip (v2 f4)
✗ removed: toolbox (v1 f4)
∘ added: white chair (v2 f1, score 0.026)
✗ removed: vertical blinds (v1 f1)
✗ moved: white chair (v1 f5)
∘ added: laptop (v2 f3, score 0.013)
✗ removed: plant (v1 f1)
✗ removed: curtain (v1 f1)
∘ added: gray chair (v2 f1, score 0.009)
Appendix
Figure S14: Raw outputs on the scene of Fig. 1 (SD-V test split, 5+5 frames). Ground truth: four moved objects (white chair, gray chair, laptop, floor lamp). ✓ / ✗ follow the benchmark rule (named frame, box IoU ≥ 0.3 with a changed object, each object counted once), and ∘ marks a repeated hit on an object already found, which is not counted. Holistic VLMs name objects that only one capture shows (curtain, mirror, light stand) and miss the moved chairs. The outputs of STILL are the SceneDiff hypotheses kept by the verifier, each named by the object it matches.
Vision-language models (VLMs) frequently fail at visual change reasoning, even when their vision encoders contain sufficient information. We observe that correct VLM outputs tend to contain explicit verbal evidence (object names, colors, spatial locations) that supports the claimed change, while incorrect outputs often lack such evidence. We propose SAVER (Selective Auditing of Verbal Evidence for Error Recovery), a lightweight, rule-based method that parses VLM responses for this evidence and triggers structured reprompting only when evidence is missing or inconsistent. Across three change detection benchmarks and four VLMs, SAVER significantly improves accuracy on tasks where errors stem from the model failing to articulate what it saw (expression failures), with gains up to +25.8% on CLEVR-Change. The evidence patterns can also be generated by an LLM in a single call, matching the hand-tuned gate on CLEVR-Change. Ablation experiments confirm that the evidence gate, not reprompting alone, drives the improvement.
Robotic operation in previously unseen environments requires both semantic understanding and reliable metric information. While vision--language models (VLMs) provide strong semantic capabilities, their geometric estimates remain less reliable. In this paper, we propose a VLM-driven, modular perception framework for scene understanding using off-the-shelf approaches. Starting from a single RGB-D observation, the scene is segmented into object-level regions, annotated by a VLM, and grounded with depth information to construct a task-independent object-centric representation. Experiments on 151 tabletop scenes show that the proposed decomposition preserves strong semantic performance while substantially improving localization and depth estimation over direct VLM inference. The resulting representation is also integrated with a task-planning framework for robotic execution.
Enrico Saccon, Tommaso Faraci, Iñigo De La Ossa Zarzuelo +3
Department of Information Engineering and Computer Science, the University of Trento, Trento, Italy · Polytechnic School, Mondragón University, Mondragón, Spain · Department of Industrial Engineering, University of Trento, Trento, Italy
Vision Language Models (VLMs) are well known for hallucinating non-existent objects in images. Objects with missing parts present a unique challenge for VLMs, stemming from both real-world knowledge bias and the scarcity of such images in training data. We present MissingBench-Verified, a benchmark designed to evaluate a specific and practically relevant scenario: when vision-language models fail to recognize that an essential component of an object has been removed. Across ten leading models, we observe consistent and significant failure rates that persist even when external tool evidence explicitly contradicts the model's visual perception. We further ask whether granting models access to image processing tools (e.g., cropping, contrast adjustment) enables autonomous inspection to resolve these failures. We find that existing mitigation strategies, including tool-assisted verification, autonomous visual reasoning, longer reasoning durations, and fine-tuning on an easier dataset, provide negligible improvement, indicating that this failure mode cannot be addressed through current prompting or post-hoc correction techniques. Our findings highlight a fundamental limitation of current VLM for inspection and monitoring tasks and underscore the need for architectural or training-level interventions that enable models to override internal expectations when confronted with contradictory evidence.
Wenqi Marshall Guo, Qingyun Qian, Shiyu Zhou +2
Department of CMPS, University of British Columbia, Canada · Weathon Software, Canada · Department of Computer Science, University of British Columbia, Canada