PinpointQA: A Benchmark for Small Object-Centric Spatial Understanding in Indoor Videos

Zhiyu Zhou; Peilin Liu; Ruoxuan Zhang; Luyang Zhang; Cheng Zhang; Hongxia Xie; Wen-Huang Cheng

cs.CVApr 10, 2026

PinpointQA: A Benchmark for Small Object-Centric Spatial Understanding in Indoor Videos

Authors: Zhiyu Zhou, Peilin Liu, Ruoxuan Zhang, Luyang Zhang, Cheng Zhang, Hongxia Xie, Wen-Huang Cheng

Organizations: Jilin University, Changchun, China · National Taiwan University, Taipei, Taiwan

Abstract

Reliable embodied interaction in indoor environments requires agents to precisely localize small everyday objects from visual observations. Yet this fundamental capability remains challenging for multimodal large language models (MLLMs), particularly when spatial understanding must be performed from indoor videos. Existing benchmarks study video spatial intelligence and embodied reasoning, but do not directly evaluate whether a model can localize a small target object and express its position with sufficient precision. We introduce PinpointQA, a benchmark built from ScanNet++ and ScanNet200. It contains 1,024 scenes and 10,094 QA pairs across four progressively challenging tasks: Target Presence Verification, Nearest Reference Identification, Fine-Grained Spatial Description, and Structured Spatial Prediction. Ground-truth annotations are constructed from intermediate spatial representations derived from aligned 3D geometry and instance-level annotations, while evaluated models receive only sampled RGB video frames. Evaluations of representative MLLMs reveal a consistent performance decline across the task progression, with structured spatial prediction remaining particularly challenging, while supervised fine-tuning substantially improves performance. By isolating the spatial grounding capabilities that precede downstream embodied interaction, PinpointQA serves as both a diagnostic benchmark and an effective training resource. The project page is available at https://rainchowz.github.io/PinpointQA/.

Figures & tables

Figure 1: Overview of the proposed PinpointQA benchmark. PinpointQA evaluates small object-centric spatial understanding in indoor videos through four tasks: Target Presence Verification (TPV), Nearest Reference Identification (NRI), Fine-Grained Spatial Description (FSD), and Structured Spatial Prediction (SSP). The top row shows an example video sequence, the surrounding panels illustrate the task formats, and the center summarizes the benchmark statistics and min–max normalized performance of representative MLLMs.

Bench.	Spatial	Obs.	Indoor	Local.
Open3D-VQA [ 40 ]	✓	✗	✗	✗
EmbSpatial-Bench [ 9 ]	✓	✗	✓	✗
V-STaR [ 5 ]	✓	✓	✗	✗
Video-MSR [ 42 ]	✓	✓	✗	✗
STI-Bench [ 16 ]	✓	✓	✓	✗
VSI-Bench [ 35 ]	✓	✓	✓	✗

Table 1: Comparison with related benchmarks. From left to right, the columns denote Benchmark ( Bench. ), Spatial Reasoning ( Spatial ), Continuous Scene Observation ( Obs. ), Everyday Indoor Object Context ( Indoor ), and Small Object Localization ( Local. ).

Figure 2: Construction pipeline of PinpointQA. Starting from the aligned 3D meshes and instance-level annotations of ScanNet++ and ScanNet200, we select candidate small object targets, construct intermediate spatial representations through scene curation, and instantiate them as TPV, NRI, FSD, and SSP QA pairs. The pipeline combines rule-based filtering with iterative manual review for quality control.

Figure 3: Benchmark statistics: (a) task distribution, (b) composition by data source and counts for each task, (c) categories of target objects shown as a word cloud, and (d) video duration distribution.

Proprietary Models
Model	TPV		NRI		FSD		SSP		Avg
	Micro	Macro	Micro	Macro	Micro	Macro	Micro	Macro	Micro	Macro
GPT-5.4 [ 26 ]	0.65	0.69	0.39	0.42	0.31	0.32	0.15	0.16	0.38	0.40
Kimi K2.5 [ 28 ]	0.80	0.84	0.42	0.44	0.32	0.33	0.15	0.15	0.42	0.44
Open-source Models
LLaVA-OneVision-1.5-8B [ 1 ]	0.76	0.79	0.30	0.30	0.26	0.27	0.07	0.06	0.35	0.36

Table 2: Main results on PinpointQA across the four tasks. For each task, Micro and Macro report the mean over all its QA pairs and the mean of per-scene scores, respectively. Avg is the arithmetic mean across the four tasks. Bold denotes the best performance in each column.

Figure 4: Human localization accuracy and average completion time under Unguided, Model-Assisted, and GT-Assisted settings.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Example 1. Errors across TPV, NRI, FSD, and SSP for target mouse in scene 062e5a23a6 . The examples show an incorrect target presence decision, an incorrect nearest reference, unsupported references and distances in FSD, and inconsistent supporting surface, reference, relation, and distance fields in SSP. Green text agrees with the ground truth, while red text indicates incorrect or unsupported content.

Figure 6: Example 2. FSD and SSP errors for target mobile phone in scene 0b031f3119 . Kimi K2.5 and InternVL3.5-8B-Instruct produce readable or parsable predictions but replace the ground truth references, spatial relations, and distances with incorrectly grounded alternatives.

Figure 7: Example 3. FSD and SSP errors for target headphone case in scene 281ba69af1 . Although the target label is preserved, the predictions contain incorrect supporting surfaces, reference objects, spatial relations, or distances.

Figure 8: Human evaluation interface in the GT-Assisted setting.

Spatial Reasoning Benchmarks

Spatial reasoning benchmarks test whether AI systems can perceive, represent, and act on geometric, relational, and viewpoint-dependent information.

See all 162 papers

From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
May 4, 2026Le Zhang, Jihan Yang, Soundarya Krishnan +11Spatial Reasoning Benchmarks Visual Spatial Reasoning
1Mila - Qu´ebec AI Institute, UdeM · 2NYU · 3Apple +1
Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models
May 27, 2026Jiyao Zhang, Mingxu Zhang, Yitong Peng +8VLM Evaluation Spatial Reasoning Benchmarks
CFCS, School of CS, PKU Beijing, China · BUPT Beijing, China · PKU Beijing, China +2
Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests
Jul 24, 2026Patrick Rim, Tom Long, Ekta Prashnani +63D Spatial Reasoning Visual Spatial Reasoning
1NVIDIA Research · 2Yale University

May 4, 2026J/K move · Enter open · S save

From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs

Le Zhang, Jihan Yang, Soundarya Krishnan, Jimit Majmudar+10

1Mila - Qu´ebec AI Institute, UdeM · 2NYU · 3Apple+1

Spatial Reasoning Benchmarks Visual Spatial Reasoning Multimodal Large Language Models Video Reasoning Multimodal Model Evaluation

Open paper AskPDF arXiv

Human-level agentic intelligence extends beyond low-level geometric perception, evolving from recognizing where things are to understanding what they are for. While existing benchmarks effectively evaluate the geometric perception capabilities of multimodal large language models (MLLMs), they fall short of probing the higher-order cognitive abilities required for grounded intelligence. To address this gap, we introduce the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark with over 1,500 expert-annotated questions derived from diverse egocentric indoor video scans. SFI-Bench systematically evaluates two complementary dimensions of advanced reasoning: (1) Structured Spatial Reasoning, which requires understanding complex layouts and forming coherent spatial representations, and (2) Functional Reasoning, which involves inferring object affordances and their context-dependent utility. The benchmark includes tasks such as conditional counting, multi-hop relational reasoning, functional pairing, and knowledge-grounded troubleshooting, directly challenging models to integrate perception, memory, and inference. Our experiments reveal that current MLLMs consistently struggle to combine spatial memory with functional reasoning and external knowledge, highlighting a critical bottleneck in achieving grounded intelligence. SFI-Bench therefore provides a diagnostic tool for measuring progress toward more cognitively capable and truly grounded multimodal agents.

Abstract

Spatial Reasoning Benchmarks

Explore similar work

From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs

Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models

Spatial-IQ: Deconstructing Spatial Intelligence via Hierarchical Capability Tests