QVR3D: Query-Guided Visual Reasoning for Training-Free 3D Grounding
Organizations: National Taiwan University · Research Center for Information Technology Innovation, Academia Sinica · National Yang Ming Chiao Tung University
Abstract
3D visual grounding localizes objects in 3D scenes from natural language descriptions, requiring reasoning over semantics, spatial relationships, and multi-view observations. Training-free approaches avoid costly annotations but often rely on explicit scene transformation or scene-level frame selection, which may waste limited visual capacity on query-irrelevant regions. We observe that effective grounding should prioritize observations that provide evidence for the query rather than uniformly represent the entire scene. Based on this insight, we introduce QVR3D, a query-guided visual reasoning framework that operates on RGB-D observations using off-the-shelf foundation models. QVR3D combines Query-Guided Maximum Coverage (QGMC) for selecting complementary query-relevant views with Grounding Probability Estimation (GPE) that integrates instance confidence, attribute consistency, and relationship consistency. The framework supports both single- and multi-object grounding without task-specific adaptation. Experiments on ScanRefer and Nr3D achieve state-of-the-art zero-shot performance, while QGMC also improves frame efficiency across multiple 3D vision-language tasks. Our code is available at https://github.com/jwliao1209/QVR3D.
Figures & tables
| Method | Venue | Agent | Unique | Multiple | Overall | |||
| [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | |||
| LLM-G [ 47 ] | ICRA 2024 | GPT-4 Turbo | - | - | - | - | 17.1 | 5.3 |
| ZSVG3D [ 49 ] | CVPR 2024 | GPT-4 Turbo | 63.8 | 58.4 | 27.7 | 24.6 | 36.4 | 32.7 |
| SeeGround [ 25 ] | CVPR 2025 | Qwen2-VL-72B | 75.7 | 68.9 | 34.0 | 30.0 | 44.1 | 39.4 |
| VoG [ 27 ] | AAAI 2026 | Qwen2-VL-72B | 78.6 | 71.5 | 34.0 | 30.4 | 44.8 | 40.3 |
| QVR3D † (Ours) | WACV 2027 | Qwen2-VL-72B | 78.3 | 73.2 | 44.9 | 39.7 | 51.4 | 46.2 |
| Method | Agent | Easy | Hard | Dep. | Indep. | Overall |
| ZSVG3D [ 49 ] | GPT-4 Turbo | 46.5 | 31.7 | 36.8 | 40.0 | 39.0 |
| SeeGround [ 25 ] | Qwen2-VL-72B | 54.5 | 38.3 | 42.3 | 48.2 | 46.1 |
| LASP [ 31 ] | GPT-4o | 60.7 | 45.3 | 49.2 | 54.7 | 52.9 |
| VoG [ 27 ] | Qwen2-VL-72B | 58.9 | 37.2 | 39.4 | 52.1 | 47.6 |
| VIZOR [ 30 ] | GPT-4o | 62.7 | 43.5 | 43.0 | 57.7 | 52.8 |
| QVR3D † (Ours) | Qwen2-VL-72B | 57.4 | 43.7 | 47.0 | 52.0 | 50.3 |
| QGMC | GPE | VLM-TS | ScanRefer | Nr3D | |
| [email protected] | [email protected] | [email protected] | |||
| ✓ | ✓ | ✗ | 44.7 | 40.3 | 36.7 |
| ✓ | ✗ | ✓ | 56.1 | 50.0 | 53.1 |
| ✓ | ✓ | ✓ | 56.9 | 51.0 | 53.6 |
| Sampling | Avg. Frames | [email protected] | [email protected] |
| Uniform | 16.00 | 40.5 | 35.7 |
| MC | 21.52 | 42.6 | 38.1 |
| QGMC (Ours) | 10.16 | 56.1 | 50.0 |
| Frame | Sampling | Coverage | Avg. | [email protected] | [email protected] |
| Budget | Threshold | Frames | |||
| 8 | Uniform | - | 8.00 | 49.80 | 45.40 |
| 8 | MC | 1.0 | 8.00 | 53.55 | 48.54 |
| 16 | Uniform | - | 16.00 | 54.95 | 49.82 |
| 16 | MC | 1.0 | 16.00 | 56.65 | 51.39 |
| 32 | Uniform | - | 31.99 | 58.02 | 52.68 |
| Method | ZT | ST | MT | Overall | ||||||
| w/o D | w/ D | w/o D | w/ D | |||||||
| F1 | F1 | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | [email protected] | |
| Supervised Expert | ||||||||||
| M3DRef-CLIP [ 52 ] | 81.8 | 39.4 | 53.5 | 47.8 | 34.6 | 30.6 | 43.6 | 37.9 | 42.8 | 38.4 |
| Supervised Generalist | ||||||||||
| D3Net [ 8 ] | 81.6 | 32.5 | - | 38.6 | - | 23.3 | - | 35.0 | - | 32.2 |
| QGMC | GPE | VLM-TS | ScanRefer | Nr3D | |
| [email protected] | [email protected] | [email protected] | |||
| ✓ | ✓ | ✗ | 44.2 | 40.0 | 35.6 |
| ✓ | ✗ | ✓ | 51.4 | 46.2 | 50.3 |
| ✓ | ✓ | ✓ | 49.5 | 44.6 | 46.4 |
| Frame | Sampling | Coverage | Avg. | [email protected] | [email protected] |
| Budget | Threshold | Frames | |||
| 8 | Uniform | - | 8.00 | 48.93 | 43.50 |
| 8 | MC | 1.0 | 8.00 | 53.47 | 47.41 |
| 16 | Uniform | - | 16.00 | 55.42 | 49.17 |
| 16 | MC | 1.0 | 16.00 | 56.46 | 50.11 |
| 32 | Uniform | - | 31.99 | 58.11 | 51.72 |
| Frame | Sampling | Coverage | Avg. | [email protected] | [email protected] |
| Budget | Threshold | Frames | |||
| 8 | Uniform | - | 8.00 | 37.34 | 68.82 |
| 8 | MC | 1.0 | 8.00 | 38.77 | 73.08 |
| 16 | Uniform | - | 16.00 | 39.39 | 76.96 |
| 16 | MC | 1.0 | 16.00 | 39.59 | 76.84 |
| 32 | Uniform | - | 31.99 | 41.30 | 83.76 |
| Frame | Sampling | Coverage | Avg. | CIDEr | EM |
| Budget | Threshold | Frames | |||
| 8 | Uniform | - | 8.00 | 94.98 | 27.57 |
| 8 | MC | 1.0 | 8.00 | 96.37 | 28.00 |
| 16 | Uniform | - | 16.00 | 99.86 | 28.96 |
| 16 | MC | 1.0 | 16.00 | 100.63 | 29.49 |
| 32 | Uniform | - | 31.97 | 102.06 | 30.09 |
| Frame | Sampling | Coverage | Avg. | EM |
| Budget | Threshold | Frames | ||
| 8 | Uniform | - | 8.00 | 56.77 |
| 8 | MC | 1.0 | 8.00 | 56.97 |
| 16 | Uniform | - | 16.00 | 57.70 |
| 16 | MC | 1.0 | 16.00 | 57.82 |
| 32 | Uniform | - | 32.00 | 58.56 |
3D Visual Grounding
Three-dimensional visual grounding links language or visual queries to specific objects, parts, regions, and spatial relations in 3D scenes.
See all 149 papers