Evaluating Language Models as Spatial Priors for Object-Goal Navigation
Organizations: Department of Computer Science and Engineering, Toyohashi University of Technology, Japan
Abstract
Semantic exploration methods for object-goal navigation use large language models (LLMs) to score how strongly a target object is associated with a room type or with a nearby object. This paper considers obtaining from the LLM a spatial prior instead: a distribution over the position of the target in a room. We evaluate whether two open-weight LLMs, Qwen3.5-27B and Gemma-4-31B-it, can provide such a prior. Using real room layouts from CubiCasa5K and ScanNet, we measure the likelihood the models assign to real and perturbed layouts, and the layouts they generate. The object frequencies and room sizes of the generated rooms follow the room type, but the proximity of the objects to the walls does not follow the differences between room types found in real rooms. The likelihood distinguishes real layouts from perturbed ones partly through their geometry and partly through the agreement between the coordinates and a field that states the wall each object touches; for the movable furniture of ScanNet, it is higher for layouts with all objects piled near the room center than for the real layouts in most rooms. When an object is given in the prompt, the models place a related object relative to it. When the wall of each object is given, they place most objects against that wall, but do not keep all objects inside a given room rectangle. As a prior for object search, the generated placements reduce the searched area relative to a uniform prior, but by less than a third of the reduction obtained with a prior built from real layouts, and by less than two thirds when both priors are spatially smoothed. These results support the use of LLMs for object-room associations and for writing coordinates under constraints determined by another component, followed by a geometric check, but not as a source of placement priors where real layouts are available.
Figures & tables
| Qwen | Gemma | |||
| Move | Kitchens | Bathrooms | Kitchens | Bathrooms |
| tangential, along the wall | 0.712 | 0.742 | 0.644 | 0.711 |
| normal, away from the wall | 0.893 | 0.729 | 0.832 | 0.724 |
| normal, into the wall | 0.910 | 0.738 | 0.838 | 0.746 |
| gap, normal outward tangential | 0.181 | 0.013 | 0.188 | 0.013 |
| tangential, isolated objects | 0.656 | 0.812 | 0.595 | 0.775 |
| Keep | Claim only | Drop field | |
|---|---|---|---|
| Qwen, kitchens | 0.945 | 0.866 | 0.875 |
| Qwen, bathrooms | 0.814 | 0.857 | 0.669 |
| Gemma, kitchens | 0.892 | 0.828 | 0.849 |
| Gemma, bathrooms | 0.671 | 0.855 | 0.581 |
| Data | Model | None | Wrong | Same | Neighbor effect | Proximity effect |
|---|---|---|---|---|---|---|
| CubiCasa | Qwen | 0.925 (506) | 0.844 | 0.832 | 0.012 [ 0.003, 0.020] | 0.016 [ 0.000, 0.034] |
| Gemma | 0.866 (506) | 0.811 | 0.798 | 0.013 [ 0.004, 0.022] | 0.012 [ 0.032, 0.008] | |
| ScanNet | Qwen | 0.712 (66) | 0.638 | 0.625 | 0.013 [ 0.038, 0.064] | 0.000 [ 0.076, 0.076] |
| Gemma | 0.758 (66) | 0.593 | 0.599 | 0.006 [ 0.061, 0.051] | 0.091 [ 0.000, 0.182] |
| Level | Statistic | Qwen | Gemma |
| Object identity | Spearman vs. real | 0.75 | 0.67 |
| Object identity | Spearman vs. real, other room types | 0.03 | 0.09 |
| Room dimensions | SD ratio, model / real | 1.50 | 1.09 |
| Room dimensions | regression slope of the medians | 1.30 | 0.93 |
| Placement | SD ratio, real / model | 10.58 | 1.27 |
| Placement | , equality across room types |
| Wall-proximity excess | Outside | Overlap | |||||
| Population | Bath. | Bedr. | Office | Living | Slope | % | rate |
| Real data | 0.146 | 0.092 | 0.018 | 0.163 | — | — | — |
| Qwen A | 0.121 | 0.093 | 0.113 | 0.119 | 0.029 | 8–12 | 0.051–0.075 |
| Qwen B-neutral | 0.017 | 0.145 | 0.153 | 0.261 | 0.677 | 8–27 | 0.046–0.147 |
| Qwen B-walls | 0.182 | 0.059 | 0.148 | 0.193 | 0.188 | 34–43 | 0.043–0.077 |
| Gemma A | 0.035 | 0.190 | 0.023 | 0.185 | 0.227 | 2–17 | 0.057–0.135 |
| Qwen | Gemma | |
|---|---|---|
| desk chair | 0.694 [ 0.655, 0.733] | 0.817 [ 0.795, 0.840] |
| bed night stand | 0.794 [ 0.742, 0.842] | 0.915 [ 0.876, 0.955] |
| bed chair (control) | 0.410 [ 0.489, 0.326] | 0.544 [ 0.612, 0.480] |
| desk chair control | 1.104 [ 1.016, 1.194] | 1.362 [ 1.292, 1.434] |
| bed night stand control | 1.204 [ 1.104, 1.303] | 1.459 [ 1.390, 1.533] |
| other objects, desk chair runs | 0.134 [ 0.160, 0.108] | 0.196 [ 0.214, 0.179] |
| Kitchens | Bathrooms | ||||
|---|---|---|---|---|---|
| Model | Comparison | Pairwise acc. | Pairwise acc. | ||
| Qwen | snapped original | 0.085 | 13.180 | 0.015 | 14.282 |
| random original | 0.000 | 51.947 | 0.000 | 40.567 | |
| snapped random | 1.000 | 38.766 | 0.995 | 26.285 | |
| declared wall other wall | 0.940 | 21.969 | 0.910 | 13.849 | |
| other wall random | 0.870 | 16.797 | 0.860 | 12.436 | |
| A | A ′ | A ′′ | A ′′ -i | |
|---|---|---|---|---|
| coords first | wall first | wall given | wall withheld | |
| Qwen | 0.244 | 0.429 | 0.896 | 0.225 |
| Gemma | 0.467 | 0.905 | 0.989 | 0.349 |
| chance (Qwen) | 0.093 | 0.125 | 0.300 | 0.193 |
| objects outside the room, Qwen | 10% | 15% | 32% | 26% |
| Condition | Instruction added | Qwen | Gemma |
|---|---|---|---|
| A ′′ | wall given, nothing about containment | 32% | 13% |
| A ′′ -c | “keep the fixture inside the room” | 30% | 12% |
| A ′′ -r | “keep the fixture touching its wall” | 31% | 12% |
| A ′′ -per | containment stated per object | 28% | 12% |
| A ′′ -i | no wall requirement at all | 26% | 11% |
| Prior | Model | Bathroom | Bedroom | Office | Living room |
|---|---|---|---|---|---|
| W | — | 0.651 | 0.572 | 0.579 | 0.595 |
| M-A | Qwen | 0.425 | 0.466 | 0.444 | 0.453 |
| Gemma | 0.433 | 0.459 | 0.463 | 0.487 | |
| M-Bw | Qwen | 0.418 | 0.482 | 0.484 | 0.502 |
| Gemma | 0.489 | 0.493 | 0.485 | 0.512 | |
| M-A/room | Qwen | 0.417 | 0.465 | 0.419 | 0.424 |
| R | Qwen | Gemma | |
| Objects | 98 | 200 | 200 |
| Cells, all objects | 63 | 106 | 46 |
| Cells, equal number of objects | 51 | 65 | 31 |
| Distinct positions | 0.94 | 0.93 | 0.45 |
| Most frequent position | 0.02 | 0.02 | 0.12 |
| Room type | Prior | ||||
|---|---|---|---|---|---|
| Bathrooms | R | 0.209 | 0.153 | 0.179 | 0.262 |
| M-A, Qwen | 0.425 | 0.393 | 0.394 | 0.409 | |
| M-A, Gemma | 0.433 | 0.396 | 0.430 | 0.457 | |
| Bedrooms | R | 0.250 | 0.220 | 0.246 | 0.297 |
| M-A, Qwen | 0.466 | 0.421 | 0.427 | 0.453 | |
| M-A, Gemma | 0.459 | 0.387 | 0.380 | 0.406 |
| Level | Agreement with reality | Usefulness for navigation | Sec. |
|---|---|---|---|
| Object identity | high (Spearman 0.75 / 0.67) | used by existing systems as an affinity | 5.1 |
| Room dimensions | moderate (SD ratio 1.50 / 1.09) | not evaluated | 5.1 |
| Placement (generated) | low; does not follow the room type (slope 0.029 / 0.227) | small gain over a uniform prior | 5.1 , 7.2 |
| Placement (likelihood) | responds to geometry and to the consistency of the text | piled layout preferred to the real one on ScanNet | 4.1 , 4.2 |
| Relations (likelihood) | a change of neighbor barely registered (neighbor effect on CubiCasa 0.012 / 0.013) | not evaluated | 4.3 |
| Relations (generated) | target follows an object given in the prompt (bedrooms; desk chair 0.694 / 0.817) | not evaluated | 5.3 |
| Property | Comparison | Pairwise acc. | (nats) | Sec. |
| Geometric perturbations | ||||
| move along a wall | real vs. moved m | 0.712 | — | 4.1.2 |
| detach | real vs. detached | 0.945 | 6.47 | 4.1.1 |
| Properties of the text | ||||
| order of the entries | real vs. sorted list | 0.191 | 9.089 | 4.1.1 |
| frame | real vs. mirrored | 0.437 | 0.833 | 3.3 |
Appendix figures & tables52 assets
Supplementary material from the paper’s appendix.
Appendix
| Perturbation | Room | Pairwise acc. | 95% CI | ||
|---|---|---|---|---|---|
| transplant | Kitchen | 2,848 | 0.141 | [0.128, 0.153] | 52.30 |
| transplant | Bathroom | 4,258 | 0.954 | [0.948, 0.960] | 82.24 |
| pile | Kitchen | 2,848 | 0.955 | [0.948, 0.962] | 17.36 |
| pile | Bathroom | 4,258 | 0.955 | [0.948, 0.961] | 10.61 |
| detach † | Kitchen | 2,830 | 0.945 | [0.936, 0.953] | 6.47 |
| detach † | Bathroom | 4,258 | 0.813 | [0.802, 0.825] | 2.35 |
| Perturbation | Room | Pairwise acc. | 95% CI | ||
|---|---|---|---|---|---|
| pile | Kitchen | 2,848 | 0.928 | [0.919, 0.938] | 36.58 |
| pile | Bathroom | 4,258 | 0.862 | [0.852, 0.872] | 18.82 |
| detach † | Kitchen | 2,830 | 0.892 | [0.880, 0.903] | 15.54 |
| detach † | Bathroom | 4,258 | 0.671 | [0.657, 0.685] | 4.26 |
| swap_pos | Kitchen | 2,848 | 0.982 | [0.978, 0.987] | 40.74 |
| swap_pos | Bathroom | 4,258 | 0.950 | [0.943, 0.956] | 21.33 |
| Perturbation | Room | Pairwise acc. | 95% CI | ||
|---|---|---|---|---|---|
| pile | Bathroom | 163 | 0.589 | [0.509, 0.663] | 1.00 |
| pile | Bedroom | 272 | 0.320 | [0.265, 0.375] | 4.80 |
| pile | Office | 172 | 0.029 | [0.006, 0.058] | 13.60 |
| pile | Living room | 215 | 0.088 | [0.051, 0.130] | 14.19 |
| detach † | Bathroom | 163 | 0.785 | [0.724, 0.847] | 2.08 |
| detach † | Bedroom | 272 | 0.853 | [0.809, 0.893] | 2.12 |
| Perturbation | Room | Pairwise acc. | 95% CI | ||
|---|---|---|---|---|---|
| pile | Bathroom | 163 | 0.503 | [0.429, 0.583] | 0.43 |
| pile | Bedroom | 272 | 0.239 | [0.188, 0.290] | 16.68 |
| pile | Office | 172 | 0.256 | [0.192, 0.320] | 19.12 |
| pile | Living room | 215 | 0.288 | [0.228, 0.353] | 19.89 |
| detach † | Bathroom | 163 | 0.742 | [0.675, 0.810] | 4.70 |
| detach † | Bedroom | 272 | 0.702 | [0.647, 0.757] | 4.43 |
| Room | Edit | Pairwise acc. | 95% CI | Ties | ||
|---|---|---|---|---|---|---|
| Kitchen | tangential, m | 2,264 | 0.712 | [0.694, 0.731] | 0.20 | 1.73 |
| Kitchen | normal outward, m | 2,390 | 0.893 | [0.881, 0.905] | 0.16 | 3.46 |
| Kitchen | normal inward, m | 2,843 | 0.910 | [0.899, 0.920] | 0.00 | 3.56 |
| Kitchen | tangential, m | 2,239 | 0.724 | [0.706, 0.743] | 0.21 | 1.96 |
| Kitchen | normal outward, m | 2,374 | 0.903 | [0.890, 0.914] | 0.17 | 3.91 |
| Kitchen | tangential, m | 2,215 | 0.749 | [0.731, 0.767] | 0.22 | 2.37 |
| Model | Room | Version | Pairwise acc. | 95% CI | |
|---|---|---|---|---|---|
| Qwen | Kitchen | Keep | 2,829 | 0.945 | [0.936, 0.953] |
| Qwen | Kitchen | Claim only | 2,829 | 0.866 | [0.854, 0.879] |
| Qwen | Kitchen | Drop field | 2,829 | 0.875 | [0.863, 0.887] |
| Qwen | Bathroom | Keep | 4,255 | 0.814 | [0.802, 0.825] |
| Qwen | Bathroom | Claim only | 4,255 | 0.857 | [0.846, 0.867] |
| Qwen | Bathroom | Drop field | 4,255 | 0.669 | [0.655, 0.682] |
| Model | Room | Keep Drop | 95% CI | Keep Claim | 95% CI | |
|---|---|---|---|---|---|---|
| Qwen | Kitchen | 2,829 | 0.070 | [0.059, 0.080] | 0.078 | [0.063, 0.093] |
| Qwen | Bathroom | 4,255 | 0.145 | [0.133, 0.157] | 0.043 | [ 0.058, 0.027] |
| Gemma | Kitchen | 2,829 | 0.042 | [0.031, 0.054] | 0.064 | [0.046, 0.081] |
| Gemma | Bathroom | 4,255 | 0.090 | [0.077, 0.103] | 0.183 | [ 0.200, 0.166] |
| Room | Model | (m) | Pairwise acc. | 95% CI | Norm. | Spread | |
|---|---|---|---|---|---|---|---|
| Bathroom | Qwen | 0.2 | 163 | 0.540 | [0.466, 0.614] | 0.17 | 0.23 |
| Bathroom | Qwen | 0.35 | 163 | 0.589 | [0.509, 0.663] | 0.30 | 0.28 |
| Bathroom | Qwen | 0.6 | 163 | 0.583 | [0.509, 0.656] | 0.52 | 0.38 |
| Bathroom | Qwen | 1.0 | 163 | 0.595 | [0.515, 0.669] | 0.86 | 0.55 |
| Bathroom | Gemma | 0.2 | 163 | 0.509 | [0.436, 0.589] | 0.17 | 0.23 |
| Bathroom | Gemma | 0.35 | 163 | 0.503 | [0.429, 0.583] | 0.30 | 0.28 |
| Data | Room | anchor target | Rooms | median (m) |
|---|---|---|---|---|
| CubiCasa | Kitchen | sink dishwasher | 627 | 0.96 |
| sink wall cabinet | 620 | |||
| stove wall cabinet | 281 | |||
| sink stove | 227 | |||
| dishwasher sink | 165 | |||
| sink counter | 100 |
| Model | Rooms | None | Wrong | Same | Neighbor eff. | Proximity eff. | |
|---|---|---|---|---|---|---|---|
| Qwen | CubiCasa (all) | 4,980 | 0.925 (506) | 0.844 | 0.832 | 0.012 [0.003, 0.020] | 0.016 [0.000, 0.034] |
| Qwen | ScanNet (all) | 312 | 0.712 (66) | 0.638 | 0.625 | 0.013 [ 0.038, 0.064] | 0.000 [ 0.076, 0.076] |
| Qwen | CubiCasa kitchen | 2,532 | 0.920 (312) | 0.795 | 0.768 | 0.027 [0.013, 0.041] | 0.029 [0.006, 0.051] |
| Qwen | CubiCasa bathroom | 2,448 | 0.933 (194) | 0.895 | 0.898 | 0.004 [ 0.013, 0.006] | 0.005 [ 0.031, 0.015] |
| Qwen | ScanNet bedroom | 156 | 0.600 (30) | 0.660 | 0.583 | 0.077 [0.006, 0.147] | 0.100 [ 0.233, 0.000] |
| Qwen | ScanNet living room | 100 | 0.824 (34) | 0.660 | 0.650 | 0.010 [ 0.080, 0.100] | 0.088 [0.000, 0.206] |
| Rooms | Set | Version | Int. parts | First dec. | Spread | Overlap | Outside | Wall agr. | (m) | |
|---|---|---|---|---|---|---|---|---|---|---|
| CubiCasa kitchen | all | Real | 2,532 | 3.59 | 8.85 | 0.96 | 3.54 | 0.02 | 1.000 | — |
| all | Wrong | 2,532 | 3.65 | 9.05 | 0.97 | 3.01 | 0.02 | 0.979 | — | |
| all | Same | 2,532 | 3.64 | 9.05 | 0.96 | 3.05 | 0.02 | 0.979 | — | |
| no-neighbor subset | None | 312 | 3.65 | 8.65 | 0.99 | 1.73 | 0.02 | 0.953 | 1.21 | |
| no-neighbor subset | Wrong | 312 | 3.59 | 8.64 | 0.97 | 1.76 | 0.02 | 0.953 | 1.13 | |
| no-neighbor subset | Same | 312 | 3.57 | 8.62 | 0.97 | 1.79 | 0.02 | 0.953 | 1.07 |
| Data | Room | Triples | Same-cat. neighbor | % | Rooms with 2+ anchors |
|---|---|---|---|---|---|
| CubiCasa | Kitchen | 2,532 | 25 | 1.0 | 163 |
| CubiCasa | Bathroom | 2,448 | 25 | 1.0 | 145 |
| ScanNet | Bedroom | 156 | 6 | 3.8 | 27 |
| ScanNet | Living room | 100 | 12 | 12.0 | 61 |
| ScanNet | Bathroom | 30 | 0 | 0.0 | 3 |
| ScanNet | Office | 26 | 6 | 23.1 | 14 |
| Category | Bath. R | Bath. Q | Bath. G | Bedr. R | Bedr. Q | Bedr. G | Offi. R | Offi. Q | Offi. G | Livi. R | Livi. Q | Livi. G |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| bag | 0.11 | 0.42 | 0.00 | 0.23 | 0.52 | 0.00 | 0.21 | 0.72 | 0.60 | 0.08 | 0.52 | 0.00 |
| bathtub | 0.55 | 0.95 | 1.00 | 0.04 | 0.01 | 0.00 | 0.00 | 0.01 | 0.00 | 0.00 | 0.00 | 0.00 |
| bed | 0.00 | 0.01 | 0.00 | 0.99 | 1.00 | 1.00 | 0.00 | 0.02 | 0.00 | 0.01 | 0.00 | 0.00 |
| bookshelf | 0.00 | 0.00 | 0.00 | 0.19 | 0.72 | 0.86 | 0.43 | 0.92 | 1.00 | 0.08 | 0.97 | 1.00 |
| box | 0.05 | 0.46 | 0.03 | 0.19 | 0.56 | 0.04 | 0.52 | 0.94 | 0.10 | 0.16 | 0.78 | 0.02 |
| cabinet | 0.63 | 1.00 | 1.00 | 0.35 | 0.09 | 0.00 | 0.75 | 0.99 | 1.00 | 0.24 | 0.90 | 1.00 |
| Room type | Real | Real med. | Real IQR | Qwen med. | Qwen IQR | Gemma med. | Gemma IQR |
|---|---|---|---|---|---|---|---|
| Bathroom | 210 | 7.0 | 5.0–12.1 | 16.0 | 12.0–20.0 | 6.2 | 6.2–7.5 |
| Bedroom | 273 | 19.5 | 13.1–25.5 | 30.0 | 25.0–36.0 | 16.0 | 16.0–20.0 |
| Office | 173 | 18.1 | 14.5–21.6 | 42.0 | 30.0–56.0 | 25.0 | 25.0–30.0 |
| Living room | 224 | 27.0 | 20.7–40.5 | 42.0 | 30.0–54.0 | 25.0 | 25.0–25.0 |
| SD of the medians | 8.23 | 12.37 | 8.95 | ||||
| Ratio to real | 1.00 | 1.50 | 1.09 |
| Layouts | Slope | 95% CI | Pearson | SD ratio | 95% CI |
|---|---|---|---|---|---|
| Qwen A | 0.029 | [ 0.065, 0.120] | 0.31 | 10.58 | [4.88, 23.59] |
| Qwen B-neutral | 0.677 | [0.524, 0.832] | 0.92 | 1.36 | [1.10, 1.71] |
| Qwen B-walls | 0.188 | [ 0.358, 0.022] | 0.42 | 2.24 | [1.56, 3.43] |
| Gemma A | 0.227 | [0.097, 0.354] | 0.29 | 1.27 | [1.05, 1.53] |
| Gemma B-neutral | 0.345 | [0.201, 0.489] | 0.41 | 1.19 | [1.01, 1.39] |
| Gemma B-walls | 0.661 | [ 0.878, 0.473] | 0.53 | 0.80 | [0.67, 0.95] |
| Statistic | Qwen | Gemma |
|---|---|---|
| Wall-proximity excess | 0.029 | 0.227 |
| Orientation, | 0.437 | 1.198 |
| Orientation, | 0.182 | 0.708 |
| Orientation, | 0.232 | 1.126 |
| Orientation, | 0.772 | 1.364 |
| Orientation, combined | 0.320 | 1.124 |
| Layouts | Bathroom | Bedroom | Office | Living room | SD | (equality) |
|---|---|---|---|---|---|---|
| Real data | 0.146 | 0.092 | 0.018 | 0.163 | 0.1363 | 0.001 |
| Qwen A | 0.121 | 0.093 | 0.113 | 0.119 | 0.0129 | 0.274 |
| Qwen B-neutral | 0.017 | 0.145 | 0.153 | 0.261 | 0.0999 | 0.001 |
| Qwen B-walls | 0.182 | 0.059 | 0.148 | 0.193 | 0.0607 | 0.001 |
| Gemma A | 0.035 | 0.190 | 0.023 | 0.185 | 0.1075 | 0.001 |
| Gemma B-neutral | 0.053 | 0.313 | 0.155 | 0.257 | 0.1148 | 0.001 |
| Model | Condition | Room | Objects | Outside | Overlap |
|---|---|---|---|---|---|
| Qwen | A | Bathroom | 1,403 | 0.102 | 0.075 |
| Qwen | A | Bedroom | 1,736 | 0.124 | 0.051 |
| Qwen | A | Office | 1,872 | 0.085 | 0.060 |
| Qwen | A | Living room | 1,554 | 0.104 | 0.074 |
| Qwen | B-neutral | Bathroom | 680 | 0.269 | 0.147 |
| Qwen | B-neutral | Bedroom | 924 | 0.177 | 0.105 |
| Room | Layouts | Objects | N | E | S | W | JSD vs real |
|---|---|---|---|---|---|---|---|
| Bathroom | Real | 531 | 0.332 | 0.267 | 0.185 | 0.217 | — |
| Bathroom | qwen A | 1,397 | 0.232 | 0.271 | 0.227 | 0.270 | 0.0070 |
| Bathroom | qwen Bn | 680 | 0.260 | 0.222 | 0.281 | 0.237 | 0.0084 |
| Bathroom | qwen Bw | 647 | 0.278 | 0.261 | 0.238 | 0.223 | 0.0029 |
| Bathroom | gemma A | 1,206 | 0.219 | 0.312 | 0.119 | 0.350 | 0.0181 |
| Bathroom | gemma Bn | 769 | 0.273 | 0.202 | 0.261 | 0.264 | 0.0082 |
| Room | Layouts | Objects | N | E | S | W | JSD vs real |
|---|---|---|---|---|---|---|---|
| Bathroom | Real | 411 | 0.331 | 0.280 | 0.153 | 0.236 | — |
| Bathroom | qwen A | 371 | 0.275 | 0.291 | 0.181 | 0.253 | 0.0021 |
| Bathroom | qwen Bn | 395 | 0.266 | 0.225 | 0.306 | 0.203 | 0.0169 |
| Bathroom | qwen Bw | 500 | 0.296 | 0.262 | 0.228 | 0.214 | 0.0046 |
| Bathroom | gemma A | 624 | 0.345 | 0.314 | 0.048 | 0.293 | 0.0166 |
| Bathroom | gemma Bn | 422 | 0.355 | 0.258 | 0.344 | 0.043 | 0.0561 |
| Model | Pair | Rooms | Dropped | Used | 95% CI | (m) | Other objects | |
|---|---|---|---|---|---|---|---|---|
| Qwen | desk chair | 146 | 10 | 136 | 0.694 | [0.655, 0.733] | 3.10 | 0.134 |
| Qwen | bed night stand | 129 | 37 | 92 | 0.794 | [0.742, 0.842] | 1.76 | 0.275 |
| Qwen | bed chair (control) | 183 | 55 | 128 | 0.410 | [ 0.489, 0.326] | 1.98 | — |
| Qwen | night stand bed (reversed) | 129 | 3 | 126 | 0.027 | [ 0.048, 0.006] | 3.67 | — |
| Gemma | desk chair | 146 | 10 | 136 | 0.817 | [0.795, 0.840] | 3.10 | 0.196 |
| Gemma | bed night stand | 129 | 37 | 92 | 0.915 | [0.876, 0.955] | 1.76 | 0.310 |
| Model | Room | Comparison | Pairwise acc. | 95% CI | Ties | ||
|---|---|---|---|---|---|---|---|
| Qwen | Kitchen | snapped original | 200 | 0.085 | [0.050, 0.125] | 13.2 | 0.00 |
| Qwen | Kitchen | snapped random | 200 | 1.000 | [1.000, 1.000] | 38.8 | 0.00 |
| Qwen | Kitchen | random original | 200 | 0.000 | [0.000, 0.000] | 51.9 | 0.00 |
| Qwen | Kitchen | declared wall other wall | 200 | 0.940 | [0.905, 0.970] | 22.0 | 0.00 |
| Qwen | Kitchen | other wall random | 200 | 0.870 | [0.820, 0.915] | 16.8 | 0.00 |
| Qwen | Bathroom | snapped original | 200 | 0.015 | [0.000, 0.035] | 14.3 | 0.00 |
| Model | Room | Version / quantity | Values | Distinct | On grid | Tokens (median) |
|---|---|---|---|---|---|---|
| Qwen | Kitchen | original | 7,036 | 107 | 1.00 | 372.5 |
| Qwen | Kitchen | snapped | 7,036 | 154 | 1.00 | 372.5 |
| Qwen | Kitchen | other wall | 7,036 | 149 | 1.00 | 372.5 |
| Qwen | Kitchen | random | 7,036 | 180 | 1.00 | 372.5 |
| Qwen | Kitchen | moved objects / median / mean / p90 (m) | 1,520 | 0.75 | 1.36 | 3.50 |
| Qwen | Kitchen | clamped (snapped / random) | 87 | 9 |
| Model | Room | Grid | Gap | 5 objects | 6 objects | 7 objects | 8 objects | 9+ objects |
|---|---|---|---|---|---|---|---|---|
| Qwen | Kitchen | as emitted | 86.1 | 87.1 (52/39) | 81.3 (32/59) | 89.8 (27/99) | ||
| Qwen | Kitchen | m | 51.6 | 55.4 (52/39) | 46.4 (32/59) | 50.4 (27/99) | ||
| Qwen | Kitchen | m | 35.4 | 38.7 (52/39) | 30.7 (32/59) | 34.6 (27/99) | ||
| Qwen | Kitchen | m | 7.1 | 12.3 (52/39) | 2.7 (32/59) | 2.1 (27/99) | ||
| Qwen | Bathroom | as emitted | 56.9 | 56.9 (17/47) | ||||
| Qwen | Bathroom | m | 26.6 | 26.6 (17/47) |
| Model | Room | Layouts | Values | Distinct | On grid |
|---|---|---|---|---|---|
| — | Kitchen | real | 39,172 | 605 | 0.289 |
| Qwen | Kitchen | generated | 3,518 | 109 | 0.998 |
| — | Bathroom | real | 35,392 | 396 | 0.223 |
| Qwen | Bathroom | generated | 2,978 | 89 | 0.999 |
| Gemma | Kitchen | generated | 3,152 | 49 | 1.000 |
| Gemma | Bathroom | generated | 2,590 | 36 | 1.000 |
| Model | Cond. | Rooms | Declared | Consistency | Chance | Gap med. | Gap p90 | Outside | Overlap |
|---|---|---|---|---|---|---|---|---|---|
| Qwen | A | 799 | 3,711 | 0.244 | 0.093 | 0.55 | 2.40 | 0.102 | 0.063 |
| Qwen | A ′ | 800 | 3,626 | 0.429 | 0.125 | 0.20 | 1.25 | 0.151 | 0.060 |
| Qwen | A ′′ | 565 | 1,829 | 0.896 | 0.300 | 0.00 | 0.21 | 0.321 | 0.091 |
| Qwen | A ′′ -c | 565 | 1,829 | 0.890 | 0.293 | 0.00 | 0.22 | 0.305 | 0.099 |
| Qwen | A ′′ -r | 565 | 1,829 | 0.894 | 0.292 | 0.00 | 0.19 | 0.307 | 0.110 |
| Qwen | A ′′ -i | 565 | 1,829 | 0.225 | 0.193 | 1.15 | 3.23 | 0.265 | 0.091 |
| Room | ScanNet | Vocabulary | Door | Size | Used | Objects | No wall |
|---|---|---|---|---|---|---|---|
| Bathroom | 210 | 197 | 189 | 155 | 155 | 497 | 116 |
| Bedroom | 273 | 273 | 240 | 237 | 200 | 1,431 | 618 |
| Living room | 224 | 221 | 95 | 85 | 85 | 581 | 441 |
| Office | 173 | 173 | 146 | 125 | 125 | 1,040 | 545 |
| Total | 880 | 864 | 670 | 602 | 565 | 3,549 | 1,720 |
| Model | Cond. | Decl. N | Decl. E | Decl. S | Decl. W | Near. N | Near. E | Near. S | Near. W | JSD | Agree. | Chance | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen | A | 3,711 | 0.270 | 0.283 | 0.178 | 0.269 | 0.275 | 0.301 | 0.182 | 0.241 | 0.0005 | 0.642 | 0.257 |
| Qwen | B-neutral | 2,057 | 0.352 | 0.327 | 0.112 | 0.208 | 0.365 | 0.314 | 0.136 | 0.185 | 0.0010 | 0.732 | 0.285 |
| Qwen | B-walls | 2,932 | 0.299 | 0.278 | 0.193 | 0.231 | 0.311 | 0.268 | 0.233 | 0.188 | 0.0022 | 0.803 | 0.256 |
| Gemma | A | 3,589 | 0.296 | 0.270 | 0.127 | 0.307 | 0.259 | 0.328 | 0.113 | 0.301 | 0.0022 | 0.827 | 0.272 |
| Gemma | B-neutral | 2,086 | 0.235 | 0.339 | 0.105 | 0.320 | 0.245 | 0.321 | 0.113 | 0.321 | 0.0002 | 0.948 | 0.281 |
| Gemma | B-walls | 2,998 | 0.291 | 0.268 | 0.155 | 0.286 | 0.288 | 0.274 | 0.158 | 0.280 | 0.0001 | 0.972 | 0.262 |
| Layouts | Bathroom | Bedroom | Office | Living room | SD | Slope | 95% CI |
|---|---|---|---|---|---|---|---|
| Real data | 0.146 | 0.092 | 0.018 | 0.163 | |||
| Qwen A | 0.121 | 0.093 | 0.113 | 0.119 | 0.0129 | 0.029 | [ 0.065, 0.120] |
| Qwen A ′ | 0.019 | 0.038 | 0.051 | 0.107 | 0.0379 | 0.272 | [0.167, 0.381] |
| Gemma A | 0.035 | 0.190 | 0.023 | 0.185 | 0.1075 | 0.227 | [0.103, 0.352] |
| Gemma A ′ | 0.224 | 0.104 | 0.226 | 0.004 | 0.1067 | 0.531 | [0.435, 0.636] |
| Model | Condition | Rooms | outside | 95% CI |
|---|---|---|---|---|
| Qwen | A ′′ -c | 565 | 0.010 | [ 0.032, 0.011] |
| Qwen | A ′′ -r | 565 | 0.008 | [ 0.030, 0.014] |
| Qwen | A ′′ -per | 565 | 0.038 | [ 0.060, 0.015] |
| Qwen | A ′′ -i | 565 | 0.087 | [ 0.113, 0.061] |
| Gemma | A ′′ -c | 565 | 0.017 | [ 0.026, 0.008] |
| Gemma | A ′′ -r | 565 | 0.013 | [ 0.022, 0.003] |
| Model | Natural | Consist. | Unnatural | Consist. | None | Near wall | Uniform |
|---|---|---|---|---|---|---|---|
| Qwen | 1,437 | 0.891 | 392 | 0.911 | 1,720 | 0.163 | 0.406 |
| Gemma | 1,437 | 0.986 | 392 | 0.997 | 1,720 | 0.056 | 0.406 |
| Room type | Comparison | |||
|---|---|---|---|---|
| Bathrooms | smoothed unsmoothed, R | 0.057 | 0.030 | 0.053 |
| [ 0.070, 0.044] | [ 0.051, 0.011] | [ 0.026, 0.081] | ||
| smoothed unsmoothed, Qwen | 0.032 | 0.031 | 0.016 | |
| [ 0.042, 0.022] | [ 0.054, 0.010] | [ 0.043, 0.011] | ||
| smoothed unsmoothed, Gemma | 0.038 | 0.003 | 0.024 | |
| [ 0.057, 0.018] | [ 0.024, 0.020] | [ 0.002, 0.046] |
| Room | Prior | Rooms | Objects | Cost | 95% CI | Top-1 dist. (m) | S@10% |
|---|---|---|---|---|---|---|---|
| Bathroom | U | 187 | 493 | 0.502 | [0.502, 0.502] | 1.38 | 0.098 |
| Bathroom | W | 187 | 493 | 0.651 | [0.640, 0.660] | 1.60 | 0.008 |
| Bathroom | M-A | 187 | 493 | 0.425 | [0.399, 0.451] | 1.09 | 0.139 |
| Bathroom | M-Bw | 187 | 493 | 0.418 | [0.394, 0.443] | 1.25 | 0.134 |
| Bathroom | M-A/cat | 187 | 493 | 0.390 | [0.364, 0.415] | 1.31 | 0.178 |
| Bathroom | M-A/room | 187 | 493 | 0.417 | [0.390, 0.444] | 1.14 | 0.136 |
| Room | Prior | Rooms | Objects | Cost | 95% CI | Top-1 dist. (m) | S@10% |
|---|---|---|---|---|---|---|---|
| Bathroom | U | 187 | 493 | 0.502 | [0.502, 0.502] | 1.38 | 0.098 |
| Bathroom | W | 187 | 493 | 0.651 | [0.640, 0.660] | 1.60 | 0.008 |
| Bathroom | M-A | 187 | 493 | 0.433 | [0.410, 0.456] | 1.10 | 0.162 |
| Bathroom | M-Bw | 187 | 493 | 0.489 | [0.471, 0.506] | 1.23 | 0.126 |
| Bathroom | M-A/cat | 187 | 493 | 0.396 | [0.373, 0.420] | 1.15 | 0.190 |
| Bathroom | M-A/room | 187 | 493 | 0.437 | [0.413, 0.460] | 1.16 | 0.146 |
| Model | Difference | Bathroom | Bedroom | Office | Living room |
|---|---|---|---|---|---|
| Qwen | M-A U | 0.077 | 0.036 | 0.058 | 0.049 |
| [ 0.102, 0.048] | [ 0.050, 0.022] | [ 0.074, 0.044] | [ 0.071, 0.029] | ||
| Qwen | M-Bw U | 0.084 | 0.020 | 0.018 | 0.000 |
| [ 0.107, 0.060] | [ 0.033, 0.008] | [ 0.033, 0.004] | [ 0.021, 0.019] | ||
| Qwen | M-Bw M-A | 0.007 | 0.017 | 0.040 | 0.049 |
| [ 0.040, 0.024] | [0.002, 0.032] | [0.021, 0.060] | [0.020, 0.077] |
| Model | Prior | SD | 95% CI | R / this | (equality) |
|---|---|---|---|---|---|
| Qwen | W | 0.0357 | [0.0297, 0.0416] | 1.63 [1.19, 2.15] | 0.001 |
| Qwen | M-A | 0.0170 | [0.0081, 0.0311] | 3.15 [1.78, 7.32] | 0.063 |
| Qwen | M-Bw | 0.0366 | [0.0241, 0.0503] | 1.58 [1.04, 2.53] | 0.001 |
| Qwen | M-A/cat | 0.0491 | [0.0357, 0.0643] | 1.18 [0.80, 1.76] | 0.001 |
| Qwen | M-A/room | 0.0226 | [0.0151, 0.0338] | 2.43 [1.54, 4.13] | 0.007 |
| Qwen | R | 0.0581 | [0.0434, 0.0742] | 1.00 [1.00, 1.00] | 0.001 |
| Room | Band (m) | Predicted | Measured | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Bathroom | 0.10 | 0.272 | 0.012 | 0.260 | 0.130 | 0.130 | [0.123, 0.136] |
| Bathroom | 0.15 | 0.329 | 0.031 | 0.298 | 0.149 | 0.149 | [0.138, 0.159] |
| Bathroom | 0.20 | 0.402 | 0.091 | 0.311 | 0.156 | 0.156 | [0.138, 0.172] |
| Bathroom | 0.30 | 0.516 | 0.481 | 0.036 | 0.018 | 0.018 | [ 0.010, 0.046] |
| Bathroom | 0.50 | 0.708 | 0.871 | 0.163 | 0.081 | 0.081 | [ 0.097, 0.065] |
| Bedroom | 0.10 | 0.240 | 0.077 | 0.164 | 0.082 | 0.082 | [0.074, 0.089] |
| Category | real | Own vs pooled | Qwen | vs own | vs pooled | Closer | Gemma | vs own | vs pooled | Closer |
|---|---|---|---|---|---|---|---|---|---|---|
| chair | 1,000 | 0.0006 | 800 | 0.0099 | 0.0082 | pooled | 688 | 0.0083 | 0.0092 | own |
| cabinet | 591 | 0.0022 | 603 | 0.0020 | 0.0043 | own | 621 | 0.0091 | 0.0182 | own |
| desk | 437 | 0.0066 | 296 | 0.0161 | 0.0057 | pooled | 410 | 0.0320 | 0.0401 | own |
| table | 360 | 0.0025 | 412 | 0.0094 | 0.0032 | pooled | 314 | 0.0198 | 0.0130 | pooled |
| bed | 307 | 0.0109 | 209 | 0.0348 | 0.0213 | pooled | 200 | 0.2196 | 0.2058 | pooled |
| toilet | 210 | 0.0126 | 230 | 0.0233 | 0.0036 | pooled | 200 | 0.0829 | 0.0780 | pooled |
| Subset | Rooms | Excess | 95% CI | Median area |
|---|---|---|---|---|
| all living rooms | 221 | 0.163 | [ 0.188, 0.137] | 26.9 |
| at most one sofa | 160 | 0.176 | [ 0.205, 0.146] | 26.1 |
| two or more sofas | 61 | 0.139 | [ 0.186, 0.092] | 33.9 |
| at most three chairs | 153 | 0.153 | [ 0.185, 0.122] | 26.9 |
| four or more chairs | 68 | 0.175 | [ 0.216, 0.132] | 27.5 |
| below median area | 110 | 0.156 | [ 0.198, 0.114] | 20.5 |
| Coverage | Rooms | Bathroom | Bedroom | Office | Living room | Slope, Qwen | Slope, Gemma |
|---|---|---|---|---|---|---|---|
| all rooms | 864 | 0.146 | 0.092 | 0.018 | 0.163 | 0.029 | 0.227 |
| (197) | (273) | (173) | (221) | ||||
| 283 | 0.145 | 0.127 | 0.041 | 0.110 | 0.044 | 0.287 | |
| (80) | (98) | (73) | (32) | ||||
| 153 | 0.102 | 0.117 | 0.049 | 0.109 | 0.062 | 0.340 | |
| (51) | (47) | (40) | (15) |
Object-Goal Navigation
Object-goal navigation studies how robots find and approach requested objects in unfamiliar environments using visual, spatial, and language cues.
See all 93 papers