GeoScope: Training-Free Program Execution beyond the Counting Horizon for Ultra-High-Resolution Remote Sensing Question Answering
Organizations: School of Internet, Anhui University, Hefei 230039, China · National Engineering Research Center for Agro-Ecological Big Data Analysis & Application, Anhui University, Hefei 230601, China · School of Computer Science and Cyberspace Security, Xiangtan University, Xiangtan 411105, China · School of Architecture and Urban Planning, Shenzhen University, Shenzhen 518060, China ·
Abstract
In ultra-high-resolution (UHR) remote sensing images, the objects that questions count in a region or relate by direction often span only a few pixels. Existing methods mainly decide where a vision-language model (VLM) should look and trust its answer in the chosen view. We show that this assumption fails for counting, and measure VLMs as one calibrates a sensor. All eight open VLMs tested fall increasingly short once a view holds more than a certain number of objects, about five for most on DOTA aerial images. This counting horizon narrows as views grow and objects appear smaller. An empirical power-law response model predicts the undercount a split recovers and the spurious counts it adds, and supports the rule of splitting only views beyond the horizon. In GeoScope, a text-only language model parses the question into a typed spatial program that fixes the region and its resolution. The VLM only grounds and counts within its operating range, and directions are computed explicitly from the grounded boxes. To our knowledge, it is the first UHR method to split views by a measured limit of the VLM. Without training, it improves Qwen3-VL-8B by 16.3 points on regional counting and 10.5 on spatial relations in XLRS-Bench, and has the highest reported accuracy, trained methods included, on regional counting and on the whole benchmark by question. On UHR-Micro it improves global and regional counting by 16.8 and 36.2 points, 15.6 of the former from splitting. On MME-RealWorld counting it gains 11.4 points, and we show that an option that is almost never correct yet often chosen distorts comparisons there. Project page: https://avalon-s.github.io/GeoScope/
Figures & tables
| Model | Shortfall from | Crops in fit | |||
|---|---|---|---|---|---|
| Qwen3-VL-2B | 3–5 | 0.48 | 1.76 | 0.79 | 316 |
| Qwen3-VL-4B | 6–10 | 0.64 | 1.59 | 1.79 | 314 |
| Qwen3-VL-8B, first set | 6–10 | 0.76 | 1.36 | 1.56 | 297 |
| Qwen3-VL-8B, second set | 3–5 | 0.81 | 1.28 | 1.41 | 299 |
| Qwen3-VL-32B | 6–10 | 0.80 | 1.33 | 0.91 | 285 |
| Qwen2.5-VL-7B | 6–10 | 0.70 | 1.10 | 2.15 | 305 |
| Signed | ||||||
| Method | Cost | Median | Within 20% | WAPE | 1–5 | 160 |
| DOTA, second set, 458 crops | ||||||
| One view | 1.0 | 0.50 | 27 | 0.67 | 0.15 | 0.58 |
| Fixed 4 4 | 9.0 | 0.30 | 37 | 0.33 | +1.36 | 0.12 |
| Subdivision as in GeoScope | 10.6 | 0.33 | 35 | 0.35 | 0.15 | 0.09 |
| MM-Grounding-DINO-L | 22.8 | 0.88 | 12 | 0.82 | +4.83 | 0.49 |
| Benchmark | Task | Dev. | Test | Image side | Image sources |
| XLRS-Bench (lite) | RC | 20 | 80 | 10,000 | DOTA, MiniFrance, HRSCD, ITCVD |
| OSR | 100 | 400 | 10,000 | ||
| all 13 subtasks | 3,080 | 10,000 | |||
| MME-RealWorld-RS | counting | 245 | 981 | 6,000 | DOTA, Potsdam, Toronto |
| position, colour | 502 | 2,010 | 6,000 | ||
| UHR-Micro | five task types | 500 | 1,369 | 4,800 | DOTA-v2, FAIR1M, SODA-A, xView |
| RC (80) | OSR (400) | MME-RS count (981) | ||||||
| Method | Venue | Backbone | Acc. | Acc. | Acc. | |||
| Zero-shot | – | Qwen3-VL-8B | 47.50 | – | 33.50 | – | 25.18 | – |
| numeric options only a | – | Qwen3-VL-8B | – | – | – | – | 35.47 | – |
| Zero-shot | – | LLaVA-OV-7B | 35.00 | – | 25.00 | – | 32.31 | – |
| numeric options only a | – | LLaVA-OV-7B | – | – | – | – | 37.82 | – |
| Constant answer b | – | – | 37.50 | – | 30.25 | – | 31.50 | – |
| Method | Backbone | Absent | Largest | Other | All | to numeric |
|---|---|---|---|---|---|---|
| Constant answer B | – | 0 | 19.4 | 35.0 | 31.5 | – |
| Zero-shot | Qwen3-VL-8B | 35 | 5.4 | 31.0 | 25.2 | – |
| numeric options only | Qwen3-VL-8B | 0 | 7.2 | 43.7 | 35.5 | – |
| Zero-shot | Qwen3-VL-4B | 57 | 5.0 | 17.0 | 14.3 | – |
| numeric options only | Qwen3-VL-4B | 0 | 7.7 | 33.3 | 27.5 | – |
| Zero-shot | LLaVA-OV-7B | 18 | 29.3 | 33.2 | 32.3 | – |
| XLRS-Bench | MME-RealWorld-RS | |||||||||
| Method | Venue | Backbone | RC | OSR | Avg. | By q. | Position j | Colour j | Count | Avg. |
| Trained | ||||||||||
| GeoChat ( Kuckreja et al., 2024 ) a | CVPR’24 | Vicuna-7B | 29.0 | 24.2 | 22.9 | – | – | – | – | – |
| EarthDial ( Soni et al., 2025 ) b | CVPR’25 | Phi-3-mini | 42.0 | 24.8 | 40.3 | 33.8 | – | – | – | – |
| GeoLLaVA-8K ( Wang et al., 2025a ) | NeurIPS’25 | – | 38.0 | 35.0 | 51.5 | – | 34.90 c | 27.92 c | 22.27 c | 28.36 c |
| ZoomEarth ( Liu et al., 2026a ) | CVPR’26 | Qwen2.5-VL-3B | 35.0 d | 20.8 d | 39.2 d | – | 46.6 d | 41.9 d | 27.6 d | 38.7 d |
| Method | Backbone | Global (304) | Regional (544) | Conditional (202) | Direction (160) | Distance (159) |
|---|---|---|---|---|---|---|
| Zero-shot | ||||||
| Zero-shot | Qwen2-VL-7B | 23.06 | 11.67 | 27.83 | 52.50 | 37.74 |
| Zero-shot | Qwen2.5-VL-7B | 26.44 | 18.41 | 39.30 | 63.12 | 50.31 |
| Zero-shot | Qwen3-VL-8B | 20.37 | 12.08 | 15.96 | 68.12 | 70.44 |
| our re-run | Qwen3-VL-8B | 23.71 | 13.13 | 23.45 | 65.63 | 71.70 |
| Zero-shot | Qwen3-VL-32B | 20.70 | 12.75 | 13.67 | 76.88 | 76.10 |
| OSR | MME-RS count | |||||
|---|---|---|---|---|---|---|
| dev | test | dev | test | –20 | ||
| GeoScope | 56.0 | 44.00 | 43.3 | 36.60 | 37.29 | 35.25 |
| centre angle for directions | 46.0 | 42.00 | – | – | – | – |
| no hierarchical grounding | 49.0 | – | – | – | – | – |
| zeros from downsampled views accepted | – | – | 41.2 | 36.70 | 38.14 | 33.33 |
| no subdivision | – | – | 40.0 | 35.98 | 37.71 | 32.57 |
Appendix figures & tables38 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Splits | 6–10 | 11–20 | 21–40 | 41–80 | 81–160 | 160 | |
|---|---|---|---|---|---|---|---|---|
| Qwen3-VL-8B, first set | 0.76 | 897 | 1.10 / 1.18 | 1.22 / 1.23 | 1.34 / 1.27 | 1.22 / 1.29 | 1.27 / 1.32 | 2.11 / 1.33 |
| Qwen3-VL-8B, second set | 0.76 | 996 | 1.10 / 1.16 | 1.20 / 1.18 | 1.35 / 1.25 | 1.11 / 1.28 | 1.23 / 1.31 | 2.14 / 1.32 |
| new images | 0.76 | 609 | 1.10 / 1.16 | 1.20 / 1.18 | 1.29 / 1.26 | 1.02 / 1.29 | 1.17 / 1.31 | 2.14 / 1.28 |
| Qwen3-VL-4B | 0.64 | 1,111 | 1.20 / 1.27 | 1.30 / 1.35 | 1.60 / 1.44 | 1.67 / 1.47 | 1.48 / 1.51 | 1.74 / 1.54 |
| Qwen3-VL-2B | 0.48 | 810 | 1.10 / 1.43 | 1.20 / 1.62 | 1.83 / 1.75 | 2.17 / 1.77 | 2.50 / 1.89 | 1.83 / 1.91 |
| Qwen2.5-VL-7B | 0.70 | 871 | 1.30 / 1.23 | 1.40 / 1.32 | 1.55 / 1.37 | 1.63 / 1.39 | 1.67 / 1.43 | 1.71 / 1.45 |
| First set | Second set | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Crops | Single | Recursive | Crops | Single | Recursive | |||||||
| 1–2 | 60 | +0.07 | +0.17 | 0.00 | +0.07 | +0.10 | 55 | 0.02 | 0.02 | 0.00 | 0.02 | 0.00 |
| 3–5 | 60 | 0.00 | +0.19 | 0.00 | 0.07 | +0.26 | 59 | 0.27 | 0.28 | 0.00 | 0.38 | +0.10 |
| 6–10 | 60 | 0.25 | +0.30 | 0.33 | +0.15 | +0.47 | 57 | 0.19 | 0.14 | 0.21 | +0.01 | +0.05 |
| 11–20 | 60 | 0.29 | 0.01 | 0.24 | +0.09 | +0.14 | 57 | 0.36 | 0.15 | 0.29 | +0.07 | +0.07 |
| 21–40 | 60 | 0.35 | 0.13 | 0.19 | +0.02 | +0.04 | 57 | 0.30 | +0.06 | 0.12 | +0.06 | +0.11 |
| Objects in view | Views | Split | Zero | 1–5 | Unsplit |
|---|---|---|---|---|---|
| 6–7 | 283 | 0.64 | 0.07 | 0.29 | 0.36 |
| 8–10 | 355 | 0.84 | 0.07 | 0.10 | 0.16 |
| 11–20 | 508 | 0.93 | 0.04 | 0.03 | 0.06 |
| 21–40 | 425 | 0.95 | 0.03 | 0.02 | 0.05 |
| 41–80 | 241 | 0.97 | 0.03 | 0.00 | 0.02 |
| 81–160 | 134 | 0.93 | 0.07 | 0.01 | 0.04 |
| Median | Signed | ||||||
| Strategy | Calls | All | 20 | 20 | Within 20% | 20 | 20 |
| Qwen3-VL-8B, 937 crops | |||||||
| One view | 1.0 | 0.50 | 0.49 | 0.57 | 27 | 0.16 | 0.35 |
| Fixed 2 2 | 3.2 | 0.37 | 0.44 | 0.32 | 34 | +0.30 | 0.12 |
| Fixed 4 4 | 9.0 | 0.33 | 0.50 | 0.25 | 35 | +0.85 | +0.05 |
| Fixed 8 8 | 18.8 | 0.36 | 0.58 | 0.28 | 33 | +1.45 | +0.19 |
| Model | View | 1–2 | 3–5 | 6–10 | 11–20 | 21–40 | 41–80 | 81–160 | 160 |
|---|---|---|---|---|---|---|---|---|---|
| Qwen3-VL-2B | native | 0.01 | 0.33 | 0.36 | 0.51 | 0.60 | 0.65 | 0.75 | 0.64 |
| split | 0.01 | 0.25 | 0.23 | 0.40 | 0.40 | 0.38 | 0.48 | 0.05 | |
| Qwen3-VL-4B | native | +0.14 | +0.04 | 0.35 | 0.42 | 0.48 | 0.52 | 0.56 | 0.75 |
| split | +0.21 | +0.11 | +0.14 | 0.15 | 0.21 | 0.16 | 0.29 | 0.24 | |
| Qwen3-VL-8B, first set | native | +0.07 | 0.00 | 0.25 | 0.29 | 0.35 | 0.28 | 0.42 | 0.60 |
| split | +0.17 | +0.08 | +0.29 | 0.03 | 0.13 | 0.01 | 0.16 | 0.23 |
| Crops | View | 1–2 | 3–5 | 6–10 | 11–20 | 21–40 | 41–80 | 81–160 | 160 |
|---|---|---|---|---|---|---|---|---|---|
| First set, all | native | +0.07 | 0.00 | 0.25 | 0.29 | 0.35 | 0.28 | 0.42 | 0.60 |
| fixed | +0.07 | +0.02 | 0.32 | 0.31 | 0.36 | 0.30 | 0.43 | 0.64 | |
| Second set, all | native | 0.00 | 0.27 | 0.19 | 0.38 | 0.32 | 0.04 | 0.26 | 0.55 |
| fixed | 0.03 | 0.28 | 0.20 | 0.38 | 0.33 | 0.08 | 0.31 | 0.66 | |
| Both sets, large | native | 0.46 | 0.71 | 0.45 | 0.74 | 0.72 | 0.53 | 0.65 | 0.61 |
| fixed | 0.61 | 0.67 | 0.65 | 0.81 | 0.77 | 0.65 | 0.76 | 0.73 |
| Side | Scale | 1–2 | 3–5 | 6–10 | 11–20 | 21–40 | 41–80 | 81–160 | 160 | Zero answers | |||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 256 | 3 | error | +0.21 | +0.04 | 0.08 | 0.06 | +0.02 | +0.16 | 0.13 | +0.88 | 9 of 147 | 0.95 | 0.99 |
| crops | 48 | 42 | 45 | 34 | 31 | 22 | 12 | 3 | |||||
| 512 | 1.5 | error | +0.19 | +0.10 | 0.06 | 0.23 | 0.30 | 0.00 | 0.24 | 0.60 | 11 of 182 | 1.21 | 0.84 |
| crops | 37 | 22 | 33 | 37 | 35 | 34 | 34 | 9 | |||||
| 1,024 | 1 | error | 0.29 | 0.15 | 0.55 | 0.41 | 0.49 | 0.18 | 0.27 | 0.61 | 26 of 188 | 0.97 | 0.84 |
| crops | 21 | 35 | 21 | 21 | 37 | 32 | 44 | 33 |
| Model | 256 px, | 512 px, | 1,024 px, | 2,048 px, |
|---|---|---|---|---|
| Qwen3-VL-2B | 0.33 | 0.42 | 0.65 | 0.61 |
| Qwen3-VL-4B | 0.30 | 0.35 | 0.56 | 0.57 |
| Qwen3-VL-8B | 0.05 | 0.20 | 0.48 | 0.64 |
| Qwen2.5-VL-7B | 0.28 | 0.36 | 0.61 | 0.65 |
| InternVL3.5-4B | 0.01 | 0.06 | 0.20 | 0.38 |
| InternVL3.5-8B | 0.01 | 0.02 | 0.11 | 0.08 |
| Exact (%) | Mean error | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Condition | 1–2 | 3–5 | 6–10 | 11–20 | All | 1–2 | 3–5 | 6–10 | 11–20 | Unanswered |
| Direct question | 33 | 15 | 7 | 10 | 16.3 | +0.08 | 0.15 | 0.21 | 0.16 | 0 |
| Enumerate-first | 40 | 20 | 8 | 2 | 17.5 | +0.50 | 0.26 | 0.42 | 0.59 | 24 |
| Qwen3-VL-8B-Thinking | 48 | 20 | 12 | 3 | 20.8 | +0.10 | 0.16 | 0.32 | 0.49 | 1 |
| Measure | Condition | 1–2 | 3–5 | 6–10 | 11–20 | 21–40 | 41–80 | 81–160 | 160 |
|---|---|---|---|---|---|---|---|---|---|
| Mean signed relative error | count, one view | 0.13 | 0.01 | 0.18 | 0.29 | 0.39 | 0.21 | 0.48 | 0.64 |
| enumerate, one view | +0.50 | +4.45 | +1.55 | +1.30 | +0.58 | +0.19 | 0.37 | 0.75 | |
| count, leaves | 0.13 | +0.26 | +0.36 | 0.13 | 0.23 | +0.01 | 0.27 | 0.24 | |
| enumerate, leaves | +0.50 | +4.77 | +3.43 | +0.94 | +0.47 | +1.41 | +0.28 | +0.38 | |
| Median absolute relative error | count, one view | 0.00 | 0.55 | 0.42 | 0.22 | 0.53 | 0.77 | 0.34 | 0.83 |
| enumerate, one view | 0.00 | 0.45 | 0.83 | 0.62 | 0.94 | 0.77 | 0.50 | 0.90 |
| Crops | View | 1–2 | 3–5 | 6–10 | 11–20 | 21–40 | 41–80 | 81–160 |
|---|---|---|---|---|---|---|---|---|
| VisDrone, full detail | native | +0.09 | 0.01 | 0.03 | 0.06 | 0.14 | 0.20 | 0.23 |
| split | +0.09 | 0.01 | +0.06 | +0.06 | +0.06 | +0.03 | 0.08 | |
| VisDrone, factor 1.66 | native | +0.08 | 0.09 | 0.07 | 0.11 | 0.19 | 0.21 | 0.25 |
| split | +0.08 | 0.09 | 0.00 | 0.01 | 0.03 | 0.02 | 0.17 | |
| VisDrone, factor 3 | native | 0.06 | 0.11 | 0.14 | 0.17 | 0.33 | 0.36 | 0.45 |
| split | 0.06 | 0.11 | 0.09 | 0.08 | 0.15 | 0.21 | 0.30 |
| Class | View | 1–2 | 3–5 | 6–10 | 11–20 | 21–40 | 41–80 | 81–160 | 160 |
|---|---|---|---|---|---|---|---|---|---|
| FAIR1M car | native | +52.00 (4) | 0.16 (8) | 0.01 (12) | 0.11 (33) | 0.31 (49) | +0.04 (58) | 0.36 (58) | 0.55 (57) |
| split | +7.50 (4) | 0.16 (8) | 0.03 (12) | 0.03 (33) | 0.00 (49) | +0.12 (58) | 0.11 (58) | 0.19 (57) | |
| FAIR1M truck | native | 0.51 (40) | 0.52 (38) | 0.78 (41) | 0.84 (24) | 0.95 (11) | 0.84 (1) | – | 0.67 (2) |
| split | 0.51 (40) | 0.46 (38) | 0.69 (41) | 0.85 (24) | 0.94 (11) | 0.79 (1) | – | 0.31 (2) | |
| FAIR1M ship | native | 0.22 (16) | 0.15 (14) | 0.53 (7) | 0.80 (3) | – | +0.92 (1) | 0.93 (2) | 1.00 (1) |
| split | 0.22 (16) | 0.29 (14) | 0.53 (7) | 0.80 (3) | – | +0.92 (1) | +0.27 (2) | 1.00 (1) |
| Operator | Evaluation |
|---|---|
| detect | boxes of grounded in the current window, coarse to fine (Section S3.3 ) |
| refer | grounded as a whole, in the whole image or, if is a place, in a window around it that is 15% larger than the place on each side and at least 1,024 pixels wide; an image part is appended to as a phrase. Each box is grounded again with in a close-up of at least 1,024 pixels and three times its size. One box is kept, or up to four if is true |
| in_region | members of whose centre lies in ; is grounded in the box of enlarged by 15% on each side |
| filter | members of within of the centre of whose direction from it lies within of , or within for CENTER |
| near | members of within of the centre of |
| top1 | the first member of in the order of the VLM’s answer |
| Question phrase | Program | Example |
|---|---|---|
| object noun with its colour and shape words | class phrase cls , singular, without positions, counts or relations | white storage tank |
| in the entire image | detect as the counted set | How many airplanes are there in the entire image? |
| inside or within the red circle | in_region with red_circle | How many wind turbines are there inside the red circle? |
| position in the image frame | in_region with part ; relation prompt: within a part | in the upper left corner of the picture , UL |
| position relative to another object | filter with that object as anchor | below the long white building , BOTTOM |
| near another object | near ; relation prompt: part of expr | the yellow bus near the gas station |
| Kind | Words or patterns | Covered if |
|---|---|---|
| Colour | red, blue, white, black, gray, grey, green, yellow, orange, brown, purple, pink, silver, golden, dark, light, beige, cyan | the word occurs in a class phrase or referring expression; gray and grey count as one word |
| Shape and size | square, rectangular, circular, round, triangular, L-shaped, T-shaped, U-shaped, oval, long, large, small, big, tall, irregular, curved, straight | the word occurs in a class phrase or referring expression |
| Place | parking lot, parking space, road, highway, street, river, bridge, lake, pond, harbor, harbour, port, dock, pier, field, park, square, runway, stadium, court, playground, forest, grassland, greenland, grass, lawn, water, shore, beach, yard, roundabout, intersection, wall, roof, building, factory, residential area, town, village, farmland; also in the plural | the noun occurs within a class phrase or referring expression |
| Position | upper or top left; upper or top right; lower or bottom left; lower or bottom right; left or right before side, edge, part, area or half, or after on the , for LEFT and RIGHT ; upper or top before side, edge, part, area, half, of the picture or of the image , or at the top , for TOP , and likewise lower or bottom for BOTTOM ; center, centre, middle or central, for CENTER | the token is a part or predicate of the program, both tokens of a corner are, or the matched words occur in a referring expression |
| Step | Trigger | RC | OSR | Count |
| Synthesis | ||||
| Repair | a known slip makes the program ill typed | 0 | 3 | 92 |
| Second synthesis | the program is ill typed after repair | – | – | 13 |
| the program misses a word of the question | – | – | 458 | |
| Execution | ||||
| Red circle | no red ring is found | 3 | – | – |
| Task | Prompt | Well typed | Identical | Same structure | Accuracy | |
|---|---|---|---|---|---|---|
| RC | final | 100.0 | – | – | 60.0 | – |
| no guidance | 100.0 | 100.0 | 100.0 | 60.0 | [ , ] | |
| every other example | 100.0 | 100.0 | 100.0 | 60.0 | [ , ] | |
| OSR | final | 100.0 | – | – | 56.0 | – |
| no guidance | 100.0 | 33.0 | 57.0 | 50.0 | [ , ] | |
| every other example | 95.0 | 40.0 | 56.0 | 46.0 | [ , ] |
| Synthesiser | Input | OSR | MME-RS count | RC |
|---|---|---|---|---|
| Perception by Qwen3-VL-8B | ||||
| Qwen3-4B | question | 56.0 | 43.3 | 60.0 |
| Qwen3-VL-4B | question | 49.0 | 38.8 | 60.0 |
| Qwen3-VL-4B | question and image | 51.0 | 39.6 | 60.0 |
| Qwen3-VL-8B | question | 49.0 | 37.6 | 60.0 |
| Qwen3-VL-8B | question and image | 52.0 | 38.0 | 60.0 |
| Input | Answer | Position | Colour | Counting | RC | OSR |
|---|---|---|---|---|---|---|
| ( ) | ( ) | ( ) | ( ) | ( ) | ||
| 4 M px | logits (ours) | 73.71 | 65.74 | 26.53 | 45.00 | 23.00 |
| 4 M px | generated | 72.91 | 64.14 | 29.39 | 30.00 | 20.00 |
| 1 M px | logits | 62.55 | 54.98 | 17.14 | 35.00 | 25.00 |
| 4 M px | official | 73.71 | 64.14 | 26.94 | – | – |
| 1 M px | official | 61.75 | 56.97 | 19.59 | – | – |
| Task | Strongest training-free result | Reported by | Reproduced | Backbones |
|---|---|---|---|---|
| RC | WeaveEarth, 47.0 | RS-OPSD | WeaveEarth | Qwen3-VL-8B |
| OSR | GazeEarth, 37.8 | GazeEarth | GazeEarth | Qwen3-VL-8B |
| XLRS-Bench, by question | GazeEarth, 53.51 | GazeEarth | GazeEarth | Qwen3-VL-8B |
| MME-RS count | RAP, 40.25 | ZoomSearch | RAP | LLaVA-OV-7B, Qwen3-VL-8B |
| ZoomSearch, 39.15 | ZoomSearch | ZoomSearch | LLaVA-OV-7B | |
| Not reproduced | Strongest reported results | Reason | ||
| Method | Answering model | Other model | Main settings | Tasks |
|---|---|---|---|---|
| WeaveEarth | Qwen3-VL-8B | SigLIP2 | 2,048-px thumbnail, grid, 6 tiles of 448 px | RC, OC, OSR, MME-RS all |
| GazeEarth | Qwen3-VL-8B | – | of cells, foveated 2,048-px view | RC, OC, OSR, CCR, RCCD, MME-RS all |
| the 8 other XLRS-Bench subtasks | ||||
| ZoomSearch, paper | LLaVA-OV-7B | VisRAG-Ret | , ; 10 steps | RC, OC, OSR, MME-RS all |
| ZoomSearch, code | LLaVA-OV-7B | VisRAG-Ret | , ; 10 steps | RC, OSR, MME-RS count |
| RAP | Qwen3-VL-8B | VisRAG-Ret | 10 steps, bias 0.2, crops 224 or 336 px | MME-RS count |
| Method | Backbone | Task | Reported | Ours | Re-runs by other papers |
| ZoomSearch | LLaVA-OV-7B | MME-RS position | 67.62 | 60.93 | 59.83 a , 54.97 b |
| MME-RS colour | 66.14 | 59.76 | 48.62 a , 41.67 b | ||
| MME-RS count | 39.15 | 31.50 | 20.05 a , 17.86 b | ||
| RC | – | 42.50 | 42.0 c | ||
| OSR | – | 32.50 | 32.2 c , 28.8 d , 25.8 b | ||
| XLRS-Bench OC | – | 37.50 | 46.7 c |
| Task | GazeEarth | Overview | Direct | ||
|---|---|---|---|---|---|
| MME-RS position | 67.10 | 69.78 | [ , ] | 71.67 | [ , ] |
| MME-RS colour | 63.05 | 56.47 | [ , ] | – | – |
| MME-RS count | 27.73 | 26.61 | [ , ] | – | – |
| RC | 47.50 | 52.50 | [ , ] | – | – |
| OSR | 30.75 | 28.50 | [ , ] | – | – |
| Overall counting | 12.50 | 12.50 | [ , ] | – | – |
| Text-Before-Vision (trained, Qwen2.5-VL-7B) | Qwen2.5-VL-7B | Qwen3-VL-8B | ||||
| Subtask | greedy, ours | greedy, its own | sampled | zero-shot | GeoScope | GeoScope |
| Overall counting, OC | 18.33 | 28.33 | 18.33 | 28.33 | 35.00 | 36.67 |
| Regional counting, RC | 40.00 | 49.00 | 37.00 | 44.00 | 51.00 | 63.00 |
| Object spatial relationship, OSR | 28.40 | 32.00 | 29.00 | 35.20 | 40.00 | 46.40 |
| Object classification, OCC | 40.50 | 55.63 | 36.88 | 41.75 | 82.88 | 91.25 |
| Object colour, OCL | 35.75 | 45.50 | 28.75 | 41.50 | 74.75 | 84.63 |
| All questions | Held-out 80% | |||||||
| Subtask | Category | Route | Questions | Zero-shot | GeoScope | Questions | Zero-shot | GeoScope |
| Overall counting, OC | Cnt | P | 60 | 15.00 | 36.67 | 48 | 10.42 | 35.42 |
| Regional counting, RC | Cnt | P | 100 | 47.00 | 63.00 | 80 | 47.50 | 63.75 |
| Object spatial relationship, OSR | OSR | P | 500 | 31.40 | 46.40 | 400 | 33.50 | 44.00 |
| Object classification, OCC | OP | B | 800 | 46.63 | 91.25 | 640 | 48.44 | 91.41 |
| Object colour, OCL | OP | B | 800 | 41.88 | 84.63 | 640 | 43.44 | 85.00 |
| All questions | Held-out 80% | |||||
|---|---|---|---|---|---|---|
| Category | Questions | Zero-shot | GazeEarth | GeoScope | Zero-shot | GeoScope |
| Cnt | 160 | 35.00 | 44.38 | 53.13 | 33.59 | 53.13 |
| SC | 300 | 61.67 | 66.67 | 61.67 | 62.50 | 62.50 |
| OSR | 500 | 31.40 | 37.80 | 46.40 | 33.50 | 44.00 |
| OP | 1,660 | 45.06 | 53.07 | 87.35 | 46.54 | 87.50 |
| Plan | 100 | 35.00 | 49.00 | 35.00 | 33.75 | 33.75 |
| Method | Backbone | Position | Colour | position | colour |
| Zero-shot | Qwen3-VL-8B | 74.65 | 63.55 | – | – |
| Zero-shot | LLaVA-OV-7B | 62.62 | 58.47 | – | – |
| WeaveEarth | Qwen3-VL-8B | 69.28 | 58.57 | [ , ] | [ , ] |
| GazeEarth | Qwen3-VL-8B | 67.10 | 63.05 | [ , ] | [ , ] |
| ZoomSearch | LLaVA-OV-7B | 60.93 | 59.76 | [ , ] | [ , ] |
| GeoScope | Qwen3-VL-8B | 74.65 | 63.55 |
| True count | Questions | Largest number | Smallest number |
|---|---|---|---|
| 1–2 | 285 | 0% | 44% |
| 3–5 | 415 | 21% | 13% |
| 6–10 | 195 | 52% | 4% |
| 11–20 | 66 | 41% | 5% |
| More than 20 | 15 | 40% | 0% |
| Split | Model | Questions | Absent | As given | Numbers only |
|---|---|---|---|---|---|
| Test | Qwen3-VL-8B | 981 | 35% | 25.18 | 35.47 |
| Test | Qwen3-VL-4B | 981 | 57% | 14.27 | 27.52 |
| Test | Qwen2.5-VL-7B | 981 | 45% | 18.65 | 32.52 |
| Test | LLaVA-OV-7B | 981 | 18% | 32.31 | 37.82 |
| All | Qwen3-VL-8B | 1,226 | 34% | 25.45 | 35.64 |
| All | Qwen3-VL-4B | 1,226 | 57% | 13.87 | 26.84 |
| Qwen3-VL-8B | LLaVA-OV-7B | Qwen2.5-VL-7B | |||||
|---|---|---|---|---|---|---|---|
| Task | Source | Questions | Zero-shot | GeoScope | |||
| RC | DOTA | 21 | 42.86 | 47.62 | [ , ] | [ , ] | [ , ] |
| MiniFrance | 36 | 44.44 | 77.78 | [ , ] | [ , ] | [ , ] | |
| HRSCD | 22 | 54.55 | 54.55 | [ , ] | [ , ] | [ , ] | |
| non-DOTA | 59 | 49.15 | 69.49 | [ , ] | [ , ] | [ , ] | |
| OSR | DOTA | 129 | 28.68 | 41.86 | [ , ] | [ , ] | [ , ] |
| Final | Zeros accepted | No subdivision | Fixed 4 4 grid | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Setting | Questions | Acc. | Acc. | Acc. | Acc. | ||||
| Qwen3-VL-8B, programs | all | 981 | 36.60 | 36.70 | [ , ] | 35.98 | [ , ] | 33.74 | [ , ] |
| 700 | 37.29 | 38.14 | [ , ] | 37.71 | [ , ] | 29.29 | [ , ] | ||
| 6–20 | 261 | 35.25 | 33.33 | [ , ] | 32.57 | [ , ] | 46.74 | [ , ] | |
| 15 | 40.00 | 40.00 | [ , ] | 26.67 | [ , ] | 26.67 | [ , ] | ||
| Qwen3-VL-8B, whole image | all | 981 | 32.62 | 33.33 | [ , ] | 32.72 | [ , ] | – | – |
| Questions | GeoScope | Direct answer | Difference | |
|---|---|---|---|---|
| RC | 77 | 64.94 | 68.83 | [ , 6.5] |
| OSR | 382 | 45.29 | 38.48 | 6.8 [0.8, 12.8] |
| MME-RS count | 911 | 37.43 | 24.92 | 12.5 [8.6, 16.5] |
| numeric options only | 911 | 37.43 | 34.25 | 3.2 [ , 7.2] |
| RC (20) | OSR (100) | MME-RS count (245) | |
| Zero-shot, Qwen3-VL-8B | 45.0 | 23.0 | 26.5 |
| GeoScope, Qwen3-VL-8B | 60.0 | 56.0 | 43.3 |
| abstained | 0% | 2% | 2% |
| Zero-shot, InternVL3.5-8B | 40.0 | 37.0 | 27.8 |
| GeoScope, InternVL3.5-8B | 40.0 | 42.0 | 40.8 |
| abstained | 0% | 15% | 18% |
| 1 M | 2.5 M | 4 M | |
| GeoScope, OSR (100) | 42 | 56 | 43 |
| GeoScope, MME-RS count (245) | 38.8 | 43.3 | 41.2 |
| Zero-shot, RC (20) | 35 | 40 | 45 |
| Zero-shot, OSR (100) | 25 | 25 | 23 |
| Zero-shot, MME-RS count (245) | 17.1 | 19.2 | 26.5 |
| 3 | 5 | 8 |
| Method | RC | OSR | MME-RS count |
|---|---|---|---|
| Zero-shot (Qwen3-VL-8B) | 2.8 | 3.1 | 2.8 |
| WeaveEarth | 6.9 | 6.3 | 5.6 |
| GazeEarth | 12.9 | 18.5 | 16.0 |
| ZoomSearch | 96.3 | 86.0 | 62.1 |
| RAP (Qwen3-VL-8B) | – | – | 400.1 |
| RAP (LLaVA-OV-7B) | – | – | 827.6 |
| RC (20) | OSR (60) | MME-RS count (60) | |
| Synthesis, single question (s) | 1.17 | 3.95 | 2.55 |
| Synthesis, batch of 16 (s) | 0.40 | 1.02 | 1.26 |
| Grounding (s) | – | 24.3 [6.7] | 12.6 [1.4] |
| Counting (s) | 1.8 [2.9] | – | 7.3 [16.4] |
| Execution (s) | 0.01 | 0.01 | 0.01 |
| Wall time, median / mean (s) | 1.3 / 1.8 | 17.2 / 24.3 | 6.7 / 19.8 |
| Method | Backbone | Accuracy | |
|---|---|---|---|
| Constant answer D | – | 31.25 | – |
| Second-largest number | – | 37.50 | – |
| Zero-shot | Qwen3-VL-8B | 10.42 | – |
| Zero-shot | LLaVA-OV-7B | 22.92 | – |
| WeaveEarth | Qwen3-VL-8B | 6.25 | [ , ] |
| GazeEarth | Qwen3-VL-8B | 12.50 | [ , ] |
Remote Sensing Visual Question Answering
Remote sensing visual question answering (VQA) studies how AI answers natural-language questions about satellite, aerial, and other Earth-observation imagery.
See all 41 papers