GeoPilot: Diagnosis-Guided Interventions and Geometric Evaluation for Aerial Reconstruction
Organizations: Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China · School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing 100876, China
Abstract
When a mesh reconstructed from aerial remote-sensing imagery disagrees with reference geometry, deciding what to change requires a diagnosis of the reconstruction process. We present GeoPilot, a workflow that turns diagnostics and prior observations into executable program or camera-configuration interventions. A large language model proposes the change, numerical tools reconstruct the mesh, and an external evaluator measures its geometry under a stated coordinate and reference protocol. The main case tests a shared-intrinsics hypothesis proposed before the comparison. On 224 Metashape-raster development images, shared intrinsics improved all six native-coordinate measures and reduced mean surface-to-reference distance by 7.04%. Reference-assisted alignment fitted and evaluated on separate within-scene checkerboard samples reversed the mean-distance ranking, limiting that gain to native-frame delivery. A separate matched comparison on publisher-undistorted images favored fixed published cameras on all six measures, reducing mean distance by 12.11%; the cameras retain upstream image and laser-scanning adjustment information. With one reconstruction per configuration, these comparisons provide conditional evidence for camera handling. Three end-to-end runs each delivered a mesh and six-metric evaluation from all 327 images of a second development scene.
Figures & tables
| Surface | Recorded content | Execution check |
|---|---|---|
| Knowledge and experience | Sources, observations and applicability | Source identities and input compatibility |
| Diagnostics | Named measurements and artifact origins | Valid values and explicit unknown handling |
| Program structure | Calls, branches and stopping rules | Acyclic graph and stage requirements |
| Numerical arguments | Registered parameter values | Parameter types and bounds |
| Configurations and adapters | Camera settings, input bindings and source versions | Source identities and matched reconstruction settings |
| Comparison | Input and unit | Change | Readout |
|---|---|---|---|
| Camera A/B | Dataset-1 M; one mesh per arm | Per-image/shared intrinsics | Six metrics; proposal provenance; alignment sensitivity |
| S-U/F-U | Dataset-1 U; one mesh per arm | Estimated/fixed published cameras | Six metrics and image participation |
| P0/P1/P2 | Dataset-1 M; shared historical P0 | Restricted program edits | Complete vectors; retained parent |
| Experience | Dataset-2; three responses per condition | Direct, procedure, procedure plus experience | Nine recommendations and content checks |
| R0/RF/RJ | Dataset-2 U; one fresh mesh per arm | No added review/full-model/Jev review | Delivery, six metrics and complete elapsed time |
| Metric | Per-image A | Shared B | B A |
|---|---|---|---|
| Surface L1, m | 1.9897 | 1.8497 | -0.1400 |
| Surface RMSE, m | 2.3925 | 2.2614 | -0.1311 |
| Best-90% L1, m | 1.7101 | 1.5688 | -0.1413 |
| Best-90% RMSE, m | 2.0185 | 1.8767 | -0.1418 |
| Precision at 0.20 m, % | 6.3999 | 7.8461 | +1.4462 pp |
| Completeness at 0.20 m, % | 2.3646 | 3.0695 | +0.7049 pp |
| Metric | Estimated S-U | Fixed F-U | F-U S-U |
|---|---|---|---|
| Surface L1, m | 0.2059 | 0.1809 | -0.0249 |
| Surface RMSE, m | 0.3359 | 0.3241 | -0.0118 |
| Best-90% L1, m | 0.1352 | 0.1087 | -0.0264 |
| Best-90% RMSE, m | 0.1743 | 0.1465 | -0.0279 |
| Precision at 0.20 m, % | 69.7435 | 75.6188 | +5.8753 pp |
| Completeness at 0.20 m, % | 66.8045 | 72.3557 | +5.5512 pp |
| Context | Selected F-U | Valid responses |
|---|---|---|
| Direct analysis | 3/3 | 3/3 |
| GeoPilot procedure without experience | 3/3 | 3/3 |
| GeoPilot procedure with M/U experience | 3/3 | 3/3 |
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Action | Parameter | Type | Allowed range |
|---|---|---|---|
| Densify | resolution-level | Integer | 0–2 |
| Densify | number-views | Integer | 2–20 |
| Densify | number-views-fuse | Integer | 2–5 |
| Densify | iters | Integer | 1–6 |
| Densify | geometric-iters | Integer | 1–4 |
| Densify | fusion-depth-diff-threshold | Number | 0.001–0.05 |
| Level | Available in the research release | Additional requirements or limits |
|---|---|---|
| Table recomputation | Complete vectors and original source identities for Tables III–IV and B1–B2 | Recompute displayed differences from released measurements; complete original evaluation records remain in the local archive |
| Geometric rescoring | Evaluation utilities and protocol description | Original meshes, reference data and frozen runtime must be acquired separately |
| Numerical reconstruction | Selected source, numerical entry points and reproduction instructions | Raw rasters, compatible camera inputs, external binaries and experiment artifacts are required |
| Agent replay | Curated experience and selected workflow code | Complete historical agent interaction and execution records are not distributed; exact provider model snapshots are unavailable |
| Arm or attempt | Delivery status | Artifact validation | Complete six-metric result |
|---|---|---|---|
| Native-A, original attempt | Completed and exported | Validated mesh | Unavailable; evaluation interrupted before completion |
| Native-B, original attempt | Failed supervision after numerical calls | Original execution remains failed | Absent |
| Native-B, artifact diagnostic | Recovered product only; no new reconstruction | Separate diagnostic submission valid | Not started |
| Contemporary P0, original attempt | Failed during dense fusion; no mesh | No valid mesh | Not applicable |
| Contemporary P0, additional serial attempt | Completed with revised supervisor | Validated mesh | Not started |
| Review condition | F-G | F-T | J-G | J-T | Live calls |
|---|---|---|---|---|---|
| Nine questions, 4,096-token MiniMax response limit | 7/9; 67.10 | 7/9; 58.17 | 7/9; 22.79 | 7/9; 19.81 | 41 |
| Same nine questions, MiniMax client response limit removed | 7/9; 73.51 | 8/9; 54.54 | 7/9; 14.85 | 7/9; 20.13 | 38 |
| Two previously failed questions, additional execution evidence | 1/2; 25.23 | 2/2; 18.08 | 1/2; 1.69 | 1/2; 2.30 | 8 |
| Nine questions, clarified claim target | 8/9; 85.67 | 7/9; 56.89 | 7/9; 8.34 | 7/9; 31.94 | 38 |
| Same two questions, additional evidence and clarified target | 2/2; 25.64 | 2/2; 7.68 | 1/2; 1.04 | 1/2; 1.18 | 8 |
| Time or count | R0 | RF | RJ |
|---|---|---|---|
| Complete episode, min | 89.81 | 93.35 | 102.23 |
| First valid delivery observed, min | 85.41 | 89.66 | 91.76 |
| Reconstruction, min | 47.70 | 52.55 | 53.18 |
| Geometric evaluation, min | 32.30 | 30.26 | 32.55 |
| Review and coordination, min | 0.00 | 0.81 | 8.76 |
| Time inside the review-call routine, s | 0.000 | 4.371 | 1.494 |
| Transform | A: per-image intrinsics | B: shared intrinsics |
|---|---|---|
| Identity | 1.999156 | 1.848829 |
| Translation | 1.030263 | 1.100730 |
| Rigid | 1.011898 | 1.073042 |
| Similarity | 0.959510 | 1.032006 |
| Arm | Valid deliveries | Fresh BA attempts | API calls | Provider tokens | Episode time, s |
|---|---|---|---|---|---|
| R | 4/4 | 4 | 0 | N/A | 14.901 |
| L | 4/4 | 4 | 20 | 192,628 | 327.012 |
| G0 | 3/4 | 4 | 24 | 236,572 | 319.271 |
| GE | 4/4 | 4 | 25 | 360,230 | 390.101 |
| Study or component | Source location and records |
|---|---|
| P2 proposal and execution | out/geopilot-learning-20260923/proposal-format-replay/ : request.json , response.json , program.json , proposal.json , context.json , candidate.json ; p2-run/ , p2-score-r2/ , comparison-r2.json under the same dated directory |
| Context and program validation | experiments/geopilot_rsih_learning/continuation.py ; code/geopilot_rsih/program.mjs |
| Dataset-1 camera comparisons | out/geopilot-research-20260925/ : camera-six-metric-scores/ , publisher-raster-full-sparse-run/ , publisher-raster-mesh-r2-run/ , publisher-raster-r2-scores/ |
| Publisher-camera implementation | experiments/geopilot_rsih_learning/ : publisher_raster_full_sparse.py , publisher_raster_mesh_r2.py , publisher_raster_score_r2.py |
| Native proposal pilot | out/geopilot-research-20260925/native-post-prepare-pilot/ : pack-a.json , pack-b.json , response-a.txt , response-b.txt , import-a-r2/program.json , import-b-r2/program.json |
| Native-pilot execution status | Same pilot directory: candidate-a-result-review.json , candidate-b-run/run.json , candidate-b-run/supervision.json , candidate-b-artifact-diagnostic/independent-result-review.json , p0-run/result.json , p0-serial-control-result-review.json |
| Component | SHA-256 |
|---|---|
| Original request | 097d483ba704e9306fefc7aa9f97529b700ac14830e979cc20352918586ed58f |
| UTF-8 system instruction | 15731b5c144e9d7f86d3f262a6b8b27e1353b5f8aedb23704722f22b399dc712 |
| Tool knowledge | 14d3c6ab60f70d3b112443ca2cc89a933a7698eab15d79f5e1510811a4eb4e65 |
| Ordered history | 4e0fb641cd6ace0574fdd479ef5886b49f9ec303513159e2d0e151b44179404a |
| Metric | Fixed P0 | Candidate P1 | P1 P0 |
|---|---|---|---|
| Surface L1, m | 1.988085 | 2.019519 | +0.031434 |
| Surface RMSE, m | 2.392095 | 2.417309 | +0.025215 |
| Best-90% L1, m | 1.707988 | 1.741331 | +0.033343 |
| Best-90% RMSE, m | 2.017265 | 2.046680 | +0.029415 |
| Precision at 0.20 m, % | 6.406500 | 6.119500 | -0.287000 pp |
| Completeness at 0.20 m, % | 2.369538 | 2.244838 | -0.124700 pp |
| Metric | Fixed P0 | Candidate P2 | P2 P0 |
|---|---|---|---|
| Surface L1, m | 1.988085 | 1.993562 | +0.005478 |
| Surface RMSE, m | 2.392095 | 2.396214 | +0.004120 |
| Best-90% L1, m | 1.707988 | 1.714124 | +0.006136 |
| Best-90% RMSE, m | 2.017265 | 2.022759 | +0.005494 |
| Precision at 0.20 m, % | 6.406500 | 6.409800 | +0.003300 pp |
| Completeness at 0.20 m, % | 2.369538 | 2.370035 | +0.000497 pp |
| Proposal context | Responses | First choice | Valid configurations | Content violations | Geometry scored |
|---|---|---|---|---|---|
| Direct model analysis | 3 | F-U: 3/3 | 3/3 valid | 0/3 | Not executed |
| GeoPilot procedure, no experience | 3 | F-U: 3/3 | 3/3 valid | 0/3 | Not executed |
| Same procedure with frozen experience | 3 | F-U: 3/3 | 3/3 valid | 0/3 | Not executed |
| Measure | R0 | RF | RJ |
|---|---|---|---|
| Surface L1, m | 0.232631 | 0.231508 | 0.231882 |
| Surface RMSE, m | 0.693818 | 0.687521 | 0.691681 |
| Best-90% L1, m | 0.125353 | 0.125092 | 0.125199 |
| Best-90% RMSE, m | 0.166716 | 0.166361 | 0.166525 |
| Precision at 0.20 m, % | 71.740000 | 71.864200 | 71.869800 |
| Completeness at 0.20 m, % | 69.807339 | 69.868023 | 69.670118 |