Pointillist Maize: 3D Reconstruction of Field-Grown Maize Using RGB UAV Imagery
Authors: Dewi Endah Kharismawati, Toni Kazic
Organizations: Department of Electrical Engineering and Computer Science, University of Missouri, Columbia, MO, USA · Missouri Maize Center, University of Missouri, Columbia, MO, USA · Interdisciplinary Plant Group, University of Missouri, Columbia, MO, USA · Department of Food, Agricultural and Biological Engineering, The Ohio State University, Columbus, OH, USA · MU Institute for Data Science and Informatics, University of Missouri, Columbia, MO, USA
Three-dimensional reconstruction can extend field phenotyping to individual maize organs, but thin, flexible, occluding structures and plant movement complicate reconstruction. We evaluated whether consumer-grade UAV RGB video processed with Structure from Motion and multi-view stereo could recover recognizable field-grown maize structures, and examined the effects of acquisition trajectories and frame sampling. COLMAP reconstructions preserved recognizable leaves, stems, tassel branches, ears, silks, and brace roots. For an equal-frame comparison, we reconstructed one plant from two sets of 295 frames from the same video: a single orbit sampled at 5 frames/second and three orbits at different altitudes (ribbon skirt) sampled at 2 frames/second. After ground removal, the reconstructions contained 520,601 and 375,318 points, respectively. The single orbit yielded greater point concentration within plant structures and fewer visible gaps, while the ribbon skirt yielded fewer scattered points and slightly lower proportions of isolated points. In densely planted nursery rows, visible voids occupied 17.57% of the rendered scene with one orbit at 1 frame/second, decreasing to 5.07% with additional serpentine passes and 2.68% when the original orbit was sampled at 3 frames/second. Additional observations also increased point dispersion associated with plant movement. A curlicue trajectory tested in a different field in 2024 produced distinguishable individual plants without visible duplicated structures, although substantial voids remained, mainly on the soil surface. These results demonstrate the feasibility of recovering recognizable maize organs from consumer UAV imagery and reveal trade-offs between acquisition strategies, reconstruction completeness, and point dispersion. Independent geometric validation is still needed before quantitative phenotyping.
Figures & tables
Scene
Trajectory
camera pose
frames/second
frames
purpose
plant
one orbital level
single level/view
5
295
equal-frame comparison
plant
three complete orbits
parallel to oblique
2
295
equal-frame comparison
plant
two orbits
parallel
1
177
wild-type example
plant
two orbits
parallel
1
129
mutant example
plant
two orbits
parallel and oblique
2
219
pose and masking examples
tiling
one orbit
oblique
1
150
separated plant group
Table 1: Verified video subsets represented in the manuscript sources. The two 295-frame configurations came from the same source video. Other rows are not controlled single-factor comparisons. Frames/second is the rate at which frames were extracted from the 30 frames/second source video.
Figure 1: Three-dimensional reconstruction of an individual W23 maize plant using two orbital trajectories with the same parallel camera orientation. (a) Representative RGB frame showing the field-grown plant. (b) Estimated camera poses and sparse point cloud from the two orbital acquisitions. (c) Dense point cloud showing recognizable plant architecture, including the stem, curved leaves, brace roots, a senescent lower leaf, and the tassel.
Figure 2: Three-dimensional reconstruction of an individual maize plant using two orbital trajectories with parallel and oblique camera orientations. (a) Representative RGB frame. (b) Estimated camera poses and sparse point cloud showing the two orbital acquisitions. (c) Dense point cloud showing the reconstructed plant morphology. (d,e) Detailed views of the reconstructed tassel, ear, and silks. (f,g) Surface meshes generated using the ball pivoting algorithm, showing recognizable structures in the ear and tassel regions alongside areas of incomplete surface continuity. In both (a) and (c), the red arrows identify a leaf torn along the midrib, with the broken midrib protruding above the leaf blade, while the white arrows identify a leaf that moved considerably during acquisition.
Metric
One orbit, 5 fps
Ribbon skirt, 2 fps
Retained points
520,601
375,318
Bounding-box diagonal
1.895069
1.857731
Median nearest-neighbor distance
0.001304
0.001243
Median Ni(5s)
71
66
Median Ni(10s)
401
339
Ni(3s)=0
2.696%
2.411%
Table 2: Point-density and isolation diagnostics for the equal-frame reconstructions. Both configurations used 295 frames from the same source video. The common reference spacing, s=0.001279 reconstructed coordinate units, was calculated as the median of the pooled nearest-neighbor distances from both point clouds. Ni(r) represents the number of neighboring points within radius r , excluding the query point itself. The metrics characterize local point concentration and isolation but do not directly measure the completeness or geometric accuracy of reconstructed plant organs.
Figure 3: Comparison of two acquisition configurations using an equal number of frames from the same UAV video. (a) Single-orbit trajectory at approximately plant height and ribbon skirt trajectory consisting of three orbits at different elevations and camera orientations. (b) The single-orbit reconstruction contains more points and fewer visible gaps within the plant structure, but also shows more stray points outside the plant. Brown points along some plant boundaries are consistent with the soil background visible from the downward camera orientation. (c) The ribbon skirt reconstruction contains fewer scattered points outside the plant, but shows more gaps within the reconstructed plant structure. Blue points near some plant boundaries are consistent with the sky background visible from approximately parallel viewpoints. Both reconstructions used 295 frames. Ground and soil points were removed manually using comparable procedures in MeshLab, while above-ground scattered points were left unedited. The displayed point counts correspond to the clouds after ground removal.
Figure 4: Three-dimensional reconstruction of a separated plant tiling using one circumferential orbit with an oblique camera orientation at approximately 5–7 m altitude. (a) Representative RGB frame showing several spatially separated maize plants. (b) Estimated camera poses and sparse point cloud. (c) Dense point cloud showing recognizable individual plants and surrounding field features, including tire tracks and the edge of a neighboring genetic nursery.
Figure 5: Close views of four reconstruction experiments conducted on densely planted maize nursery ranges. Panels (a)–(c) show reconstructions from the same nursery range in 2022, while panel (d) shows the curlicue reconstruction from a different field with the same planting layout in 2024. (a) A single orbital trajectory with an oblique camera orientation produced 16,248,202 points from 126 frames. Substantial voids are visible between densely planted rows. (b) Combining orbital and serpentine trajectories produced 25,369,115 points from 208 frames. Additional viewpoints reduced the visible voids, but leaf movement between passes produced less sharply defined structures. (c) Increasing the sampling rate of the original orbital trajectory to 3 frames/second produced 77,361,305 points from 426 frames. Many visible voids were reduced, although greater point dispersion appeared around the plant structures. (d) The curlicue trajectory produced 7,811,270 points from 287 frames sampled at approximately 0.5 frames/second. Individual plants appeared more distinguishable, but substantial reconstruction voids remained.
Figure 6: Examples of geometric reconstruction artifacts observed in densely planted maize nursery ranges. (a) Separated soil surfaces indicating inconsistent reconstruction of the ground. (b) Duplicated plant structures that do not correspond to distinct physical plants. These artifacts illustrate errors in scene reconstruction and registration that are separate from gaps caused by occlusion or insufficient observations.
Figure 7: Detailed views of the 2024 curlicue reconstruction. (a) Individual maize plants are distinguishable without visible duplicated or ghost plant structures. Red arrows indicate representative plants. (b) The reconstructed soil surface appears as a single layer, without visible double-layering. The red arrow indicates the reconstructed soil surface.
Figure 8: Effects of ExG masking on the reconstruction of an individual maize plant. (a) Representative RGB frame after applying the ExG mask. (b) Reconstructed point cloud showing missing plant structures following background suppression. The upper red arrow identifies the missing tassel, while the lower red arrow identifies missing brace roots. These structures were excluded because their colors did not satisfy the green vegetation mask. (c) Triangular surface mesh generated using ball pivoting. The missing structures remain absent from the surface reconstruction because the corresponding image information was removed before reconstruction.
Accurate 3D crop monitoring underpins data-driven precision agriculture by enabling field-scale analysis of plant structure, growth dynamics, and management response. Modern 3D reconstruction methods perform strongly on generic benchmarks, but rendered appearance may not translate into metrically and agronomically useful geometry in crop fields. We introduce UAV3DCrop, a public benchmark of repeated multi-angle unmanned aerial vehicle (UAV) crop surveys. It contains 88,830 RGB images at 5280×3956 pixels, with a ground sampling distance of 3.6-5.8 mm, from 91 scenes spanning corn, soybean, wheat, and oat. Track A evaluates seven scene-optimized methods -- Neural Radiance Field (NeRF) and 3D Gaussian Splatting (3DGS) variants -- on held-out views, photogrammetry-referenced depth, and canopy-height recovery. Track B tests four pretrained feed-forward models on zero-shot camera-pose and geometry estimation. The scene-optimized methods rank differently across the three targets: Splatfacto-big leads appearance, whereas Scaffold-GS leads depth and is statistically tied with Splatfacto for canopy height. Among feed-forward models, MapAnything leads on seven of the eight metrics, while the remaining models vary more across crops and fail severely on absolute scale in a way that alignment conceals. Repeated acquisitions reveal further sensitivities that differ by output type and by model, associated with position within the acquisition sequence and with tie-point multiplicity. Current 3D reconstruction methods are therefore not yet interchangeable for agronomic use: no single method wins on appearance, geometry, and canopy height at once, and only one of four feed-forward models recovers usable metric scale. The dataset is publicly available at https://link-dev.github.io/UAV3DCrop/
Junxiong Zhou, Xuechen Li, Chonghao Qiu +11
University of Minnesota, Twin Cities, USA · University of Pittsburgh, USA · Max Planck Institute for Biogeochemistry, Germany +3
Maize kernel traits such as row number, kernels per row, and kernel size vary largely for genetic reasons and are consistently associated with regions of the genome that influence yield. Manual measurement of these traits, however, cannot keep pace with the volume of maize generated in a breeding program. To address this, we developed and validated a fully automated pipeline for extracting these traits from 3D point clouds of corn ears, built on a recently developed video-to-point-cloud platform. Raw video frames are processed through COLMAP and NeRF, the ear is isolated via density-based separation, and the point cloud is distance-calibrated to physical units. The calibrated ear point cloud was Z-axis aligned via PCA and cylindrically unwrapped to a 2D image. We enhanced contrast and performed zero-fine-tuning instance segmentation using Cellpose-SAM. A triple-juxtaposed unwrap strategy was used to prevent double-counting at the seam. The pipeline achieved kernel count R^2 = 0.921 (MAPE = 10.33%) and kernel row number within +-2 rows for 95.2% of ears (MAE = 0.75 rows) on a 168-ear held-out set from the 268-ear labeled dataset. The resulting multi-trait dataset has known genotype identity for each ear, positioning it for phenotype-to-genotype association analyses.
Ritwesh A. Kumar, Som Tripathi, Peja Matthews +5
Department of Electrical and Computer Engineering, Iowa State University, Ames, IA 50011, USA · Translational AI Research and Education Center, Iowa State University, Ames, IA 50011, USA · Department of Mathematics and Computer Science, Fayetteville State University, Fayetteville, NC 28301, USA +3
Editable 3D models of field-grown crops support high-throughput phenotyping and in silico breeding trials, but building them from scanned point clouds requires organ-level segmentation and fitting. Procedural generators can turn an organ-level parameter set into an analysis-suitable 3D model, but obtaining that set requires hours of manual tuning per plant or segmentation models trained on species-specific labels. We present an automated pipeline that reconstructs procedural maize models from raw 3D point clouds without manual tuning or species-specific training data. A multimodal vision-language model (VLM) annotates leaf midlines in rendered orthographic views. Deterministic geometric algorithms back-project the annotations onto the point cloud, merge them into 3D leaves by cross-view consensus, and grow the midlines to full blades on an orientation-weighted surface graph. Measured organ parameters populate a plant descriptor for a Non-Uniform Rational B-Spline (NURBS)-based procedural model generator. Each leaf surface is then refined against its scan points by differentiable NURBS fitting. The pipeline reached a median whole-plant Chamfer distance of 5.4 mm on 100 genotypically diverse field-grown maize plants from the MaizeField3D dataset. The reconstructions were closer to the scans than those of an earlier semi-automated pipeline based on manual annotations. The pipeline recovered 1,017 of 1,023 (99.4%) curated reference leaves at an intersection-over-union of at least 0.5 without using those labels as input. These results show that VLM annotations become usable organ-level measurements when downstream geometric stages can correct them. This makes automated generation of editable 3D plant assets feasible at the scale of modern phenotyping experiments.
Mozhgan Hadadi, Talukder Z. Jubery, Adarsh Krishnamurthy +1