cs.CVMar 20, 2026

An Annotation-to-Detection Framework for Autonomous and Robust Vine Trunk Localization in the Field by Mobile Agricultural Robots

Authors: Dimitrios Chatziparaschis, Elia Scudiero, Brent Sams, Konstantinos Karydis

Organizations: Dept. of Electrical and Computer Engineering, Univ. of California, Riverside, 900 University Avenue, Riverside, CA 92521, USA · Dept. of Environmental Sciences, Univ. of California, Riverside, 900 University Avenue, Riverside, CA 92521, USA · Gallo, 600 Yosemite Blvd, Modesto, CA 95354, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

The dynamic and heterogeneous nature of agricultural fields presents significant challenges for object detection and localization, particularly for autonomous mobile robots that are tasked with surveying previously unseen unstructured environments. Concurrently, there is a growing need for real-time detection systems that do not depend on large-scale manually labeled real-world datasets. In this work, we introduce a comprehensive annotation-to-detection framework designed to train a robust multi-modal detector using limited and partially labeled training data. The proposed methodology incorporates cross-modal annotation transfer and an early-stage sensor fusion pipeline, which, in conjunction with a multi-stage detection architecture, effectively trains and enhances the system's multi-modal detection capabilities. The effectiveness of the framework was demonstrated through vine trunk detection in novel vineyard settings that featured diverse lighting conditions and varying crop densities to validate performance. When integrated with a customized multi-modal LiDAR and Odometry Mapping (LOAM) algorithm and a tree association module, the system demonstrated high-performance trunk localization, successfully identifying over 70% of trees in a single traversal with a mean distance error of less than 0.37m. The results reveal that by leveraging multi-modal, incremental-stage annotation and training, the proposed framework achieves robust detection performance regardless of limited starting annotations, showcasing its potential for real-world and near-ground agricultural applications.

Figures & tables

Fig. 1 : Our annotation-to-detection framework deployed onboard for multi-modal vine trunk detection. Integration with a customized LOAM framework enables concurrent vine trunk localization and the generation of attribute-rich sparse maps.
Fig. 2 : Our proposed annotation-to-detection framework for multi-modal vine trunk detection. The annotation pipeline initiates a tree trunk dataset in the form of annotation masks Mtm\textbf{M}_{t}^{m} by combining multi-modal input ( Itv\boldsymbol{\mathcal{I}}^{v}_{t} , Itn\boldsymbol{\mathcal{I}}^{n}_{t} , Itτ\boldsymbol{\mathcal{I}}^{\tau}_{t} ) and available spatial information during the robot’s operation. The detection pipeline is coupled with the annotated (pseudo-labeled) dataset to iteratively refine and augment the available data, towards training a robust multi-modal vine Vtm\boldsymbol{\mathcal{V}}_{t}^{m} detector.
Fig. 3 : Generated semantic masks Mtτ\mathbf{M}_{t}^{\tau} , based on a thermal image Itτ\boldsymbol{\mathcal{I}}^{\tau}_{t} input, by using SAM [ 14 ] and our shape-based mask filter.
Fig. 4 : Mask association and pseudo-labeling . By employing the interpolated point clouds Ptm\mathcal{P}_{t}^{m} and filtered masks Mtm\mathbf{M}_{t}^{m} , the vines are spatially associated to determine if they represent the same object across different modalities.
Fig. 5 : Evolution of vine trunk detection. By transitioning from the DS\mathcal{D}_{S} model, trained on the partially annotated dataset (b), to DT\mathcal{D}_{T} model, which incorporates the enriched pseudo-labeled dataset (d), the detector achieves superior performance across all input modalities.
MetricsDS\mathcal{D}_{S}DS+\mathcal{D}_{S^{+}}DT\mathcal{D}_{T}
Precision0.020.160.83
Recall0.140.350.53
mAP 500.170.370.55
mAP 50:950.080.190.26
TABLE I : Evaluation Stages of Detector D\mathcal{D} Aiming Low-False-Positive Vine Trunk Detection
Experiment Single-Row with 1010 Trees Dual-Row with 7×77\times 7 Trees
Metrics1st−5th1st-5th Tree5th−10th5th-10th TreeTotalTotal
L2L^{2} Distance Error0.32 m0.24 m0.28 m0.37 m
MAE <0.50.05 m0.11 m0.09 m0.06 m
RMSE <0.50.05 m0.12 m0.10 m0.07 m
Recall <0.560%80%80%71%
Total Tree Detections4 out of 54 out of 58 out of 1010 out of 14
TABLE II : Evaluation of Vine Trunk Localization
Fig. 6 : Field assessment of vine detection and localization. Our system accurately detects and localizes tree trunks in a single pass across both single-row (10 trees, left) and dual-row ( 7×77\times 7 trees, right) configurations. By incorporating multi-modal information into the LOAM registration process, the system generates feature-rich sparse maps −- such as the illustrated thermal visualizations −- simultaneously with real-time vine trunk detection.

Explore similar work

CardsList

  1. VinePT-Map: Pole-Trunk Semantic Mapping for Resilient Autonomous Robotics in Vineyards

    Mar 5, 2026Giorgio Audrito, Mauro Martini, Alessandro Navone +2Semantic MappingAgricultural Robotics

    Department of Electronics and Telecommunications, Politecnico di Torino, 10129, Torino, Italy.

  2. Unsupervised Memory-Enhanced Video Transformers: Obstacle Detection for Autonomous Agricultural Rover

    Jun 23, 2026Théo Biardeau, Anne-Sophie Capelle-Laizé, Salwan Alwan +1Video Anomaly DetectionAgricultural Robotics

    XLIM-ASALI, UFR SFA · Universit´e de Poitiers, CNRS, XLIM, France · Cyclair, Pressac, France +2

  3. Multimodal Urban Tree Detection from Satellite and Street-Level Imagery via Annotation-Efficient Deep Learning Strategies

    Apr 3, 2026In Seon Kim, Ali MoghimiObject DetectionActive Learning

    Department of Computer Science, University of California, Davis, , Davis, 95616, CA, USA · Department of Biological and Agricultural Engineering, University of California, Davis, Davis, CA, 95616, USA