cs.ROAug 19, 2026

Autonomous Agricultural Tractor: Integrated Weed Detection and LiDAR Navigation for Precision Paddy Farming

Authors: Benjamin Merryman-Smith, Tony Nguyen, Bilal Dogutas, Krish Shah, Anthony Raphael, Sudip Dhakal

Organizations: Department of Computing and Software Engineering Florida Gulf Coast University, Fort Myers, FL 33965, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Site-specific weed management in paddy farming offers substantial reductions in herbicide use over conventional broadcast spraying, but field deployment has been limited by three persistent challenges: robust crop-row navigation under canopy where GNSS degrades, real-time visual discrimination between rice and morphologically diverse weeds, and the asymmetric cost of misclassifying rice as weed, which is irreversible. This paper presents AgriNav, an integrated autonomous tractor system built around four ROS-coupled modules: a custom PyTorch reimplementation of WeedDet for rice detection, a parallel lightweight 1.68M-parameter CNN-FPN variant with asymmetric class weighting, an inverted-logic discrimination module that protects the rice class through a hardcoded confidence-gate veto, and a 6-state constant-velocity-turn-rate Extended Kalman Filter fusing GNSS, IMU, and wheel odometry with three-level outage bridging. Our primary system-level contribution is a four-mechanism LiDAR-camera fusion bridge that uses the navigation LiDAR for region-of-interest constraint, world-coordinate projection, ground-plane filtering, and bidirectional confidence fusion at zero additional hardware cost. Simulation experiments demonstrate continuous position tracking through a 20-second GNSS outage, crop row detection confidence above 0.9 throughout operation, and rice-detection confidences from 0.32 to 0.95 across paddy, aerial, and post-flood imagery. The LiDAR ROI constraint reduces detection inference region by an estimated 30 to 50 percent.

Figures & tables

Fig. 1: AgriNav system architecture. Five sensor streams (top) feed three perception and localization modules. The LiDAR row detection module provides both a navigation centerline and a region-of-interest mask back to the object detection module (red dashed arrow), forming the team’s primary system-level novel contribution. Detection results flow through the confidence gate to the spray actuator.
SplitImagesRice boxesWeed boxesRatio W:R
Train3,23214,8912,2171:6.7
Validation4041,8682771:6.7
Test4051,8602781:6.7
Total4,04118,6192,7721:6.7
TABLE I: Dataset Split Statistics
StageOutput SizeChannelsStrideFPN
Stage 196×9696\times 96322No
Stage 248×4848\times 48642No
Stage 324×2424\times 241282P3
Stage 412×1212\times 122562P4
Stage 56×66\times 65122P5
TABLE II: Lightweight Backbone Architecture (192 ×\times 192 input)
HyperparameterValue
OptimizerAdam
Initial LR5×10−45\times 10^{-4}
LR scheduleCosine annealing
Batch size4 (CPU) / 16 (GPU)
Epochs12
Gradient clipmax norm =1.0=1.0
TABLE III: Lightweight Model Training Hyperparameters
ClassAP (%)Notes
Rice24.7Adequate localization at 192×192192\times 192
Weed0.0 ∗High classification confidence; IoU is the bottleneck
mAP12.4CPU training, 192×192192\times 192 resolution
∗ Weed confidence scores confirmed up to 0.995.
Box IoU limited by low resolution; GPU training resolves this.
TABLE IV: Lightweight Model Detection Results (VOC mAP@IoU 0.5)
FieldValue
Output topic/weeddet/detections
Message typeCustom BoundingBoxArray
Box format[x1, y1, x2, y2] in pixels
Label encoding1 = rice , 2 = weed
Score range[0,1][0,1]
Weed conf. threshold0.15
TABLE V: ROS 2 Detection Interface Specification
NodeFunction
lidar_row_detectionK-means + RANSAC + EKF crop row detection from 2D LiDAR
ekf_localization6-state CVTR EKF fusing GNSS + IMU + wheel odometry
gnss_outage_bridgeGNSS health monitoring and dead-reckoning mode management
row_centerlineCenterline computation + pure pursuit lane-following controller
TABLE VI: ROS Nodes in the Lane Detection and Localization Subsystem
Fig. 2: Data flow for the lane detection and localization subsystem. Sensor data flows through detection and fusion nodes to produce centerline output and velocity commands.
MetricValue
GNSS outage duration20 s
Robot forward speed0.3 m/s
Final positional drift2.46 m
Localization methodEKF dead reckoning
Sensor fusion inputsGNSS + IMU + odometry
TABLE VII: GNSS Outage Localization Performance
Fig. 3: EKF-fused localization during a simulated 20-second GNSS outage (red shaded region). The filter maintained continuous trajectory tracking through dead reckoning with bounded positional drift relative to simulator ground truth.
Fig. 4: GNSS signal status over time. The system detects signal loss at t≈17t\approx 17 s and re-acquires at t≈35t\approx 35 s.
Fig. 5: EKF-filtered crop row detection. Top: left and right row distances. Bottom: confidence stays above 0.9 throughout, including during GNSS outage.
Fig. 6: Row centerline tracking performance. Lateral offset (top), heading error (middle), and row width (bottom).
Fig. 7: Pure pursuit controller output: linear velocity (top) and angular velocity (bottom) for lane following.
MetricValueReference
Detection Models
WeedDet architectureDet-ResNet-50 + eFPN + ERetinaHead (PyTorch reimplementation)Section III
WeedDet best training loss1.2346 over 94 epochsSection III
WeedDet rice confidence range0.32 to 0.95 across paddy / aerial / post-flood imagerySection III
Lightweight model parameters1.68M (5-stage backbone + 3-level FPN)Section IV
Lightweight training resolution192×192192\times 192 (CPU); 512×512512\times 512 GPU run pendingSection IV
TABLE VIII: AgriNav Consolidated System Results and Configuration
TopicProducerConsumer
/crop_row_stateKrish/BilalBenny (ROI)
/cmd_velKrishVehicle Control
/gnss/statusKrishAll
TABLE IX: Key ROS Topic Interfaces Between Modules
More in this topic

Agricultural Robotics

Agricultural robotics develops autonomous machines that sense, navigate, and perform tasks such as monitoring, weeding, spraying, and harvesting crops.

See all 97 papers

Explore similar work

CardsList
Apr 27, 2026

Real-time windrow detection from onboard tractor sensors for automated following

Proprietary design in commercial windrow-detection systems restricts transparency and limits progress in open autonomous forage-harvesting research. We present a multi-modal dataset combining stereo vision and LiDAR from tractor-mounted sensors during real baling operations. The dataset includes synchronized sensor data with GNSS trajectories, partly released as ROS2 Humble bags on Zenodo, with additional data available on request. Using this dataset, we implement a real-time (>20 Hz) centroid-based windrow-following method on an NVIDIA Jetson AGX Orin. Across the critical 4-10 m guidance range, stereo and LiDAR depth measurements show strong agreement (0.965 +/- 0.021), indicating that low-cost stereo sensors can approach LiDAR performance. Our open-source ROS 2 pipeline provides a reproducible benchmark for GPS-free windrow detection and supports development of practical autonomous forage-harvesting systems. Dataset: https://zenodo.org/records/17486318

Lorenz Gunreben, Nico Heider, Sebastian Zürner +2

Leipzig University, Smart Farming Lab, Grimmaische Straße 12, 04109 Leipzig

Object DetectionAgricultural Robotics
AskPDF
Jun 30, 2026

LeCropFollow: Latent Space Planning for Navigation in Unstructured Crop Fields

Unstructured navigational features, such as irregular planting or discontinuities, remain the primary failure mode for under-canopy agricultural robots. Existing geometric approaches often fail in these scenarios because they compress high-dimensional visual data into deterministic spatial references, effectively discarding the uncertainty and semantic context required to navigate ambiguous terrain. To address this, we present LeCropFollow, a visual navigation framework that bypasses explicit geometric modeling in favor of a learned latent representation. By integrating a self-supervised semantic heatmap extractor with TD-MPC2, a Model-Based Reinforcement Learning (MBRL) planner, our system optimizes trajectories directly within a latent manifold. The framework operates over the uncompressed heatmap signal, preserving the semantic context that geometric reductions discard. We demonstrate that this representational shift enables zero-shot transfer from simplified simulation to the physical world without fine-tuning. Extensive field experiments in late-stage corn fields show that LeCropFollow matches state-of-the-art baselines in unstructured rows but significantly outperforms them in plantation gaps, achieving a 2.4x reduction in semantic failures compared to keypoint-based methods. These results suggest that latent planning offers a robust alternative to geometric estimation for operations in heterogeneous agricultural environments. Code, models, and data available: https://felipe-tommaselli.github.io/lecropfollow .

Felipe Tommaselli, Francisco Affonso, Arthur Pompeu +4

Mobile Robotics Group, Center for Robotics (CRob), São Carlos School of Engineering (EESC), University of São Paulo, São Carlos, SP, Brazil · DASLab, Department of Computer Science, University of Illinois Urbana-Champaign, Champaign, IL, USA

Robot NavigationAgricultural RoboticsVision-Based Robot ControlLatent-Space Planning
AskPDF
May 11, 2026

Neural Distance-Guided Path Integral Control for Tractor-Trailer Navigation

Autonomous and safe navigation of tractor-trailer systems requires accurate, real-time collision avoidance and dynamically feasible control, particularly in cluttered and complex agricultural environments. This is challenging due to their articulated, deformable geometries and nonlinear dynamics. Traditional methods oversimplify vehicle geometry or rely on precomputed distance fields that assume a known map, limiting their applicability in dynamic, partially unknown environments. To address these limitations, we propose a geometric neural encoder that provides fast and accurate distance estimates between the full tractor-trailer body and raw LiDAR perception, enabling real-time, map-free geometric reasoning. These learned distances are integrated into a Model Predictive Path Integral (MPPI) controller, allowing the system to incorporate true articulated geometry directly into its cost evaluation and enabling more responsive navigation in challenging agricultural settings. Simulation results demonstrate that the proposed framework generates dynamically feasible and safe trajectories for navigating tractor-trailer systems in cluttered and complex environments.

Peng Wei, Chen Peng, Stavros Vougioukas

Department of Biological and Agricultural Engineering, University of California Davis, Davis, CA 95616 USA · ZJU-Hangzhou Global Scientific and Technological Innovation Center, Zhejiang University, Hangzhou, 311215, China

Robot NavigationMPPI ControlDynamic Obstacle Avoidance
AskPDF