cs.RONov 12, 2026

AgriSense: Vision-Language Decision Support and Robotic Intervention in a Virtual Greenhouse Copy title

Authors: Lyes Saad Saoud, Oualid Doukhi, Reza Ghorbani, Moussa Ayyash

Organizations: Jeonbuk National University, Deokjin-gu, Jeonju-si, Jeollabuk-do, Republic of Korea · University of Hawai‘i at M¯anoa, Honolulu, Hawaii, United States · Chicago State University, Chicago, Illinois, United States

Abstract

Autonomous greenhouse management increasingly requires robotic systems that integrate perception, navigation, environmental regulation, and plant-level intervention. This paper presents AgriSense, a framework combining an Isaac Sim virtual greenhouse, a quadruped mobile manipulator, RGB and LiDAR-inertial sensing, environmental control, and supervised vision-language decision support. Frozen DINOv2 and BERT encoders, together with structured environmental inputs, support joint prediction of twelve environmental references, a target zone, and an intervention flag. Checkpoint replay reproduces 41/43 zone decisions. Across five seeds on a source-grouped test set, the full, image-removed, and language-removed configurations each achieve 43/44 correct zone decisions, providing no evidence of incremental multimodal zone-accuracy improvement. The full model achieves mean absolute errors of 0.249 degrees C for temperature, 2.030 percentage points for humidity, and 0.997 percentage points for soil moisture. Sensor-only inputs match full-model zone accuracy, while language-only inputs achieve 68.64%. External real-image benchmarks reveal distribution-shift limitations. Five offline Point-LIO routes yield rigidly aligned trajectory errors of 5.22-17.32 mm. Of 40 virtual-arm targets, 32 satisfy geometric acceptance criteria and eight are rejected at boundaries. Six long-horizon climate cases and fault protocols characterize integration limits. Navigation and integrated treatment rely on simulator-state inputs; physical greenhouse performance and agronomic treatment efficacy remain unvalidated. The contribution is an evaluated decision-support and robotic-intervention framework for a virtual greenhouse.

Figures & tables

Appendix figures & tables59 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Agri-Sim: Agricultural Simulation Platform for Embodied Intelligence Evaluation in Greenhouse Robotics

    Aug 29, 2026Shuhan Shi, Zhenfeng Xue, Minghao Mei +3Robotic HarvestingRobotics Simulation

  2. Visual-Language-Guided Task Planning for Horticultural Robots

    Jan 17, 2026Jose Cuaran, Kendall Koe, Aditya Potnis +2Large Language Model-Based Robot PlanningVision-Language Grounding

  3. Remote Human-Robot Interaction In Greenhouses via Virtual Reality: How Plant Canopy Structure Affects Leaf Disease and Soil Moisture Inspection

    Aug 27, 2026Daniel Udekwe, Hasan SeyyedhasaniRobotic InspectionHuman-Robot Interaction