DINOv3 Visual Representations for Blueberry Perception Toward Robotic Harvesting

Rui-Feng Wang; Daniel Petti; Yue Chen; Changying Li

cs.CVMar 2, 2026

DINOv3 Visual Representations for Blueberry Perception Toward Robotic Harvesting

Authors: Rui-Feng Wang, Daniel Petti, Yue Chen, Changying Li

Organizations: Bio-Sensing, Automation, and Intelligence Laboratory, Department of Agricultural and Biological Engineering, Institute of Food and Agricultural Sciences, University of Florida, Gainesville, FL 32611, USA · Department of Biomedical Engineering, Georgia Institute of Technology, Atlanta, GA 30332, USA

Abstract

Vision Foundation Models trained via large-scale self-supervised learning have demonstrated strong generalization in visual perception; however, their practical role and performance limits in agricultural settings remain insufficiently understood. This work evaluates DINOv3 as a frozen backbone for blueberry robotic harvesting-related visual tasks, including fruit and bruise segmentation, as well as fruit and cluster detection. Under a unified protocol with lightweight decoders, segmentation benefits consistently from stable patch-level representations and scales with backbone size. In contrast, detection is constrained by target scale variation, patch discretization, and localization compatibility. The failure of cluster detection highlights limitations in modeling relational targets defined by spatial aggregation. Overall, DINOv3 is best viewed not as an end-to-end task model, but as a semantic backbone whose effectiveness depends on downstream spatial modeling aligned with fruit-scale and aggregation structures, providing guidance for blueberry robotic harvesting. Code and dataset will be available upon acceptance.

Figures & tables

Dataset	Total	Inst.	Train	Val.	Test	Ref.
Bruising Seg.	1001	1642	799	101	101	[ 30 ]
Cluster Det.	991	9361	855	60	76	[ 19 ]
Fruit Det.	1284	177602	898	256	130	[ 19 ]
Fruit Seg.	2772	129080	1940	554	278	[ 19 ]
Total	6048	317685	4492	971	585	–

Table 1: Datasets used for evaluating DINOv3.

Figure 1: Samples of used datasets: (a) Bruising Segmentation; (b) Cluster Detection; (c) Fruit Segmentation; (d) Fruit Detection.

Figure 2: Workflow of DINOv3-based evaluation framework.

Figure 3: Structural diagram of Patch-Level Decoders.

Dataset	Model	mIoU	Dice	P	R
Bruising	ViT-S	67.470	79.330	83.189	80.122
	ViT-S+	67.200	79.400	77.429	85.492
	ViT-B	65.171	76.810	89.103	71.272
	ViT-L	68.416	79.789	86.739	77.675
Fruit Det.	ViT-S	66.564	78.438	79.944	78.169
	ViT-S+	68.737	80.119	79.893	81.536

Table 2: Semantic segmentation results (%) on the validation set.

Dataset	Model	mIoU	Dice	P	R
Bruising	ViT-S	66.487	78.861	77.255	83.860
	ViT-S+	67.035	79.170	74.956	87.109
	ViT-B	68.022	79.300	85.305	76.885
	ViT-L	69.401	80.765	83.384	81.540
Fruit Det.	ViT-S	64.847	77.282	78.249	77.845
	ViT-S+	66.334	78.317	77.652	80.547

Table 3: Semantic segmentation results (%) on the test set.

Dataset	Model	mAP50	mAP	P	R	F1
Cluster Det.	ViT-S	1.166	0.322	30.732	3.179	5.761
	ViT-S+	1.244	0.305	23.304	3.986	6.807
	ViT-B	1.770	0.483	29.050	5.247	8.889
	ViT-L	0.750	0.203	24.348	2.825	5.063
Fruit Det.	ViT-S	11.368	2.814	25.480	35.697	29.735
	ViT-S+	13.016	3.301	21.876	39.244	28.092

Table 4: Object detection results (%) on the validation set.

Dataset	Model	mAP50	mAP	P	R	F1
Cluster Det.	ViT-S	0.000	0.000	0.000	0.000	0.000
	ViT-S+	0.000	0.000	0.000	0.000	0.000
	ViT-B	0.000	0.000	0.000	0.000	0.000
	ViT-L	0.000	0.000	0.000	0.000	0.000
Fruit Det.	ViT-S	0.898	0.256	29.230	2.246	4.171
	ViT-S+	0.844	0.216	24.839	2.765	4.977

Table 5: Object detection results (%) on the test set.

Figure 4: Visualization and PCA for bruising segmentation.

Figure 5: Segmentation and detection results with PCA analysis for the Blueberry Fruit Segmentation Dataset.

Figure 6: Visualization and PCA for cluster detection.

Figure 7: Visualization and PCA for the fruit detection.

Figure 8: Structure of AgriSight-MT Dataset Blueberry section.

Figure 9: A conceptual pipeline for blueberry cluster detection based on fruit-level evidence aggregation. Output A generates cluster proposals by fitting BBoxes to predicted foreground regions and refines them using an aggregation verification module. The verification stage evaluates fruit-level cues within each ROI (e.g., fruit spatial compactness, connectivity) to filter proposals inconsistent with the cluster definition. Output B serves as a geometric baseline, where cluster boxes are obtained solely through direct region-to-box fitting without aggregation verification.

Explore similar work

Robotic Strawberry Harvesting with Robust Vision and Deep Reinforcement Learning based Sim-to-Real Control
May 22, 2026Al Bashir, Shao-Yang Chang, Partho Ghose +3RL for Robotics Instance Segmentation
Department of Biological & Agricultural Engineering, Texas A&M University, College Station, TX 77843, USA · Department of Biomechatronics Engineering, National Taiwan University, 10617, Taipei City, Taiwan · Texas A&M AgriLife Research, Texas A&M University System, Dallas, TX 75252, USA
INSID3: Training-Free In-Context Segmentation with DINOv3
Mar 30, 2026Claudia Cuttano, Gabriele Trivigno, Christoph Reich +3Image Segmentation Few-Shot Semantic Segmentation
Politecnico di Torino · TU Darmstadt · TU Munich +2
From Vision to Harvest: Benchmarking Vision-Language Models for Multi-Arm Robotic Fruit Harvesting
Sep 11, 2026Vrishan Inukollu, Adyan Zaman, Anvi Kudaraya +4Robotics Agricultural Robotics
University of California, Merced · University of California, Davis

May 22, 2026J/K move · Enter open · S save

Robotic Strawberry Harvesting with Robust Vision and Deep Reinforcement Learning based Sim-to-Real Control

Al Bashir, Shao-Yang Chang, Partho Ghose, Prem Raj+2

Department of Biological & Agricultural Engineering, Texas A&M University, College Station, TX 77843, USA · Department of Biomechatronics Engineering, National Taiwan University, 10617, Taipei City, Taiwan · Texas A&M AgriLife Research, Texas A&M University System, Dallas, TX 75252, USA

RL for Robotics Instance Segmentation Agricultural Robotics

Open paper AskPDF arXiv

This study presents a closed-loop robotic strawberry harvesting system that combines a robust vision module, simulation-trained deep reinforcement learning (DRL) control, and ROS-based realrobot execution. For perception, we propose HRAttnEdge-YOLO26-seg, a modified YOLO26-seg architecture that incorporates a high-resolution P2 branch, segmentation-path attention, and edgesupervised prototype learning to improve instance segmentation in cluttered scenes. For control, we train a target-conditioned Proximal Policy Optimization (PPO) policy in Isaac Lab to produce smooth joint-position commands for a UR10e manipulator and deploy it on a UR10e robot for targetfruit reaching and harvesting. This simulation-based approach reduces hardware dependency, lowers development cost, and allows scalable policy training without exhaustive physical trials before real deployment. The proposed vision model demonstrated the highest overall performance among the evaluated methods. On both self-collected and public datasets, the model showed a 10 to 14% improvement in segmentation performance. In controlled in-house tests, the PPO controller produced stable and dynamically smoother motion than a inverse kinematics (IK)-based MoveIt baseline. In greenhouse trials, the proposed integrated system harvested 281 strawberries, achieving 96.6% reaching success, 91.3% grasp-and-pull success, and 84.3% overall harvesting success. These results illustrate that task-specific perception combined with simulation-trained PPO can serve as a practical and resource-efficient alternative to conventional planner-dependent reaching in manipulation, enabling reliable closed-loop robotic harvesting in complex agricultural environments.

Abstract

Explore similar work

Robotic Strawberry Harvesting with Robust Vision and Deep Reinforcement Learning based Sim-to-Real Control

INSID3: Training-Free In-Context Segmentation with DINOv3

From Vision to Harvest: Benchmarking Vision-Language Models for Multi-Arm Robotic Fruit Harvesting