cs.ROApr 7, 2026

Occlusion Handling by Pushing for Enhanced Fruit Detection

Authors: Ege Gursoy, Dana Kulić, Andrea Cherubini

Organizations: LIRMM, Univ. Montpellier, CNRS, Montpellier, France · Monash University, Clayton, Australia

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In agricultural robotics, effective observation and localization of fruits present challenges due to occlusions caused by other parts of the tree, such as branches and leaves. These occlusions can result in false fruit localization or impede the robot from picking the fruit. The objective of this work is to push away branches that block the fruit's view to increase their visibility. Our setup consists of an RGB-D camera and a robot arm. First, we detect the occluded fruit in the RGB image and estimate its occluded part via a deep learning generative model in the depth space. The direction to push to clear the occlusions is determined using classic image processing techniques. We then introduce a 3D extension of the 2D Hough transform to detect straight line segments in the point cloud. This extension helps detect tree branches and identify the one mainly responsible for the occlusion. Finally, we clear the occlusion by pushing the branch with the robot arm. Our method uses a combination of deep learning for fruit appearance estimation, classic image processing for push direction determination, and 3D Hough transform for branch detection. We validate our perception methods through real data under different lighting conditions and various types of fruits (i.e. apple, lemon, orange), achieving improved visibility and successful occlusion clearance. We demonstrate the practical application of our approach through a real robot branch pushing demonstration.

Figures & tables

Figure 1: The robot arm pushes an occluding branch to improve the view of an orange.
Figure 2: Camera field of view (left), with corresponding image (right). The tree branches and leaves are green/brown, and the fruit is orange.
Figure 3: Outline of the fruit estimation method. Left to right: Visible fruit pixels in masked depth image DM\mathbf{D}_{M} , cropped ROI depth image DM′\mathbf{D}^{\prime}_{M} fed into generative model which outputs the estimated fruit depth image, center point of the fruit (white circle) (u^c′,v^c′)(\hat{u}^{\prime}_{c},\hat{v}^{\prime}_{c}) , center point of the Mˉ′\mathbf{\bar{M}}^{\prime} (blue circle) defined as (uˉc′,vˉc′)(\bar{u}^{\prime}_{c},\bar{v}^{\prime}_{c}) shown on DˉM′\mathbf{\bar{D}}_{M}^{\prime} , occlusion direction vector O⃗f\vec{\mathbf{{O}}}_{f} (red arrow). Cross hatched area represents the area D^M′\mathbf{\hat{D}}_{M}^{\prime} , estimated by the generative model.
Figure 4: Line segments (grey) and their extremities (black circles). The pushing line lpush\mathbf{l}_{push} lpush\mathbf{l}_{push} is selected as the topmost according to Algorithm 2 .
Figure 5: Field of view from the camera (left) and Image (right). The figure shows: fruit centroid (white circle) (u^c,v^c)(\hat{u}_{c},\hat{v}_{c}) , occlusion centroid (blue circle) (uˉc,vˉc)(\bar{u}_{c},\bar{v}_{c}) , view frustum (blue cone), its center line Ffruitz\mathcal{F}^{z}_{fruit} (green arrow) and view gradient O⃗f\vec{\mathbf{O}}_{f} (red arrow).
Figure 6: Line detection before (left) and after (right) removing similar lines
Figure 7: From left to right: segmented fruit in depth image input to the fruit estimator, estimation result, difference between input and output, and view gradient (red vector) on the estimated fruit.
Figure 8: From left to right: scene, segmentation results, and line detection results. For line detection: detected and filtered lines are yellow, lines below the distance threshold are blue, and the selected line is green. The pushing point is red.
Figure 9: Robot branch pushing demonstration.

Explore similar work

CardsList

  1. GDA-YOLO11: Amodal Instance Segmentation for Occlusion-Robust Robotic Fruit Harvesting

    Feb 27, 2026Caner Beldek, Emre Sariyildiz, Son Lam Phung +1Instance SegmentationAgricultural Robotics

    School of Engineering, University of Wollongong, 2522, NSW, Australia

  2. Active perception for robotic harvesting: 3D reconstruction and localisation of tomatoes hidden within clusters in a Mediterranean greenhouse

    Sep 16, 2026Fernando Cañadas-Aránega, Rowan Border, José C. Moreno +1Active Perception3D Reconstruction

    Department of Informatics, CIESOL, ceiA3, Universidad de Almería, Ctra. Sacramento s/n, Almería, 04120, Spain · Vision for Robotics Lab (V4RL), University of Cyprus, allipoleos 75, Nicosia, 1678, Cyprus · Department of Engineering, CIESOL, ceiA3, Universidad de Almería, Ctra. Sacramento s/n, Almería, 04120, Spain

  3. HarvestPoint-ACT: Explicit Target Selection and Harvest-Point Conditioning for Robotic Fruit Harvesting under Occlusion

    Aug 19, 2026Hanying Hu, Weipeng Li, Yikun Huang +4Robotic ManipulationAgricultural Robotics

    College of Computer and Information Sciences, Fujian Agriculture and Forestry University, Fuzhou, China. · School of Mechatronic Engineering and Automation, Shanghai University, Shanghai, China. · Graduate School of Engineering Science, The University of Osaka, Osaka, Japan.