Learning Grasp Targeting from Point Clouds for Log Pile Clearing on a Hydraulic Crane

George Sideris; Lucas Bessai; Heshan Fernando; Elie Ayoub; Nicolas Lemieux; Inna Sharf

cs.ROOct 6, 2026

Learning Grasp Targeting from Point Clouds for Log Pile Clearing on a Hydraulic Crane

Authors: George Sideris, Lucas Bessai, Heshan Fernando, Elie Ayoub, Nicolas Lemieux, Inna Sharf

Organizations: McGill University, Montreal, QC, Canada. · FPInnovations, Pointe-Claire, QC, Canada.

Abstract

In mill yards, log loaders clear dense piles by a sequence of bundle grasps: hundreds of logs rest in contact, and each removal changes the pile available to the next grasp. A learned policy chooses where to place and orient the grapple from unsegmented point clouds and runs on a trailer-mounted hydraulic forestry crane. The policy classifies at which observed point to grasp and predicts depth and grapple orientation there. The same network outputs support behavior cloning (BC), reinforcement learning (RL), and deployment. BC learns from successful top-of-pile demonstrations; RL explores for improvements by fine-tuning the cloned policy (BC $\to$ RL) or by training from scratch. In simulation, BC clears 98 of 100 piles of 200 logs, while BC $\to$ RL improves load stability. Twelve field trials compare a geometric heuristic, RL from scratch, BC, and BC $\to$ RL through complete grasp-transport-deposit cycles. BC and BC $\to$ RL deposit 93.8% and 88.9% of pooled inventory, against 80.4% for the heuristic. BC $\to$ RL deposits logs on 83.6% of its cycles, against 79.6% for the heuristic and 65.7% for BC, while its simulated stability gain does not carry over to the crane testbed. Trained entirely in simulation and run unchanged on the crane, the learned policies clear more than the hand-filtered heuristic while observing unfiltered clouds that still contain the storage rack's rails and poles.

Figures & tables

Fig. 1: Trailer-mounted hydraulic forestry crane, suspended grapple, source rack, and destination trailer. The policy selects the grasp; a common controller transports and deposits the logs.

Panel of Figure 2 — Fig. 2: Shared scoring policy through BC, RL, and deployment. Orange blocks are trained; blue blocks carry weights unchanged. Gray arrows carry weights between phases. The RL panel shows fine-tuning from BC; the dashed arrow is BC deployed directly, skipping (b). The privileged state $\xi_{k}$ contains the positions and yaws of up to 64 logs and is available only to the critic during training. RL from scratch also trains the encoder, enters (b) without BC weights, and deploys both of its blocks. MLP: multilayer perceptron; PPO: proximal policy optimization.

Panel of Figure 3 — Fig. 3: Simulation: parallel environments (left), rack and frames (middle), and actuated links with passive suspension (right). $\psi$ is grapple yaw.

Fig. 4: Scores on simulated (left) and crane (right) clouds. Brightness gives score rank, gray marks points outside the action box, and red gives the decoded target and yaw.

Panel of Figure 5 — Fig. 5: Mast-camera views on the crane (left) and in simulation (right), using the calibrated camera and rack placement.

Fig. 6: Alignment and stability. Superscripts $w$ , $g$ , and $\ell$ denote world, grapple, and log frames; insets show the shaped scores in ( 6 ).

Panel of Figure 7 — Fig. 7: Single mound, double mound, and flat rack in simulation (top) and before crane trials (bottom). Categories match, but each physical pile is rebuilt. The source rack is in front of the destination trailer.

Fig. 8: BC loss (left) and validation grasp error (right), the median horizontal distance from the decoded grasp position to the expert target, mean and standard deviation over three seeds. Training uses fresh camera noise; validation is clean. Diamonds mark the deployed checkpoint.

Fig. 9: RL return and handling, mean and standard deviation over three seeds. Training samples actions; evaluation uses the highest score and predicted depth and yaw. Diamonds mark checkpoints.

Method	$n$	$\alpha$	$\varsigma$	Succ. (%)	Full (%)	$c_{95}$	Cycles
Privileged expert	$11.51\pm 1.18$	$0.977\pm 0.010$	$0.948\pm 0.016$	$76.3\pm 16.3$	63	$16.1\pm 1.7$	$23.9\pm 5.2$
Geometric heuristic	$11.23\pm 1.05$	$0.974\pm 0.010$	$0.931\pm 0.017$	$79.8\pm 9.9$	89	$16.9\pm 1.6$	$22.9\pm 3.7$
RL from scratch	$11.01\pm 1.15$	$0.950\pm 0.016$	$0.862\pm 0.020$	$81.3\pm 9.6$	87	$18.3\pm 2.1$	$23.0\pm 4.1$
BC, regression head	$10.65\pm 1.92$	$0.979\pm 0.011$	$0.951\pm 0.015$	$60.8\pm 19.8$	33	$20.4\pm 4.6$	$27.5\pm 4.1$
BC, scoring head	$11.77\pm 1.00$	$0.980\pm 0.008$	$0.920\pm 0.024$	$91.1\pm 6.2$	98	$15.2\pm 1.4$	$18.9\pm 2.5$
BC $\to$ RL	$11.89\pm 0.81$	$0.984\pm 0.006$	$0.955\pm 0.017$	$88.3\pm 5.6$	99	$16.2\pm 1.3$	$19.2\pm 1.9$

TABLE I: Simulation on 100 shared piles (mean

\pm

standard deviation).

n

: logs per successful grasp;

\alpha

: alignment;

\varsigma

: stability; Succ.: successful-cycle rate; Full: emptied-rack rate;

c_{95}

: cycles to 95% removal in episodes reaching it.

Fig. 10: Targets from the expert and RL from scratch on one shared pile. Red points mark targets by cycle; color gives remaining height. Reported spacing medians use all episodes.

Output head
Variant	Clear (%)	$n$	$\alpha$	$\varsigma$	Argmax full (%)
Categorical over points (used)	$94.0\pm 4.8$	$9.3\pm 1.0$	$0.79\pm 0.04$	$0.69\pm 0.03$	$51.7\pm 40.0$
Gaussian over coordinates	$90.6\pm 1.2$	$9.9\pm 0.1$	$0.84\pm 0.01$	$0.65\pm 0.01$	$0.0\pm 0.0$
Reward
Multiplicative, unnormalized (used)	$94.0\pm 4.8$	$9.3\pm 1.0$	$0.79\pm 0.04$	$0.69\pm 0.03$	$51.7\pm 40.0$
Multiplicative, normalized	$76.4\pm 14.5$	$8.8\pm 0.6$	$0.83\pm 0.02$	$0.70\pm 0.00$	$5.8\pm 5.1$

TABLE II: Three-seed ablations (mean

\pm

standard deviation). Clear and handling use sampled cycles 12.5k–25k; Argmax full uses the best 40-episode deterministic screen per seed.

Fig. 11: Training return over three seeds: (a) categorical and Gaussian coordinate exploration; (b) encoder and critic variants during fine-tuning. Curves share reward within each panel.

Fig. 12: Reward ablation, RL from scratch: clearing, bundle size, alignment, and stability, mean

\pm

standard deviation over three seeds.

Panel of Figure 15 — Fig. 13: Offline targets on full (left) and depleted (right) clouds, colored by height: regression (black), margin-trained scoring (red), tight-crop scoring (blue). Inner box: tight rack crop; outer box: 0.5 m margin crop.

Fig. 14: Cumulative logs moved in each field trial. Crosses mark empty cycles, dots a cleared rack; RL from scratch includes depth offsets.

Policy	$n$	Align.	$\varsigma$	Succ. (%)	Clear (%)	$c_{95}$	Cycles
Single mound
Heuristic	$13.2\pm 4.2$	$4.42\pm 0.95$	$0.947\pm 0.049$	80.0	79.0	–	15
RL from scratch	$9.1\pm 5.6$	$4.05\pm 0.89$	$0.781\pm 0.116$	86.4	86.5	–	22
BC	$12.6\pm 6.2$	$4.23\pm 0.97$	$0.942\pm 0.033$	72.2	82.4	–	18
BC $\to$ RL	$12.5\pm 4.6$	$4.08\pm 1.07$	$0.918\pm 0.059$	81.2	81.5	–	16
Double mound

TABLE III: Twelve crane trials.

n

: logs per depositing cycle; Align.: 1–5 bundle score;

\varsigma

: stability over valid successful cycles; these three are mean

\pm

standard deviation over the trial’s successful cycles; Succ.: depositing-cycle rate; Clear: deposited inventory. RL from scratch includes depth offsets.

Fig. 15: Field campaign: (a) cumulative logs moved, mean

\pm

standard deviation over one trial per shape, ended trials holding their final value; (b) empty-cycle causes by early (E), middle (M), and late (L) thirds.

Explore similar work

Calibrated Probabilistic Obstruction Reasoning with Vision-Language Models for Grasping in Clutter
Sep 16, 2026Thanh-Tuan Tran, Ngoc-Chien Chu, Thanh Nguyen Canh +3Robotic Grasping Perception Gap
University of Engineering and Technology, Vietnam National University, Hanoi, 10000, Vietnam · School of Information Science, Japan Advanced Institute of Science and Technology, Ishikawa, 923-1211, Japan · Department of Robotics, Hanyang University, Gyeonggi-do, 15588, Korea
Learning Beyond What Humans Can Demonstrate
Sep 21, 2026Yuchen Song, Aditya Mittal, Unnat JainTeleoperation Behavior Cloning
University of California, Irvine
Whole-Body Aerial Grasping and Lifting via Partial Visual Observations
Sep 30, 2026Jiaye Jin, Rui Jin, Xinhang Xu +6Robotic Grasping Aerial Manipulation
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798 · College of Electronics and Information Engineering, Shanghai Institute of Intelligent Science and Technology, Tongji University, Shanghai 201804, China · NTU–VinUni Joint Research Laboratory for Embodied AI and Robotics, School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore 639798, and VinUniversity, Hanoi, Vietnam

Sep 16, 2026, cs.ROJ/K move · Enter open · S save

Calibrated Probabilistic Obstruction Reasoning with Vision-Language Models for Grasping in Clutter

Thanh-Tuan Tran, Ngoc-Chien Chu, Thanh Nguyen Canh, Nak Young Chong+2

University of Engineering and Technology, Vietnam National University, Hanoi, 10000, Vietnam · School of Information Science, Japan Advanced Institute of Science and Technology, Ishikawa, 923-1211, Japan · Department of Robotics, Hanyang University, Gyeonggi-do, 15588, Korea

Robotic Grasping Perception Gap Probabilistic Model

Open paper AskPDF arXiv

Retrieving a target from clutter requires deciding whether to grasp the target, remove a blocker, or defer. Existing methods typically commit to a single obstruction graph or removal strategy, ignoring uncertainty across alternative scene interpretations. They also rely on miscalibrated vision-language model (VLM) predictions and can produce pairwise obstruction relations that are jointly inconsistent. Moreover, current approximations provide no guarantees about the impact of discarded hypotheses on the final decision. We propose CPOR-Grasp, a calibrated probabilistic obstruction-reasoning framework that propagates uncertainty from pairwise evidence to action decisions. CPOR-Grasp calibrates and fuses VLM, depth, and amodal-mask cues to estimate obstruction probabilities, induces a distribution over valid obstruction graphs, and marginalizes over these graphs to compute the likelihood that the target is accessible or that a given blocker should be removed. To make inference tractable, it retains only the highest-probability graphs and derives a total-variation bound on the discarded probability mass, enabling certified decisions, adaptive stopping, and principled deferral. On synthetic and real UNOBench scenes, CPOR-Grasp outperforms state-of-the-art baselines. Calibration error decreases from 0.1416 to 0.0185 on the Gemini Robotics backbone, while graph truncation matches exact inference on 99.74% of decisions using 56 times fewer graphs. In real-world experiments, CPOR-Grasp achieves a 77.8% average success rate, surpassing SOTA baselines.

Abstract

Explore similar work

Calibrated Probabilistic Obstruction Reasoning with Vision-Language Models for Grasping in Clutter

Learning Beyond What Humans Can Demonstrate

Whole-Body Aerial Grasping and Lifting via Partial Visual Observations