Gripper-Aware Automatic Dense Packing of Irregular Objects

Tianhao Qin; Connor McCann; Berk Calli; Jing Xiao

cs.ROSep 18, 2026

Gripper-Aware Automatic Dense Packing of Irregular Objects

Authors: Tianhao Qin, Connor McCann, Berk Calli, Jing Xiao

Organizations: Robotics Engineering Department Worcester Polytechnic Institute, Worcester, MA 01609, USA

Abstract

Automatic dense packing is widely desired in warehouse operations but remains a fundamental challenge in robotic manipulation. Existing work on irregular-object packing largely targets simulation with idealized contact, treating the object as an isolated rigid body. The gripper often enters as a discrete, post-hoc feasibility check, if considered at all, and the perception and contact drift accumulated during execution are not addressed. We present a closed-loop pipeline that integrates perception, gripper-aware placement optimization, and force-guided execution on a real manipulator. The optimizer represents the object together with the gripper as a single composite body of hierarchical sphere trees. It searches over five degrees of freedom on a GPU within a CMA-ES framework, with the vertical coordinate grounded analytically against the current heightmap. During execution, a force-monitored vertical descent stops on first contact. A post-release consolidation push then closes the residual lateral clearance that gripper-aware planning leaves behind. The container is re-perceived between placements so that drift does not accumulate. We validate the system on a Franka Emika Panda robot packing a 3D-printed set of flat, curved, and concave objects, and a YCB object subset. An ablation study isolates the contribution of gripper-aware optimization, the consolidation push, and mesh-derived geometry to end-to-end success, achieved density, and computational cost. We further benchmark against the heightmap-minimization method as a baseline representative of prior irregular-object packing work.

Figures & tables

Figure 1: Dense packing pipeline. Perception detects the next target object and scans the container, Planning composes the object with its grasp and searches over placements with CMA-ES, and Execution picks, descends with force/torque monitoring, and consolidates eligible placements before looping back.

Term	Vol.	Cent.	Pack.	Height	Front. †	Coll.	Bounds	Orient.
Symbol	$\hat{f}_{\mathrm{vol}}$	$\hat{f}_{\mathrm{cen}}$	$\hat{f}_{\mathrm{pck}}$	$\hat{f}_{\mathrm{hgt}}$	$\hat{f}_{\mathrm{fro}}$	$\hat{f}_{\mathrm{col}}$	$\hat{f}_{\mathrm{bnd}}$	$\hat{f}_{\mathrm{ori}}$
Weight	2.0	2.0	2.0	10 / 5	10	50	50	50 / 4
Point set	O	O	O	O	O	O+G	O+G	G

Table 1: Optimization configuration: objective function weights with point-set assignments, and CMA-ES parameters.

Panel of Figure 2 — Figure 2: Hardware platform and object set. $\mathbf{Left}$ : (a) Intel RealSense D415 and D455 depth cameras mounted on opposite sides of the workspace, (b) 7-DoF Franka Emika Panda Haddadin et al. (2022) , (c) ATI wrist force/torque sensor, (d) 3D-printed soft parallel fin-ray gripper, (e) Container, (f) Staging area with the seven-object set. $\mathbf{Right}$ : the eight-object YCB subset used to test generalization.

Sequence	Object ordering
Optimized	support, duck, duck, vase, vase, L, L
Random 1	vase, L, vase, L, duck, duck, support
Random 2	duck, vase, L, support, L, vase, duck
YCB	mug, tomato soup can, banana, potted meat can, pudding box, gelatin box, pear, orange

Table 2: Test sequences (five trials each).

ID	Ablations	Hypothesis tested
A0	Full system	Reference for paired comparison.
A1	Prism-like geometry	Planform-extrusion proxy Cao and Xiao (2026) cannot pack as dense as using the irregular shapes directly.
A2	No consolidation push	Post-release push contributes measurably to final density.
A3	No gripper-aware optimization	Omitting the gripper from the composite body causes collision.

Table 3: Ablation variants, run on the Optimized sequence.

Metric	Optimized	Random 1	Random 2	Overall
Space utilization (%)	$\mathbf{95.59\boldsymbol{\pm}6.06}$	$81.37\pm 0.02$	$75.19\pm 5.66$	$84.05\pm 9.89$
Objects packed	$\mathbf{6.60\boldsymbol{\pm}0.55}$	$6.00\pm 0.00$	$5.40\pm 0.55$	$6.00\pm 0.65$
End-to-end success	$\mathbf{5/5}$	$4/5$	$\mathbf{5/5}$	$14/15$
Per-placement success	$\mathbf{35/35}$	$34/35$	$\mathbf{35/35}$	$104/105$
CMA wall time (s/obj)	$3.56\pm 0.20$	$3.32\pm 0.21$	$\mathbf{3.19\boldsymbol{\pm}0.11}$	$3.34\pm 0.22$

Table 4: Average system performance of three test sequences (five trials each).

Figure 3: Three final packing results (out of five) for each sequence. The Optimized sequence produces visibly flatter and tighter configurations, while the random orderings leave larger residual cavities as small objects are committed early.

	Space utilization (%)	Objects packed	CMA wall time (s/obj)
YCB subset	$82.45\pm 10.36$	$6.60\pm 0.89$	$3.00\pm 0.46$

Table 5: Average system performance on the YCB objects (five trials each).

Figure 4: Final packing results for all five trials of the YCB sequence.

Metric	A0 (Full)	A1 (Prism-like)	A2 (No-push)	A3 (No-gripper)
Space utilization (%)	$\mathbf{95.59\boldsymbol{\pm}6.06}$	$80.00\pm 9.50$	$70.62\pm 12.43$	$26.09\pm 21.23$
Objects packed	$\mathbf{6.60\boldsymbol{\pm}0.55}$	$5.20\pm 0.84$	$5.20\pm 0.84$	$1.40\pm 1.14$
End-to-end success	$\mathbf{5/5}$	$\mathbf{5/5}$	$\mathbf{5/5}$	$0/5$
Per-placement success	$\mathbf{35/35}$	$\mathbf{35/35}$	$\mathbf{35/35}$	$7/35$
CMA wall time (s/obj)	$3.56\pm 0.20$	$\mathbf{1.87\boldsymbol{\pm}0.12}$	$3.30\pm 0.28$	$2.69\pm 0.39$

Table 6: Paired ablation on the Optimized sequence.

Figure 5: Three final packing results (out of five) for each ablation variant on the Optimized sequence.

Figure 6: Comparison of space utilization and objects packed. (a) The left three bars correspond to the sequential packing tests. (b) The right three bars correspond to the ablation variants on the Optimized sequence. Bars show the mean over five trials per configuration with error bars indicating one standard deviation.

Metric	Optimized	HM_Low_Resolution	HM_High_Resolution
Space utilization (%)	$\mathbf{95.59\boldsymbol{\pm}6.06}$	$77.66\pm 8.33$	$82.35\pm 6.06$
Objects packed	$\mathbf{6.60\boldsymbol{\pm}0.55}$	$5.00\pm 0.71$	$5.40\pm 0.55$

Table 7: Full system (Optimized sequence) vs. Wang and Hauser Wang and Hauser (2022) heightmap-minimization (HM) baseline, with two search budgets: HM_Low_Resolution (10 mm XY step,

\Delta r=\pi/4

, their Sec. VI setting) and HM_High_Resolution (5 mm step, 16 yaws, a matched search budget). Five trials each.

Figure 7: Final packing results for the HM baseline on the Optimized sequence.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,704 papers

Apr 23, 2025

HERB: Human-augmented Efficient Reinforcement learning for Bin-packing

Packing objects efficiently is a fundamental problem in logistics, warehouse automation, and robotics. When dealing with highly diverse 3D objects (household or grocery items), closed-form solutions are infeasible, and heuristic or model-free Reinforcement Learning~(RL) methods tend to focus solely on geometric optimization, relying on exhaustive searches of the discretized solution space. This leads to long training times (for pure RL) and high latency (heuristics), limited transferability to robotic scenarios, and ultimately ignores object characteristics (fragility, deformability) and human preferences. We propose HERB, a human-augmented RL framework for packing irregular objects, the first to explore the potential of learning from human demonstrations to solve this complex task. It leverages human demonstrations of packing strategies, which inherently exhibit latent factors such as space optimization, stability, and object properties that are difficult to model explicitly. The human-expert data is combined with RL exploration to provide the placement of each object inside the container. Experimental results show that our method outperforms heuristic, purely RL-based, and imitation learning approaches in packing efficiency and latency. Qualitative results highlight that our packing strategy produces more stable, human-like arrangements, which we expect to be more appropriate and widely accepted. Finally, we demonstrate the real-world feasibility of our method on a robotic system.

Gojko Perovic, Nuno Ferreira Duarte, Atabak Dehban +3

BioRobotics Institute, Scuola Superiore Sant’Anna, Pisa, Italy, and with the Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, Pisa, Italy. · Institute for Systems and Robotics, Instituto Superior Técnico, Universidade de Lisboa, 1049-001 Lisbon, Portugal.

Robotic RL Robotics Robotic Manipulation Learning from Demonstration

Ask PDF

Jun 15, 2026

Differentiable Packing of Irregular 3D Objects with Adaptive Container Estimation

Most existing approaches either fix the container in advance or optimize only a single container dimension through an outer search loop, leaving the remaining dimensions as a manual tuning problem. We present a differentiable packing framework that jointly optimizes all 6N object pose parameters and all three container side lengths inside a single gradient-based loop. The formulation combines six physics-inspired, differentiable loss terms computed directly on triangle meshes through axis-aligned bounding-box proxies. An adaptive squeezing mechanism periodically tightens the container whenever the overlap loss falls below a pair-count-scaled threshold, producing a large initial drop in container volume, followed by small refinements. All pairwise computations are written in tensor-broadcasting form, giving a 3.4 to 54 times speedup over a reference loop-based implementation. The pipeline is implemented in Python and PyTorch, with no physics engine, FFT library, or convex decomposition. On multiple object categories, the method produces containers that are 11 to 32 percent smaller than time-matched DBLF and simulated-annealing baselines at N =100, while running in under 4 minutes per instance on a single consumer GPU.

Palak Gupta, Shanmuganathan Raman

Indian Institute of Technology Gandhinagar, India

3D Spatial Reasoning Differentiable Optimization

Ask PDF

Aug 25, 2026

Sequential Object Placement Optimization with Convex Decomposition

Robotic object packing has been a core challenge for robotic deployment in logistics, industry, etc., due to the curse of dimensionality in combinatorial search and the difficulty of dealing with dynamic collision constraints for irregularly shaped objects. Current heuristic and learning-based methods mainly assume a limited spatial discretization resolution of space, and computation becomes extremely inefficient as discretization accuracy increases. In this work, we eliminate this assumption by introducing SOPO-CD, which frames sequential object placement as a differentiable nonlinear optimization problem with hard constraints in a decomposed free space. We formulate the constraints of placing a convex object inside a convex hull as constraining the vertices of the object to lie inside the convex hull. The constraints and their derivatives can be written in closed form and calculated efficiently. We implement a custom solver that achieves local-optimal placements within tightly constrained space in milliseconds; a

50 \times

speedup compared to a fine-grained grid search method. We evaluate our framework on 2D Tangram, 2D Tetris, and 3D Bin Packing, and have demonstrated strong computational performance and packing utility. We also demonstrate its real-world applicability for solving the Tangram puzzle using a robot equipped with a dexterous hand.

Yuezhe Zhang, Xiangyu Lyu, Sohan Rudra +2

Interactive Robot Perception & Learning, Technical Univsersity of Darmstadt, Germany · Intelligent Autonomous Systems, Technical Univsersity of Darmstadt, Germany · Robotics and Semantic Systems, Lund University, Sweden +1

Combinatorial Optimization Robotic Manipulation

Ask PDF

Abstract

Robotic Manipulation

Explore similar work

HERB: Human-augmented Efficient Reinforcement learning for Bin-packing

Differentiable Packing of Irregular 3D Objects with Adaptive Container Estimation

Sequential Object Placement Optimization with Convex Decomposition