Mind the Shape Gap: A Benchmark and Baseline for Deformation-Aware 6D Pose Estimation of Agricultural Produce

Nikolas Chatzis; Angeliki Tsinouka; Katerina Papadimitriou; Niki Efthymiou; Marios Glytsos; George Retsinas; Paris Oikonomou; Gerasimos Potamianos

cs.CVMar 28, 2026

Mind the Shape Gap: A Benchmark and Baseline for Deformation-Aware 6D Pose Estimation of Agricultural Produce

Authors: Nikolas Chatzis, Angeliki Tsinouka, Katerina Papadimitriou, Niki Efthymiou, Marios Glytsos, George Retsinas, Paris Oikonomou, Gerasimos Potamianos,

Organizations: Robotics Institute, Athena Research Center, Marousi, Greece · HERON - Hellenic Robotics Center of Excellence, Athens, Greece · School of Electrical & Computer Engineering, NTUA, Greece · Department of Electrical & Computer Engineering, UTH, Greece ·

Abstract

Accurate 6D pose estimation for robotic harvesting is fundamentally hindered by the biological deformability and high intra-class shape variability of agricultural produce. Instance-level methods fail in this setting, as obtaining exact 3D models for every unique piece of produce is practically infeasible, while category-level approaches that rely on a fixed template suffer significant accuracy degradation when the prior deviates from the true instance geometry. To bridge such lack of robustness to deformation, we introduce PEAR (Pose and dEformation of Agricultural pRoduce), the first benchmark providing joint 6D pose and per-instance 3D deformation ground truth across 8 produce categories, acquired via a robotic manipulator for high annotation accuracy. Using PEAR, we show that state-of-the-art methods suffer up to 6x performance degradation when faced with the inherent geometric deviations of real-world produce. Motivated by this finding, we propose SEED (Simultaneous Estimation of posE and Deformation), a unified RGB-only framework that jointly predicts 6D pose and explicit lattice deformations from a single image across multiple produce categories. Trained entirely on synthetic data with generative texture augmentation applied at the UV level, SEED outperforms MegaPose on 6 out of 8 categories under identical RGB-only conditions, demonstrating that explicit shape modeling is a critical step toward reliable pose estimation in agricultural robotics.

Figures & tables

Fig. 1 : Sample annotated frames from the PEAR dataset.

Fig. 2 : Effect of template shape mismatch on pose estimation. Each row shows the results of foundation models for a selected frame of PEAR dataset. From left to right: FoundationPose predictions using the ground-truth mesh, predictions using a category-level base mesh, and a reference view for comparison; the same three visualizations are shown for MegaPose in the last three columns. The results highlight the sensitivity of pose estimators to geometric discrepancies between the template mesh and the true object shape.

Panel of Figure 3 — Fig. 3 : PEAR benchmark acquisition framework. (a) An Orbbec Femto Bolt RGB-D camera is rigidly mounted to a Franka Panda manipulator, observing the produce from a fixed distance. (b) To ensure comprehensive coverage, viewpoints are uniformly sampled across three concentric spheres along an $80^{\circ}$ azimuth arc facing the robot, always oriented directly toward the scene center.

Fig. 4 : Left : input RGB images. Middle : initial pose predictions from FoundationPose given the depth image and object mask. Right : pose estimates after our joint multi-view optimization. The first row shows the object in an unoccluded setting, while the second row shows the same object placement under occlusion.

Fig. 5 : Overview of the SEED model. A cropped RGB detection is passed through a DINOv2 backbone with LoRA adaptation, producing features that feed three prediction heads for rotation, translation, and lattice deformation. The predicted deformation warps the category base mesh to match the observed instance geometry.

		Banana			Pumpkin			Pear			Apple			Lemon			L. Pepper			Carrot			Avocado			Average
Method	Mesh	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.
FP	GT	36.1	17.6	17.6	40.1	18.3	17.9	33.5	17.7	17.6	42.9	20.9	20.5	43.3	22.5	22.2	44.3	24.4	24.7	35.5	22.6	22.9	39.3	19.6	20.0	39.4	20.4	20.4
FP	Base	69.0	26.1	26.6	155.5	30.4	43.9	65.6	29.0	27.5	69.7	24.5	22.0	58.2	23.7	23.2	88.2	49.2	51.1	46.4	24.2	26.1	68.7	29.9	30.3	77.7	29.6	31.3
MP	GT	42.0	24.2	23.3	100.0	11.7	11.3	13.5	5.3	5.0	20.6	3.6	3.4	14.1	2.7	2.5	14.8	7.8	7.7	37.2	24.3	23.8	26.6	8.2	7.9	33.6	11.0	10.6
MP	Base	97.3	62.4	61.3	223.0	93.9	104.3	110.8	72.9	72.7	72.6	29.4	28.3	92.1	59.0	57.9	109.7	74.3	77.3	107.1	85.3	85.7	91.7	55.6	53.3	113.0	66.6	67.6

TABLE I : Effect of shape deformation on pose estimation accuracy (PEAR benchmark). FP = FoundationPose [ 6 ] , MP = MegaPose [ 4 ] . We report ADD, ADD-S, and Chamfer distance (Ch.) in mm (lower is better). For each method we compare performance using the ground-truth scanned mesh (GT) versus the category-level base mesh (Base).

	Banana			Pumpkin			Pear			Apple			Lemon			L. Pepper			Carrot			Avocado			Average
Method	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.	ADD	ADD-S	Ch.
MP-Base [ 4 ]	97.3	62.4	61.3	223.0	93.9	104.3	110.8	72.9	72.7	72.6	29.4	28.3	92.1	59.0	57.9	109.7	74.3	77.3	107.1	85.3	85.7	91.7	55.6	53.3	113.0	66.6	67.6
SEED (Ours)	67.4	41.6	42.4	216.2	91.8	102.1	124.5	89.7	92.2	62.0	23.6	22.9	100.6	68.7	68.9	84.8	56.2	58.6	78.3	60.1	60.6	54.7	25.3	25.1	98.6	57.1	59.1

TABLE II : SEED vs. MegaPose (Base mesh) on the PEAR benchmark — both methods operate in a purely RGB setting with no per-instance geometry. We report ADD, ADD-S, and Chamfer distance (Ch.) in mm (lower is better). Best results in bold .

Fig. 6 : Qualitative results on the PEAR benchmark. From left to right: input RGB image, MegaPose [ 4 ] prediction overlaid with the category base mesh, SEED prediction overlaid with the estimated deformed mesh, and 3D meshes comparison from another viewpoint (green: ground truth, red: MegaPose, blue: ours). SEED produces closer alignment to the ground-truth geometry across categories.

Object	Training	ADD $\downarrow$	ADD-S $\downarrow$	Ch. $\downarrow$
Banana	Single-model	74.5	48.3	49.2
Banana	Multi-model	67.4	41.6	42.4
Apple	Single-model	74.0	33.4	31.8
Apple	Multi-model	62.0	23.6	22.9
Avocado	Single-model	60.8	28.8	29.6
Avocado	Multi-model	54.7	25.3	25.1

TABLE III : Ablation study comparing single- and multi-category training. Multi-category training achieves the lowest error (mm) across all evaluated agrifoods.

Object Pose Estimation

Object pose estimation recovers an object's 3D position and orientation from images, depth, point clouds, or contact sensing.

See all 187 papers

From Simulation to the Real-World: An In-Field 6D Pose Dataset and Baseline for Robotic Strawberry Harvesting
Jun 9, 2026Woojung Son, Won Suk Lee, Zijing Huang +4Sim-to-Real Transfer Object Pose Estimation
Department of Agricultural and Biological Engineering, University of Florida · Department of Electrical and Computer Engineering, University of Florida · Edwardson School of Industrial Engineering, Purdue University +1
Exploring 6D Object Pose Estimation with Deformation
Apr 8, 2026Zhiqiang Liu, Rui Song, Duanmu Chuangqi +3Object Pose Estimation
State Key Laboratory of ISN, Xidian University · MagicLeap
Flow6D: Discrete-to-Continuous Flow Matching for Efficient and Accurate Category-Level 6D Pose Estimation
Jun 22, 2026Mingyu Mei, Li Zhang, Zibo Dai +4Flow Matching Object Pose Estimation
Zhejiang University · University of Science and Technology of China · Shanghai Jiao Tong University

Jun 9, 2026J/K move · Enter open · S save

From Simulation to the Real-World: An In-Field 6D Pose Dataset and Baseline for Robotic Strawberry Harvesting

Woojung Son, Won Suk Lee, Zijing Huang, Daeun Choi+3

Department of Agricultural and Biological Engineering, University of Florida · Department of Electrical and Computer Engineering, University of Florida · Edwardson School of Industrial Engineering, Purdue University+1

Sim-to-Real Transfer Object Pose Estimation Synthetic Data Generation for Robotics Robot Manipulation Benchmarks Agricultural Robotics

Open paper AskPDF arXiv

Robotic strawberry harvesting requires precise 6D pose estimation; however, collecting 6D pose ground truth in real agricultural fields is inherently challenging. Existing strawberry 6D pose estimation studies have therefore relied mainly on synthetic data, leaving their in-field performance unquantified. In this work, we obtain ground truth indirectly, by recovering camera poses via PnP, reconstructing each scene at metric scale, and annotating a single 3D bounding box per strawberry that is propagated across all frames, yielding, to the best of our knowledge, the first real-world 6D pose ground-truth dataset of red-stage strawberries collected at an actual strawberry farm (12,040 images). We also introduce a synthetic dataset rendered in NVIDIA Isaac Sim, featuring scene-level realism and domain randomization. Despite this improved simulation setup, models trained on synthetic data alone fail to transfer to in-field images, while introducing a small amount of real data improves both translation and rotation accuracy across all backbones. Under the monocular RGB setting evaluated here, rotation is largely recovered once real data is used, and depth is what limits pose accuracy. These baselines across backbone encoders serve as a reference for future work. The real-world dataset is publicly available at https://huggingface.co/datasets/WoojungSon/FieldStraw6D, and the data-collection pipeline is available at https://github.com/wjson2435/FieldStraw6D-pipeline.

Abstract

Object Pose Estimation

Explore similar work

From Simulation to the Real-World: An In-Field 6D Pose Dataset and Baseline for Robotic Strawberry Harvesting

Exploring 6D Object Pose Estimation with Deformation

Flow6D: Discrete-to-Continuous Flow Matching for Efficient and Accurate Category-Level 6D Pose Estimation