ShelfChange3D: Object-Level 3D Change Detection for Retail Shelf Monitoring

Lingyi Zhou; Yunke Wang; Mengyu Zheng; Wenbo Wang; Zijian Wang; Chang Xu

cs.CVOct 1, 2026

ShelfChange3D: Object-Level 3D Change Detection for Retail Shelf Monitoring

Authors: Lingyi Zhou, Yunke Wang, Mengyu Zheng, Wenbo Wang, Zijian Wang, Chang Xu

Organizations: The University of Sydney · Beijing Jiaotong University · StellarEdge AI

Abstract

Reliable shelf monitoring is an important capability for retail automation, yet existing out-of-stock detection methods mainly operate in image space and lack metric 3D localization for downstream robotic systems. We formulate shelf monitoring as object-level 3D change detection: given two RGB-D observations captured at different times, the goal is to identify changed products and localize each change with a 3D bounding box. To support this task, we introduce ShelfChange3D, comprising 145K synthetic and 5K real-world paired RGB-D observations with object-level 3D change annotations. We further propose ChangeBox, an end-to-end framework that jointly reasons over paired observations and predicts object-level 3D change boxes. To improve localization accuracy, we introduce a geometry-based refinement stage that exploits depth and gravity prior to estimate relative pose and refine predicted boxes. Experiments show that ChangeBox outperforms existing change detection baselines, with further gains from refinement and effective transfer from synthetic to real-world observations.

Figures & tables

Fig. 1: An example of ShelfChange3D-145K, one per viewpoint-change type (rows). Red = removed, green = added.

Fig. 2: An example of a top-down view of one shelf layer. (a) The sampled layout: two SKU groups with their column/row grid cells. (b) The same layer after remove-add trajectory steps: hatched cells were vacated by removals, green rectangles are misplaced products.

Fig. 3: Camera configuration. (a) The nine nominal viewpoints on the viewing sphere around a layer anchor. (b) Azimuth/elevation of the viewpoints for one trajectory: hollow circles are nominal poses, dotted boxes the trajectory-level perturbation range, filled circles the realized trajectory-level pose, and small dots the per-frame revisit jitter over the states of the trajectory.

Fig. 4: Example of annotation pipeline for one edge. (a) Change map with difference candidates (yellow) and the one selected by the product detector (green). (b) The selected candidate and the point prompt derived from its centre (red star). (c) The SAM 3 mask. (d) The $t_{0}$ RGB-D point cloud with the SAM 3D Objects mesh placed in it and the box fitted to the mesh. (e) Translucent mesh and box over $t_{0}$ . (f) The 3D box over $t_{1}$ , where it marks the vacated slot.

Fig. 5: Overview of the proposed method. A shared image encoder extracts patch features from the paired RGB images, while the depths are divided into patches, with each patch represented by its median depth and concatenated to the corresponding RGB feature. The resulting RGB-D tokens are fed into a change-query decoder to predict coarse 3D change boxes, which are further refined by a local box refinement module using the relative extrinsics estimated by the gravity-prior extrinsics module. Orange boxes mark removed products, blue boxes added ones.

Method	mAP 0.25 $\uparrow$	mAP 0.5 $\uparrow$	mAP 0.7 $\uparrow$
ChangeBox	87.0	57.5	14.8
Size $\rightarrow$ GT	87.2	57.7	14.8
Center $\rightarrow$ GT	97.1	95.6	94.0
Both $\rightarrow$ GT	97.4	96.7	95.5
ChangeBox+Refine	93.0	81.7	48.3

TABLE I: Error analysis of ChangeBox and evaluation of the refinement on ShelfChange3D-145K.

Fig. 6: Qualitative results. Example results for ChangeBox on ShelfChange3D-145K.

Method	mAP $\uparrow$			mAR $\uparrow$			c-err (cm) $\downarrow$
Method	0.25	0.5	0.7	0.25	0.5	0.7	0.25	0.5
DenseChangeCap [ 29 ]	39.1	4.1	0.0	66.0	20.7	1.7	1.96	1.19
ChangeBox (R)	79.1	46.2	10.0	85.1	61.0	26.4	1.29	0.98
ChangeBox (R) + refine	88.6	74.2	36.0	92.3	81.5	53.2	0.89	0.73
ChangeBox (C)	85.9	57.3	16.3	89.5	69.2	33.3	1.14	0.91
ChangeBox (C) + refine	92.2	80.6	45.4	94.5	85.8	60.5	0.78	0.66

TABLE II: Comparison of ChangeBox variants with the baseline on ShelfChange3D-145K.

Method	Res.	mAP 0.25 $\uparrow$	mAP 0.5 $\uparrow$	mAR 0.25 $\uparrow$	c-err 0.25 $\downarrow$
ChangeBox (R)	320	61.9	23.9	72.1	1.67
ChangeBox (C)	320	75.0	37.3	82.5	1.44
ChangeBox (D)	320	73.6	35.8	82.0	1.44
ChangeBox (R)	640	79.1	46.2	85.1	1.29
ChangeBox (C)	640	85.9	57.3	89.5	1.14
ChangeBox (D)	640	87.0	57.5	90.5	1.14

TABLE III: Evaluation of different visual representation and resolution on ShelfChange3D-145K.

Method	Depth	mAP 0.25 $\uparrow$	mAP 0.5 $\uparrow$	mAR 0.25 $\uparrow$	c-err 0.25 $\downarrow$
ChangeBox (R)	✗	72.4	38.4	81.0	1.36
ChangeBox (R)	✓	79.1	46.2	85.1	1.29
ChangeBox (C)	✗	83.0	52.3	88.1	1.16
ChangeBox (C)	✓	85.9	57.3	89.5	1.14
ChangeBox (D)	✗	80.5	45.1	86.7	1.28
ChangeBox (D)	✓	87.0	57.5	90.5	1.14

TABLE IV: Evaluation of effect of the depth information on ShelfChange3D-145K.

Fig. 7: Accuracy and center error vs. relative camera rotation between

t_{0}

and

t_{1}

. The

<5^{\circ}

bin holds 61% of the test pairs.

Fig. 8: Extrinsics estimation on the test set. Left: solve rate per view type. Right: wrongly accepted per view type, both with and without the forward-backward check.

Training			Metrics
Syn. pre-train	Real	Refine	mAP 0.25 $\uparrow$	mAP 0.5 $\uparrow$	mAR 0.25 $\uparrow$	c-err 0.25 $\downarrow$
✗	✓	✗	21.0	0.8	44.2	3.34
✓	✗	✗	46.7	2.2	65.3	2.87
✓	✓	✗	92.9	37.5	95.6	2.20
✓	✓	✓	97.9	50.1	98.5	1.95

TABLE V: Evaluation of Sim2Real transfer on ShelfChange3D-5K.

Fig. 9: Restocking Robot. For each demonstration, the top row shows the head-camera views: the fully stocked shelf at $t_{0}$ , the shelf at $t_{1}$ after products were removed, the predicted boxes of the removed products, and the shelf after restocking. Subsequent panels show robots demonstration.

Method	Schedule	mAP 0.25 $\uparrow$	mAP 0.5 $\uparrow$	mAR 0.25 $\uparrow$	c-err 0.25 $\downarrow$
ChangeBox (R)	FT	66.5	28.5	77.4	1.56
ChangeBox (R)	frozen	64.1	24.7	74.3	1.66
ChangeBox (R)	frozen $\to$ FT	79.1	46.2	85.1	1.29
ChangeBox (C)	FT $\dagger$	60.4	17.8	74.2	1.86
ChangeBox (C)	frozen	72.9	33.7	80.8	1.51
ChangeBox (C)	frozen $\to$ FT	85.9	57.3	89.5	1.14

TABLE VI: Evaluation of different training schedules on the ShelfChange3D-145K.

Fig. 10: Qualitative results. Example results for ChangeBox on ShelfChange3D-5K.

Fig. 11: One remove-add trajectory seen from the front camera. Dashed red boxes mark the products removed at that step at their previous position, solid green boxes mark the misplaced products added at that step.

Dataset	Domain	Task	Scale	Annotation
Change detection in image space
VL-CMU-CD [ 18 ]	street	segmentation	1,362 pairs	2D mask
ChangeSim [ 27 ]	industrial	segmentation	130K images	2D mask
PSCD [ 19 ]	street	segmentation	770 pairs	2D mask
3D and multiview scene change detection
3RScan [ 28 ]	indoor rooms	re-localization	1,482 scans	instance pose

TABLE VII: Comparison with existing change detection datasets and retail datasets.

Fig. 13: Examples from the ShelfChange3D-5K test split, $t_{0}$ on the left and $t_{1}$ on the right.

Explore similar work

AnyBox: Efficient Zero-Shot 9DoF Pose Estimation of Boxes for Robotic Manipulation
Nov 19, 2025Yintao Ma, Sajjad Pakdamansavoji, Charles Eret +5Category-Level Object Pose Estimation Bounding Box
1Huawei Technologies Canada · University of Waterloo
RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection
Aug 10, 2026Zhihao Zhang, Gengwei Zhang, Tianlong Chen +13D Object Detection Monocular
Michigan State University · University of North Carolina at Chapel Hill
VGGT-CD: Training-Free Robust Registration for 3D Change Detection
May 16, 2026Wei Zhang, Songhua Li, Yihang Wu +2Image Registration Change Detection
Northwestern Polytechnical University, Xi'an, China · Northwestern Polytechnical University Xian, 710072, China

Nov 19, 2025, cs.CVJ/K move · Enter open · S save

AnyBox: Efficient Zero-Shot 9DoF Pose Estimation of Boxes for Robotic Manipulation

Yintao Ma, Sajjad Pakdamansavoji, Charles Eret, Rui Heng Yang+4

1Huawei Technologies Canada · University of Waterloo

Category-Level Object Pose Estimation Bounding Box Robotic Perception Rgb-Depth Cameras Robotic Manipulation Zero-Shot+1

Open paper AskPDF arXiv

Recovering the 9D pose of objects, both their 6D pose and 3D dimensions, under clutter and occlusion is a core requirement for warehouse automation, logistics, and manufacturing. Model-based methods are accurate but assume an instance-specific CAD model for every object, which is costly to maintain as inventories change. Model-free and category-level methods relax this assumption, yet they remain vulnerable to the symmetry, weak texture, and heavy occlusion that characterize stacked storage boxes, and they ignore the strong structural priors such scenes provide. We present \textbf{AnyBox}, an efficient zero-shot framework that exploits the geometric regularity of boxes to jointly recover pose and dimensions from a single RGB-D observation. Starting from a canonical category template, AnyBox alternates between pose and scale estimation, using the discrepancy between the reprojected template and the observed mask to drive a binary search over box dimensions. Two lightweight components make this practical: a depth-consistency filter that rejects the implausible hypotheses induced by box symmetry, and an early-stopping rule that replaces the remaining search with a single closed-form update. On public benchmarks and an in-house warehouse dataset, AnyBox improves detection AP by up to 36 points, more than doubling the previous best, and approaches instance-level pipelines that have access to ground-truth CAD models. These gains transfer downstream, raising success by 28% on a cluttered robotic box-shelving task.

Abstract

Explore similar work

AnyBox: Efficient Zero-Shot 9DoF Pose Estimation of Boxes for Robotic Manipulation

RefineAny3D: Depth Refinement as Semantic Alignment for Monocular 3D Detection

VGGT-CD: Training-Free Robust Registration for 3D Change Detection