Organizations: EPFL, Lausanne, Switzerland. · MBZUAI, Abu Dhabi, United Arab Emirates. · Southern University of Science and Technology, Shenzhen, China. · Stony Brook University, Stony Brook, New York, USA. · University of North Carolina, Charlotte, North Carolina, USA.
Visual object counting is a fundamental computer vision task in industrial inspection, where accurate, high-throughput inventory tracking and quality assurance are critical. Moreover, manufactured parts are often too light to reliably deduce their count from their weight, or too heavy to move the stack on a scale safely and practically, making automated visual counting the more robust solution in many scenarios. However, existing methods struggle with stacked 3D items in containers, pallets, or bins, where most objects are heavily occluded and only a few are directly visible. To address this important yet underexplored challenge, we propose a novel 3D counting approach that decomposes the task into two complementary subproblems: estimating the 3D geometry of the stack and its occupancy ratio from multi-view images. By combining geometric reconstruction with deep learning-based depth analysis, our method can accurately count identical manufactured parts inside containers, even when they are irregularly stacked and partially hidden. We validate our 3D counting pipeline on large-scale synthetic and diverse real-world data with manually verified total counts, demonstrating robust performance under realistic inspection conditions.
Figures & tables
Figure 1: 3D Counting (3DC). We estimate both the total volume occupied by the stack and the fraction of this volume taken up by the objects from multiple views of objects to be counted. Combining these estimates yields the total number of objects.
Figure 2: 3DC pipeline. We decompose the counting task into estimating the volume of the objects to be counted and then estimating the occupancy ratio within that volume. The first is done on the basis of geometry reconstructed from segmentations in multiple images.The second uses as input a depth-map computed by a monocular depth estimator and regresses an occupancy ratio from it.
Figure 3: Shape-dependent volume occupancy. We show a slice of the stacked objects for varying shapes. Different shapes yield different configurations and the fraction of space occupied by the gaps between objects varies. This is reflected in the images, where deeper layers remain visible for low γ values.
Figure 4: Dataset samples. We visualize generated scenes in ascending order of occupancy ratio, with ground-truth depth maps.
Figure 5: Reducing the domain gap. Instead of estimating the occupancy ratio γ from synthetic (top) and real images (bottom) (a), we identify a key view (b) and train a network to predict γ from their depth maps (c), which are indistinguishable. Top row: synthetic, γgt=62.4% . Bottom row: chocolates , γest=53.5% , Nest=119 , Ngt=131 .
Figure 6: Multi-view images. We generate 30 views from arbitrary angles for each of the scenes in our large-scale synthetic dataset.
NAE ↓
SRE ↓
MAE ↓
sMAPE ↓
BMNet+ Shi et al. (2022)
0.91
0.87
320.50
158.87
SAM+CLIP Kirillov et al. (2023) ; Radford et al. (2021)
0.73
0.61
259.22
102.77
CNN
0.66
0.48
235.74
98.44
ViT+H
0.42
0.24
149.90
47.36
Ours
0.22
0.09
79.48
27.65
Table 1: Counting evaluation on 100 synthetic scenes.
Figure 7: Additional qualitative results.
Figure 9: Intermediate results. From images (a), we find the key viewpoint and compute its depth (b) to estimate the occupancy ratio γ . Using the unit shape previously reconstructed from images of a template (c), and the overall reconstruction (d), we deduce the final count. ( Pasta : γest=30.5% , Nest=509 , Ngt=588 . Bricks : γest=31.8% , Nest=73 , Ngt=100 . Crosses : γest=29.6% , Nest=88 , Ngt=116 )
MAE ↓
RMSE ↓
sMAPE ↓
Convex hull
0.1847
0.2121
23.58
α -concave hull
0.4134
0.6724
54.07
Ours
0.0455
0.0690
9.33
Table 4: Evaluation of volume estimation.
NAE ↓
SRE ↓
MAE ↓
sMAPE ↓
( T− , D− )
0.22
0.09
79.48
27.65
( T+ , D− )
0.28
0.11
100.12
30.93
( T+ , D+ )
0.31
0.12
111.04
35.92
Table 5: Ablation study on 3D counting. If ground-truth depth maps are used during training, it is indicated as T+ , and T− otherwise. Similarly, for evaluation purposes if ground-truth depth-maps are used during validation, we indicate it as D+ .
Figure 10: Counting approximation. In an ideal setting where γ , v and V are know, comparing the estimated counts with Equation 4 and ground-truth values demonstrates that this equation is a good approximation of the real count of objects in our released dataset, with a slight tendency to overestimate at larger counts.
Figure 11: Border effects. Measuring the ground-truth γ over the complete box or over only a smaller section makes little difference, indicating that our assumption of uniform γ across the volume is justified.
Figure 12: Complexity Analysis. Each point represents the γ occupancy ratio error for a shape in the validation set.
Figure 13: Instance localization. Previous methods also produce interpretable results, representing a promising direction for future work.
Table 7: Counting visible and invisible objects separately.
Figure 15: Dataset statistics. The histograms represent the distributions of object count and occupancy ratio, respectively, and each bar plots the number of scenes in a given bin. In scatter plots, each point represents a physically simulated 3D scene. In particular, the occupancy ratio γ spans a large range between 1% and 65 %
Counting objects in an image is a task applicable across many domains. For instance, crowd counting, inventory counting, and cell counting have been the focus of recent research. The major challenges in estimating the count of objects include overlapping objects, object scale issues, occlusions, and varying lighting conditions. In this report, we explore the problem of counting machine washer parts. Our technique is an extension of FamNet with an additional loss component, trained on the given dataset. We compare to three baseline methods: a traditional image processing pipeline, instance segmentation, and density map estimation. We evaluate the performance of these algorithms by computing the Mean Absolute Error (MAE) and the Root Mean Squared Error (RMSE) between the true object counts and the model outputs. Our approach achieves a performance of 1.96 MAE.
Benedict Florance Arockiaraj, Elizabeth Dinella, Ankit Billa +1
3D object understanding and generation methods produce impressive results, yet they often overlook a pervasive source of information in real-world scenes: repeated objects. We introduce the task of lookalike object detection in indoor scenes, which leverages repeated and complementary cues from identical and near-identical object pairs. Given an input scene, the task is to classify pairs of objects as identical, similar or different using multiview images as input. To address this, we present Lookalike3D, a multiview image transformer that effectively distinguishes such object pairs by harnessing strong semantic priors from large image foundation models. To support this task, we collected the 3DTwins dataset, containing 76k manually annotated identical, similar and different pairs of objects based on ScanNet++, and show an improvement of 104% IoU over baselines. We demonstrate how our method improves downstream tasks such as enabling joint 3D object reconstruction and part co-segmentation, turning repeated and lookalike objects into a powerful cue for consistent, high-quality 3D perception. Our code, dataset and models will be made publicly available.
Our primary objective is to advance video object counting in crowded scenes, aiming to robustly count all instances of a target category based on given text or visual prompts. Existing methods rely on RGB information, limiting their discriminative ability in crowded and occluded conditions. To address this, we propose a Depth-Guided Detector (DG-Det) along with a general post-processing pipeline. By integrating depth cues with multi-scale RGB-D cross-attention and explicit occlusion prediction, our method enhances spatial understanding and achieves robust detection in crowded and occluded scenes. Furthermore, we introduce a unified de-duplication framework to eliminate cross-frame redundant counting. To facilitate future research, we also release a new RGB-D Video Object Counting dataset featuring depth information and multiple object categories persequence. Extensive experiments demonstrate that our method achieves a 62.01% reduction in MAE compared to existing baselines, and also produces consistent improvements in RMSE. We provide the source code at https://github.com/streamer-AP/DG-Net and the dataset at https://huggingface.co/datasets/aerospace123/RGBD-VideoCount.
Yuanjing Xu, Xinyan Liu, Weidong Chen +5
Harbin Institute of Technology (Weihai) Weihai, Shandong, China · City University of Hong Kong Hong Kong, China · University of Science and Technology of China Hefei, Anhui, China +1