cs.CVMar 25, 2026
SaveLookalike3D: Seeing Double in 3D
Organizations: Technical University of Munich
Abstract
3D object understanding and generation methods produce impressive results, yet they often overlook a pervasive source of information in real-world scenes: repeated objects. We introduce the task of lookalike object detection in indoor scenes, which leverages repeated and complementary cues from identical and near-identical object pairs. Given an input scene, the task is to classify pairs of objects as identical, similar or different using multiview images as input. To address this, we present Lookalike3D, a multiview image transformer that effectively distinguishes such object pairs by harnessing strong semantic priors from large image foundation models. To support this task, we collected the 3DTwins dataset, containing 76k manually annotated identical, similar and different pairs of objects based on ScanNet++, and show an improvement of 104% IoU over baselines. We demonstrate how our method improves downstream tasks such as enabling joint 3D object reconstruction and part co-segmentation, turning repeated and lookalike objects into a powerful cue for consistent, high-quality 3D perception. Our code, dataset and models will be made publicly available.
Figures & tables
Figure 2: Overview of the Lookalike3D model. Lookalike3D takes as input a single image or multiple posed images each of a pair of objects, passes these through a shared frozen DINOv2 backbone to extract image patch features, and then encodes these patch features with three kinds of attention layers: single-view, multiview and global. The resulting patch features are aggregated and compared to obtain a single similarity measure of the two objects. The model is trained with a triplet loss to separate the identical, similar and different pairs, and an alignment loss to align the similarities to the classification thresholds.
| Category | Train | Val | Overall |
|---|---|---|---|
| Identical Pairs | 32,270 | 2,496 | 34,766 |
| Similar Pairs | 5,468 | 735 | 6,203 |
| Different Pairs | 33,320 | 2,450 | 35,770 |
| Total Pairs | 71,058 | 5,681 | 76,739 |
| Unique Object Instances | 22,285 | 1,563 | 23,848 |
| Object Classes | 867 | 148 | 1015 |
Table 1: Dataset statistics for identical, similar, and different object pairs, and number of unique object instances and classes in the 3DTwins dataset.
Figure 3: Examples of identical, similar and different pairs of objects from the 3DTwins dataset. Identical objects have the exact same shape and appearance, differing only in scene context and occlusions. Similar objects differ slightly in shape or appearance, and different objects have large differences in shape or appearance.
| GT Instances | Pred. Instances | ||||||||
| Method | Id. | Sim. | Diff. | Overall | Id. | Sim. | Diff. | Overall | |
| 3D Methods | Lepard [ 14 ] | 0.06 | 0.13 | 0.47 | 0.22 | 0.05 | 0.10 | 0.24 | 0.13 |
| Predator [ 9 ] | 0.52 | 0.06 | 0.08 | 0.22 | 0.42 | 0.06 | 0.03 | 0.17 | |
| 2D Methods | DINOv2-feats [ 23 ] | 0.02 | 0.14 | 0.37 | 0.18 | 0.03 | 0.15 | 0.37 | 0.18 |
| SuperGlue [ 27 ] | 0.03 | 0.07 | 0.44 | 0.18 | 0.11 | 0.02 | 0.09 | 0.07 | |
| ASpanformer [ 5 ] | 0.06 | 0.06 | 0.44 | 0.18 | 0.10 | 0.00 | 0.10 | 0.07 | |
Table 2: Quantitative comparison on the 3DTwins dataset. We compare Lookalike3D against state-of-the-art baselines using IoU (%) for Identical (Id.), Similar (Sim.), and Different (Diff.) pairs. We report results on both Ground Truth (GT) instances and Predicted (MaskClustering) to evaluate robustness to the input instance segmentation. The best results are bolded and the second best are underlined . Lookalike3D outperforms all baselines by a significant margin.
Figure 4: Identical object groups identified by Lookalike3D and baselines. Each group is indicated by a unique color which is randomized across scenes and methods. Correct and incorrect predictions are circled in green and red , respectively. Unlike baselines that frequently misclassify or over-group instances, Lookalike3D leverages multi-view images and fine-grained appearance for accurate object grouping.
Figure 5: Joint 3D reconstruction using SAM 3D on Lookalike3D outputs. While individual reconstructions are plausible, they vary significantly in shape and size, often omitting parts due to limited context. In contrast, joint prediction leverages redundant and complementary cues across instances to produce a single, consistent reconstruction. Insets show objects from their original viewpoints.
Figure 6: Part segmentation transfer across similar objects. P3-SAM predicts high-quality part segmentation on objects with distinct parts, but undersegments objects with ambiguous geometry such as fused chair arms and legs. We make use of similar pairs predicted by Lookalike3D, and transfer the part segmentation from the correctly segmented source object to the undersegmented target object to get more detailed and consistent segmentation.
| Model/Data | Id. | Sim. | Diff. | Overall |
|---|---|---|---|---|
| Triplet InfoNCE loss | 0.36 | 0.14 | 0.56 | 0.35 |
| Without alignment loss | 0.63 | 0.14 | 0.65 | 0.47 |
| Without DINO aggr. feat. | 0.62 | 0.09 | 0.58 | 0.43 |
| Only single-view attn. | 0.62 | 0.08 | 0.60 | 0.43 |
| Only single-view & object-level attn. | 0.62 | 0.07 | 0.63 | 0.44 |
| Only single-view & global attn. | 0.62 | 0.12 | 0.61 | 0.45 |
Table 3: Ablation study. Performance is reported using IoU on pair type (identical, similar, different, and overall). Each ablation is compared independently to the full model, which uses triplet and alignment losses, aggregated DINO features, single-view, multiview and global attention, and 5 input views
| GT Instances | Pred. Instances | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Method | Id. | Sim. | Diff. | Overall | Id. | Sim. | Diff. | Overall | |
| 2D Methods | SuperGlue [ 27 ] | 0.05 | 0.07 | 0.44 | 0.19 | 0.02 | 0.05 | 0.24 | 0.10 |
| ASpanformer [ 5 ] | 0.07 | 0.06 | 0.43 | 0.19 | 0.04 | 0.03 | 0.24 | 0.10 | |
| Mast3r [ 11 ] | 0.16 | 0.07 | 0.45 | 0.23 | 0.10 | 0.06 | 0.24 | 0.13 | |
| TLFR [ 41 ] | 0.25 | 0.11 | 0.33 | 0.23 | 0.18 | 0.11 | 0.17 | 0.15 | |
| Ours | Lookalike3D | 0.65 | 0.18 | 0.65 | 0.49 | 0.50 | 0.15 | 0.29 | 0.31 |
Table 4: Lookalike3D compared with 2D matching baselines evaluated on multiview images using IoU (%) for Identical (Id.), Similar (Sim.), and Different (Diff.) pairs. We report results on both Ground Truth (GT) instances and Predicted instances to evaluate robustness. The best results are bolded and the second best are underlined . The baselines improve upon their single view versions in terms of identical IoU, but degrade in overall IoU as matches are inconsistent across views.
Figure 7: Web interface of our annotation tool. The annotator has access to multiple views of each object as well as their full views for scene context. Here we show the top 3 views of each object by coverage.
Figure 8: 3D view of object meshes during annotation. Note that 3D information is only available during annotation and not during model training.