3D object understanding and generation methods produce impressive results, yet they often overlook a pervasive source of information in real-world scenes: repeated objects. We introduce the task of lookalike object detection in indoor scenes, which leverages repeated and complementary cues from identical and near-identical object pairs. Given an input scene, the task is to classify pairs of objects as identical, similar or different using multiview images as input. To address this, we present Lookalike3D, a multiview image transformer that effectively distinguishes such object pairs by harnessing strong semantic priors from large image foundation models. To support this task, we collected the 3DTwins dataset, containing 76k manually annotated identical, similar and different pairs of objects based on ScanNet++, and show an improvement of 104% IoU over baselines. We demonstrate how our method improves downstream tasks such as enabling joint 3D object reconstruction and part co-segmentation, turning repeated and lookalike objects into a powerful cue for consistent, high-quality 3D perception. Our code, dataset and models will be made publicly available.
Figures & tables
Figure 2: Overview of the Lookalike3D model. Lookalike3D takes as input a single image or multiple posed images each of a pair of objects, passes these through a shared frozen DINOv2 backbone to extract image patch features, and then encodes these patch features with three kinds of attention layers: single-view, multiview and global. The resulting patch features are aggregated and compared to obtain a single similarity measure of the two objects. The model is trained with a triplet loss to separate the identical, similar and different pairs, and an alignment loss to align the similarities to the classification thresholds.
Category
Train
Val
Overall
Identical Pairs
32,270
2,496
34,766
Similar Pairs
5,468
735
6,203
Different Pairs
33,320
2,450
35,770
Total Pairs
71,058
5,681
76,739
Unique Object Instances
22,285
1,563
23,848
Object Classes
867
148
1015
Table 1: Dataset statistics for identical, similar, and different object pairs, and number of unique object instances and classes in the 3DTwins dataset.
Figure 3: Examples of identical, similar and different pairs of objects from the 3DTwins dataset. Identical objects have the exact same shape and appearance, differing only in scene context and occlusions. Similar objects differ slightly in shape or appearance, and different objects have large differences in shape or appearance.
GT Instances
Pred. Instances
Method
Id.
Sim.
Diff.
Overall
Id.
Sim.
Diff.
Overall
3D Methods
Lepard [ 14 ]
0.06
0.13
0.47
0.22
0.05
0.10
0.24
0.13
Predator [ 9 ]
0.52
0.06
0.08
0.22
0.42
0.06
0.03
0.17
2D Methods
DINOv2-feats [ 23 ]
0.02
0.14
0.37
0.18
0.03
0.15
0.37
0.18
SuperGlue [ 27 ]
0.03
0.07
0.44
0.18
0.11
0.02
0.09
0.07
ASpanformer [ 5 ]
0.06
0.06
0.44
0.18
0.10
0.00
0.10
0.07
Table 2: Quantitative comparison on the 3DTwins dataset. We compare Lookalike3D against state-of-the-art baselines using IoU (%) for Identical (Id.), Similar (Sim.), and Different (Diff.) pairs. We report results on both Ground Truth (GT) instances and Predicted (MaskClustering) to evaluate robustness to the input instance segmentation. The best results are bolded and the second best are underlined . Lookalike3D outperforms all baselines by a significant margin.
Figure 4: Identical object groups identified by Lookalike3D and baselines. Each group is indicated by a unique color which is randomized across scenes and methods. Correct and incorrect predictions are circled in green and red , respectively. Unlike baselines that frequently misclassify or over-group instances, Lookalike3D leverages multi-view images and fine-grained appearance for accurate object grouping.
Figure 5: Joint 3D reconstruction using SAM 3D on Lookalike3D outputs. While individual reconstructions are plausible, they vary significantly in shape and size, often omitting parts due to limited context. In contrast, joint prediction leverages redundant and complementary cues across instances to produce a single, consistent reconstruction. Insets show objects from their original viewpoints.
Figure 6: Part segmentation transfer across similar objects. P3-SAM predicts high-quality part segmentation on objects with distinct parts, but undersegments objects with ambiguous geometry such as fused chair arms and legs. We make use of similar pairs predicted by Lookalike3D, and transfer the part segmentation from the correctly segmented source object to the undersegmented target object to get more detailed and consistent segmentation.
Model/Data
Id.
Sim.
Diff.
Overall
Triplet → InfoNCE loss
0.36
0.14
0.56
0.35
Without alignment loss
0.63
0.14
0.65
0.47
Without DINO aggr. feat.
0.62
0.09
0.58
0.43
Only single-view attn.
0.62
0.08
0.60
0.43
Only single-view & object-level attn.
0.62
0.07
0.63
0.44
Only single-view & global attn.
0.62
0.12
0.61
0.45
Table 3: Ablation study. Performance is reported using IoU on pair type (identical, similar, different, and overall). Each ablation is compared independently to the full model, which uses triplet and alignment losses, aggregated DINO features, single-view, multiview and global attention, and 5 input views
GT Instances
Pred. Instances
Method
Id.
Sim.
Diff.
Overall
Id.
Sim.
Diff.
Overall
2D Methods
SuperGlue [ 27 ]
0.05
0.07
0.44
0.19
0.02
0.05
0.24
0.10
ASpanformer [ 5 ]
0.07
0.06
0.43
0.19
0.04
0.03
0.24
0.10
Mast3r [ 11 ]
0.16
0.07
0.45
0.23
0.10
0.06
0.24
0.13
TLFR [ 41 ]
0.25
0.11
0.33
0.23
0.18
0.11
0.17
0.15
Ours
Lookalike3D
0.65
0.18
0.65
0.49
0.50
0.15
0.29
0.31
Table 4: Lookalike3D compared with 2D matching baselines evaluated on multiview images using IoU (%) for Identical (Id.), Similar (Sim.), and Different (Diff.) pairs. We report results on both Ground Truth (GT) instances and Predicted instances to evaluate robustness. The best results are bolded and the second best are underlined . The baselines improve upon their single view versions in terms of identical IoU, but degrade in overall IoU as matches are inconsistent across views.
Figure 7: Web interface of our annotation tool. The annotator has access to multiple views of each object as well as their full views for scene context. Here we show the top 3 views of each object by coverage.
Figure 8: 3D view of object meshes during annotation. Note that 3D information is only available during annotation and not during model training.
We address a fundamental gap in 3D-LLMs: existing models focus on single-object/scene description, struggling with detailed, inter-object comparison. We propose a framework for detailed object-level reasoning across multiple objects with three components: (1) MO3D (Multi-Object in 3D), an instruction dataset requiring fine-grained multi-object comparison; (2) Multi-3DLLM, using a minimal Patch-Interaction Transformer (PIT) that models inter-/intra-object relationships while preserving local geometry; (3) Mini-apps, two application-driven benchmarks (Shape Mating, Change Captioning) that probe geometric understanding for practical use. Recent 3D-LLMs and 2D-VLMs perform poorly on these tasks, lacking both comparison-centric design and geometric awareness. In contrast, Multi-3DLLM trained on our mixture data learns geometric reasoning, surpasses all baselines on MO3D, and provides positive transfer to single-object classification.
Kohsuke Ide, Ryousuke Yamada, Yue Qiu +4
AIST · University of Tsukuba · University of Technology Nuremberg +1
We propose Tetris3D, a generative framework for single-image 3D scene reconstruction that recovers objects which are physically and geometrically coherent as a scene. Existing methods often generate objects independently or couple them implicitly, providing limited guidance for ensuring fine-grained spatial compatibility between neighboring objects that interact with one another. To address this, we explicitly condition the generation of each object on the geometry of surrounding objects and their physical relationships, guiding its shape and pose to remain geometrically and physically plausible within the scene. Moreover, we introduce ComOb, a physics simulation-based dataset of 1.2M scenes featuring physical interactions across diverse object categories, with per-object meshes and pairwise physical relation annotations. Comprehensive experiments on synthetic and realworld scenes show that Tetris3D recovers coherent object shapes and poses even when interacting regions are occluded, and achieves state-of-the-art performance in both generation quality and physical stability.
Single-view 3D scene reconstruction involves inferring both object geometry and spatial layout. Existing methods typically reconstruct objects independently or rely on implicit scene context, failing to exploit the repeated instances commonly present in realworld scenes. We propose FurnSet, a framework that explicitly identifies and leverages repeated object instances to improve reconstruction. Our method introduces per-object CLS tokens and a set-aware self-attention mechanism that groups identical instances and aggregates complementary observations across them, enabling joint reconstruction. We further combine scene-level and object-level conditioning to guide object reconstruction, followed by layout optimization using object point clouds with 3D and 2D projection losses for scene alignment. Experiments on 3D-Future and 3D-Front demonstrate improved scene reconstruction quality, highlighting the effectiveness of exploiting repetition for robust 3D scene reconstruction.