Organizations: Department of Electrical and Computer Engineering, Democritus University of Thrace, Greece · Institute for Language and Speech Processing, Athena Research Center, Greece · Department of Informatics & Telecommunications, National and Kapodistrian University of Athens, Greece · Archimedes, Athena Research Center, Greece
Slot-based object-centric learning represents an image as a set of latent slots with a decoder that combines them into an image or features. The decoder specifies how slots are combined into an output, but the slot set is typically fixed: the number of slots is chosen upfront and slots are only refined. This can lead to multiple slots competing for overlapping regions of the same entity rather than focusing on distinct regions. We introduce slot merging: a drop-in, lightweight operation on the slot set that merges overlapping slots during training. We quantify overlap with a Soft-IoU score between slot-attention maps and combine selected pairs via a barycentric update that preserves gradient flow. Merging follows a fixed policy, with the decision threshold inferred from overlap statistics, requiring no additional learnable modules. Integrated into the established feature-reconstruction pipeline of DINOSAUR, the proposed method improves object factorization and mask quality, surpassing other adaptive methods in object discovery and segmentation benchmarks.
Figures & tables
Figure 1: We introduce a merge operator over the slot set that adaptively refines factorization, producing coherent object-level representations.
Figure 2: Illustration of the proposed pipeline.
Figure 3: Overview of the proposed training pipeline and merging mechanism. After Slot Attention produces refined slots and attention maps, pairwise Attention-IoU scores are computed to identify the most overlapping slot pair. The selected slots are merged via barycentric aggregation and their attention maps are summed. The procedure is repeated iteratively until no overlap exceeds the threshold, yielding a refined slot set passed to the decoder.
Figure 4: Evolution of slot representations during training (left to right). Slot assignments transition from fragmented and unstable to progressively specialized and object-aligned. After stabilization (right), slot merging is enabled to resolve overlap between slots.
Figure 5: Qualitative results on COCO and PASCAL VOC.
VOC
MOVi-C
mBO i
mBO c
mIoU
mBO i
mIoU
Baseline
44.0 ± 1.9
51.2 ± 1.9
–
42.4
–
Merging during Inference
46.01 ± 0.7
52.68 ± 0.65
44.51 ± 0.55
44.53 ± 1.7
43.81 ± 1.6
Merging during Training
47.83 ± 0.17
55.34 ± 0.27
46.01 ± 0.18
48.35 ± 1.32
47.43 ± 1.26
Table 1: Comparison between inference-only and training-integrated slot merging on VOC and MOVi-C, evaluated with mBO (instance and class). Results are averaged over 3 seeds.
VOC
MOVi-C
mBO i
mBO c
mIoU
mBO i
mIoU
With Detach
45.75 ± 0.28
53.13 ± 0.42
43.97 ± 0.22
45.39 ± 2.95
44.46 ± 2.9
Without Detach
47.83 ± 0.17
55.34 ± 0.27
46.01 ± 0.18
48.35 ± 1.32
47.43 ± 1.26
Table 2: Ablation on gradient detachment in the barycentric slot merging operator during training on VOC and MOVi-C. Results are the mean and standard deviation of the decoder’s metrics over 3 seeds.
mBO i
mBO c
mIoU
Without Aggregation
47.38 ± 0.52
54.83 ± 0.58
45.53 ± 0.49
With Aggregation
47.83 ± 0.17
55.34 ± 0.27
46.01 ± 0.18
Table 3: Effect of attention map aggregation during slot merging. Results are averaged over 3 seeds.
Training / Inference
VOC
COCO
MOVi-C
MOVi-E
Mean
1.67 / 1.73
0.69 / 0.82
2.2 / 2.2
7.96 / 7.91
Std
1.0 / 0.98
0.76 / 0.81
1.32 / 1.32
2.13 / 2.08
Table 4: Merging statistics during training and inference across all four datasets, reporting the mean and standard deviation of the number of merged slots per image.
VOC
MOVi-C
Epoch
mBO i
mBO c
mIoU
Epoch
mBO i
mIoU
90
46.15
52.91
44.65
50
47.05
46.19
80
46.96
54.10
45.25
40
46.83
46.00
70
47.83
55.33
46.01
30
48.35
47.43
Table 5: Influence of the training epoch at which slot merging is activated, evaluated on VOC and MOVi-C.
VOC
COCO
Method
mBO i
mBO c
mBO i
mBO c
SA [ 24 ] † NeurIPS’20
24.6
24.9
17.2
19.2
SLATE [ 32 ] † ICLR’22
35.9
41.5
29.1
33.6
CAE [ 26 ] † TMLR
32.9 ± 0.9
37.4 ± 1.0
–
–
Rotating Features [ 25 ] † NeurIPS’23
40.7 ± 0.1
46.0 ± 0.1
–
–
DINOSAUR-MLP [ 31 ] † ICLR’23
39.5 ± 0.1
40.9 ± 0.1
27.7 ± 0.2
30.9 ± 0.2
Table 6: Comparison with object-centric methods on the VOC and COCO datasets. Results for our approach are reported as mean ± standard deviation over 3 seeds, using a ViT-B/16 backbone and 224×224 input resolution. † indicates results taken from [ 17 ] . Otherwise results are from the original papers.
MOVi-C
MOVi-E
Method
mBO i
mIoU
mBO i
mIoU
SA [ 24 ] † NeurIPS’20
26.2 ± 1.0
–
24.0 ± 1.2
–
SLASH [ 19 ] † CVPR’23
–
27.7 ± 5.9
–
–
SLATE [ 32 ] † ICLR’22
39.4 ± 0.8
37.8 ± 0.7
30.2 ± 1.7
28.6 ± 1.7
DINOSAUR-MLP [ 31 ] † ICLR’23
39.1 ± 0.2
–
35.5 ± 0.2
–
DINOSAUR [ 31 ] † ICLR’23
42.4
–
–
–
Table 7: Comparison with object-centric methods on MOVi-C and MOVi-E. Results for our approach are reported as mean ± standard deviation over 3 seeds. † indicates results taken from [ 17 ] . Otherwise results are from the original papers.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Visualizations of the iterative slot merging process. Starting from slot representations produced by Slot Attention, the algorithm repeatedly merges the pair of slots with the highest Soft-IoU overlap. The final merged slot set is then passed to the decoder for reconstruction.
Figure 7: (a) Data-driven estimation of threshold from the histogram of pairwise Soft-IoU scores using the maximum perpendicular distance criterion. (b) Sensitivity analysis of the merging threshold on the VOC dataset measured using mBOi and mBOc .
Encoder
Method
mBO i
mBO c
mIoU
DINO
DINOSAUR
44.0
51.2
–
Ours
47.8
55.3
46.0
DINOv2
DINOSAUR
43.2
47.3
42.1
Ours
44.4
50.5
43.2
MAE
DINOSAUR
37.8
42.7
36.5
Ours
38.3
43.4
36.7
Appendix
Table 8: Different encoders comparison. DINO [ 3 ] and MAE [ 15 ] use ViT-B/16 architecture [ 6 ] while DINOv2 [ 27 ] employs ViT-B/14.
Method
VOC
MOViC
DINOSAUR
∼ 53 minutes
∼ 4h 44 minutes
Ours
∼ 53.5 minutes
∼ 4h 50 minutes
Appendix
Table 9: Training time comparison between DINOSAUR and our method.
Table 10: Comparison with object-centric methods on VOC, COCO, MOVi-C, and MOVi-E datasets. Results for our approach are reported as mean ± standard deviation over 3 seeds. † indicates results taken from [ 17 ] . Otherwise results are from the original papers. The best results are shown in bold and the second-best results are underlined .
Object-Centric Representation Learning studies visual models that organize scenes into distinct objects and their properties, positions, and relations.
Video Object-Centric Learning seeks to decompose raw videos into a small set of object slots, but existing slot-attention models often suffer from severe over-fragmentation. This is because the model is implicitly encouraged to occupy all slots to minimize the reconstruction objective, thereby representing a single object with multiple redundant slots. We tackle this limitation with a reconstruction-guided slot curriculum (SlotCurri). Training starts with only a few coarse slots and progressively allocates new slots where reconstruction error remains high, thus expanding capacity only where it is needed and preventing fragmentation from the outset. Yet, during slot expansion, meaningful sub-parts can emerge only if coarse-level semantics are already well separated; however, with a small initial slot budget and an MSE objective, semantic boundaries remain blurry. Therefore, we augment MSE with a structure-aware loss that preserves local contrast and edge information to encourage each slot to sharpen its semantic boundaries. Lastly, we propose a cyclic inference that rolls slots forward and then backward through the frame sequence, producing temporally consistent object representations even in the earliest frames. All combined, SlotCurri addresses object over-fragmentation by allocating representational capacity where reconstruction fails, further enhanced by structural cues and cyclic inference. Notable FG-ARI gains of +6.8 on YouTube-VIS and +8.3 on MOVi-C validate the effectiveness of SlotCurri. Our code is available at github.com/wjun0830/SlotCurri.
Slot Attention, an approach that binds different objects in a scene to a set of "slots", has become a leading method in unsupervised object-centric learning. Most methods assume a fixed slot count K, and to better accommodate the dynamic nature of object cardinality, a few works have explored K-adaptive variants. However, existing K-adaptive methods still suffer from two limitations. First, they do not explicitly constrain slot-binding quality, so low-quality slots lead to ambiguous feature attribution. Second, adding a slot-count penalty to the reconstruction objective creates conflicting optimization goals between reducing the number of active slots and maintaining reconstruction fidelity. As a result, they still lag significantly behind strong K-fixed baselines. To address these challenges, we propose Quality-Guided K-Adaptive Slot Attention (QASA). First, we decouple slot selection from reconstruction, eliminating the mutual constraints between the two objectives. Then, we propose an unsupervised Slot-Quality metric to assess per-slot quality, providing a principled signal for fine-grained slot--object binding. Based on this metric, we design a Quality-Guided Slot Selection scheme that dynamically selects a subset of high-quality slots and feeds them into our newly designed gated decoder for reconstruction during training. At inference, token-wise competition on slot attention yields a K-adaptive outcome. Experiments show that QASA substantially outperforms existing K-adaptive methods on both real and synthetic datasets. Moreover, on real-world datasets QASA surpasses K-fixed methods.
Studying attention through the lens of interacting particle dynamics has shown how token clustering can emerge from the underlying dynamics. We extend this perspective to slot attention, a method for object-centric image segmentation and representation learning in which learned components obscure how much of the clustering behaviour is intrinsic to the attention dynamics. We therefore introduce simplified slot attention (SSA), a parameter-free variant whose dynamics are connected to soft k-means clustering and which provides a straightforward mechanistic explanation for the emergence of object-centric representations. On the Pascal VOC dataset, SSA achieves performance comparable to that of slot attention, demonstrating that competitive object-centric segmentation can be achieved without learned neural-network components.