SlotMatch: Distilling Object-Centric Representations for Unsupervised Video Segmentation

Diana-Nicoleta Grigore; Neelu Madan; Andreas Mogelmose; Thomas B. Moeslund; Radu Tudor Ionescu

cs.CVAug 5, 2025

SlotMatch: Distilling Object-Centric Representations for Unsupervised Video Segmentation

Authors: Diana-Nicoleta Grigore, Neelu Madan, Andreas Mogelmose, Thomas B. Moeslund, Radu Tudor Ionescu

Organizations: University of Bucharest, Romania · Aalborg University, Denmark

Abstract

Unsupervised video segmentation is a challenging computer vision task, especially due to the lack of supervisory signals coupled with the complexity of visual scenes. To overcome this challenge, state-of-the-art models based on slot attention often have to rely on large and computationally expensive neural architectures. To this end, we propose a simple knowledge distillation framework that effectively transfers object-centric representations to a lightweight student. The proposed framework, called SlotMatch, aligns corresponding teacher and student slots via the cosine similarity, requiring no additional distillation objectives or auxiliary supervision. Unlike other distillation losses, our cosine-based distillation of slots enforces alignment of slot directions (which typically encode relational structure and semantic similarity), while ignoring their norms (which may encode confidence overestimation and dataset-specific scale artifacts). The simplicity of SlotMatch is supported by theoretical and empirical evidence, both indicating that integrating additional losses may be suboptimal. We conduct experiments on three datasets to compare state-of-the-art models, including our SlotContrast teacher, with our distilled student. The results show that our student based on SlotMatch matches and even outperforms its teacher, while using 3.6x fewer parameters and running up to 2.7x faster. Moreover, our student surpasses all other state-of-the-art unsupervised video segmentation models. We release our code at https://github.com/dianagrigore/SlotMatch.

Figures & tables

Panel of Figure 1 — Figure 1: Intuition (a) and motivation (b) behind SlotMatch . Unlike alternative KD procedures, SlotMatch boosts both performance and speed, thus bearing a high practical motivation. This is possible due to the main focus of our KD approach based on cosine similarity between slots. SlotMatch aims to transfer only useful information (slot directions), while disregarding harmful dataset-specific artifacts (slot norms). Best viewed in color.

Figure 2: Our SlotMatch framework performs knowledge distillation from a large frozen teacher model, e.g . SlotContrast [ 25 ] , to a compact trainable student model. Both models process video frames through slot attention mechanisms, with the student learning through three loss components: reconstruction ( $\mathcal{L}_{\text{rec}}$ ), temporal consistency ( $\mathcal{L}_{\text{slot-contrast}}$ ), and our angular slot matching loss ( $\mathcal{L}_{\text{slot-KD}}$ ) that directly aligns corresponding slot directions between teacher and student models using cosine similarity.

Method	MOVi-E				YTVIS-2021				DAVIS 2017
	Image		Video		Image		Video		Image		Video
	FG-ARI $\uparrow\,$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
SAVi [ 20 ]	39.2	-	42.8	16.0	-	-	-	-	-	-	-	-
STEVE [ 31 ]	54.1	-	50.6	26.6	-	-	15.0	19.1	-	-	3.7	5.9
VideoSAUR [ 34 ]	78.4	-	73.9	35.6	-	-	28.9	26.3	12.6	23.9	8.3	18.4
VideoSAURv2 [ 25 ]	-	-	77.1	34.4	-	-	31.2	29.7	11.8	23.0	7.6	19.2

Table 1: State-of-the-art comparison on MOVi-E, YTVIS-2021, and DAVIS 2017. We report FG-ARI and mBO at both image and video levels. The top two scores for each metric are highlighted in blue bold (best) and orange bold (second), respectively. For SlotMatch , we report average scores and standard deviations over 3 runs.

Figure 3: Qualitative segmentation results on MOVi-E (left) and YTVIS-2021 (right). The first row shows raw frames; the second and third rows show slots from the student and teacher models, respectively; the final row presents results from our distillation-based SlotMatch . SlotMatch recovers missed slots, refines object boundaries, and produces sharper, more consistent slots. Mistakes by the student and teacher models are annotated in red , while corrections and additional detections introduced by SlotMatch are highlighted in green . Best viewed in color.

Method	#Params	#GFLOPs $\downarrow$	Inference Time (ms) $\downarrow$
Method	(M) $\downarrow$	#GFLOPs $\downarrow$	A100	$3090$	$2080$
Teacher	91.0	8825.7	347.4	700.5	916.5
SlotMatch	25.2	3266.5	186.0	261.4	331.0
Reduction	3.61 $\times$	2.70 $\times$	1.87 $\times$	2.68 $\times$	2.77 $\times$

Table 2: Comparing inference speed, number of parameters, and GFLOPs between the SlotContrast teacher and our SlotMatch student. Measurements are performed on three GPUs (NVIDIA A100 with 40GB VRAM, NVIDIA 3090 with 24GB VRAM, NVIDIA 2080 with 12GB VRAM), using video samples from the YTVIS-2021 dataset. Inference times correspond to a whole video of average temporal length (30 frames).

Method	Image		Video
Method	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
Student (no KD)	54.6	24.9	34.6	21.5
SlotMatch	55.8	25.5	34.8	21.5

Table 3: Zero-shot generalization results on the OVIS dataset, using models trained only on YTVIS (see performance on original dataset in Table 1 ). We compare the student trained from scratch (no KD) versus our SlotMatch student.

Method	MOVi-E				YTVIS-2021				DAVIS 2017
	Image		Video		Image		Video		Image		Video
	FG-ARI $\uparrow\,$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
Student (no KD)	80.1	28.9	76.6	30.1	44.5	38.8	37.2	32.1	93.7	11.9	76.6	11.6
Feature KD	81.8	31.6	80.1	28.1	45.2	38.2	35.9	31.4	94.6	12.8	91.6	12.7
Reconstruction KD	81.4	31.5	81.1	29.1	44.9	38.8	35.6	31.8	92.0	20.5	88.4	20.3
Contrastive KD	20.3	26.6	20.3	25.6	44.9	38.7	35.5	31.2	94.1	14.8	88.4	13.2

Table 4: Distillation baselines and ablations on MOVi-E, YTVIS-2021, and DAVIS 2017. We report FG-ARI and mBO at both image and video levels. The top two scores for each metric are highlighted in blue bold (best) and orange bold (second), respectively.

Method	$\beta$	Image		Video
Method	$\beta$	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
Teacher	-	45.5	39.7	36.8	32.4
Student (no KD)	-	44.5	38.8	37.2	32.1
SlotMatch	0.1	44.5	39.1	34.9	32.5
	0.2	45.8	39.8	37.3	32.8
	0.3	44.4	39.1	36.3	32.5

Table 5: Ablation study on the YTVIS dataset showing the effect of varying the weight

\beta

of the distillation loss

\mathcal{L}_{\text{slot-KD}}

in SlotMatch . The best performance across all metrics is achieved at

\beta=0.2

Method	Image		Video
Method	FG-ARI $\uparrow\,$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
SlotContrast (Teacher)	83.9	32.4	81.7	28.6
Student (no KD)	80.1	28.9	76.6	30.1
SlotMatch (w/ matching)	83.5	31.8	81.5	28.6
SlotMatch (w/o matching)	84.1	33.6	81.8	30.5

Table 6: Comparison of distillation performance with and without Hungarian matching on the MOVi-E dataset. Omitting Hungarian matching (as proposed) leads to better performance.

Figure S1 : Qualitative comparison on MOVi-E (left) and YTVIS-2021 (right). The second row shows outputs from the student model (no KD), while the third row presents results from our distillation-based SlotMatch . Student (no KD) errors, including missed slots, are marked in red . Corrections and additional slots introduced by SlotMatch are highlighted in green . Best viewed in color.

Method	Image mBO $\uparrow$	Video mBO $\uparrow$
VideoSAUR [ 34 ] (teacher)	23.9	18.4
Student (no KD)	17.4	14.4
SlotMatch (ours)	30.3	23.2
VideoSAURv2 [ 25 ] (teacher)	23.0	19.2
Student (no KD)	22.9	18.2
SlotMatch (ours)	31.2	28.9

Table S1: Video segmentation results on DAVIS [ 27 ] with two different teacher models, VideoSAUR [ 34 ] and VideoSAURv2 [ 25 ] . The top two scores for each metric are highlighted in blue bold (top method) and orange bold (second best).

Method	Image FG-ARI $\uparrow$	Image mBO $\uparrow$
DINOSAURv2 [ 30 ] (teacher)	44.7	29.3
Student (no KD)	41.5	28.6
SlotMatch (ours)	42.3	28.8

Table S2: Image segmentation results on MS COCO [ 23 ] with DINOSAURv2 [ 30 ] as teacher. The top two scores for each metric are highlighted in blue bold (top method) and orange bold (second best).

Figure S2 : Distribution of slot

L_{2}

slot norms for teacher and student models measured on YTVIS-2021 validation set. Left: Histogram of all slot norms showing that the student slot norm distribution is visibly shifted towards lower values, while preserving the same Gaussian shape. Right: Mean slot norm per batch, confirming the gap is stable across the entire dataset and not a local artifact. Best viewed in color.

Metric	Teacher	Student	Ratio (S/T)
Median	2.7260	2.3350	0.8566
Mean	2.7274	2.3416	0.8586
Std. dev.	0.2288	0.1887	0.8246

Table S3:

L_{2}

slot norm statistics for teacher and student models measured on the YTVIS-2021 validation set. The student consistently operates at a lower slot magnitude (

\approx 14\%

lower), confirming that cosine-based distillation decouples directional alignment from magnitude.

Method	FG BBox IoU $\uparrow$	Count acc. $\uparrow$	GFLOPs $\downarrow$
Fine-tune then distill	0.424	0.799	17,415
Distill then fine-tune (ours)	0.437	0.802	3,015

Table S4 : Comparison of “fine-tune then distill” vs. “distill then fine-tune” on DAVIS 2017 localization and counting downstream tasks. FG BBox IoU evaluates overlap with the foreground bounding box enclosing all annotated objects. Count accuracy evaluates the predicted number of annotated foreground object IDs. The top score for each metric is highlighted in blue bold .

Method	Image		Video
Method	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
Better target (DINOv2-B recon.)	41.8	39.4	33.3	31.5
SlotMatch (ours)	45.8	39.8	37.3	32.8

Table S5 : Ablation of the reconstruction target on YTVIS-2021. The top score for each metric is highlighted in blue bold .

Method	Image		Video
Method	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
SlotContrast (ViT-B)	89.1	12.1	73.3	11.8
SlotMatch (ViT-tiny)	94.6	13.9	88.0	13.0
SlotMatch (ViT-S)	95.6	30.6	92.8	31.6

Table S6 : Results with ViT-tiny as student on DAVIS 2017.

Method	Image		Video
Method	FG-ARI $\uparrow$	mBO $\uparrow$	FG-ARI $\uparrow$	mBO $\uparrow$
SlotContrast (Teacher)	89.1	12.1	73.3	11.8
Student (no KD)	93.7	11.9	76.6	11.6
Slot KD (L2)	94.4	20.0	89.6	20.0
SlotMatch + Norm KD	91.8	20.3	88.0	17.9
SlotMatch (ours)	95.6	30.6	92.8	31.6

Table S7 : Effect of explicitly constraining student slot magnitudes to match teacher slot magnitudes on DAVIS 2017. The constrained version ( SlotMatch + Norm KD) integrates the loss defined in Eq. ( 23 ) in the training objective.

Hyperparameter	MOVi-E		YTVIS-2021		DAVIS 2017
Hyperparameter	Teacher	Student	Teacher	Student	Teacher	Student
Backbone	ViT-B/14	ViT-S/14	ViT-B/14	ViT-S/14	ViT-B/14	ViT-S/14
Feature size ( $m$ )	768	384	768	384	768	384
Slot dim ( $S$ )	128	128	64	64	128	128
#Slots ( $N$ )	15	15	7	7	4	4
Input size	336 $\times$ 336	336 $\times$ 336	518 $\times$ 518	518 $\times$ 518	336 $\times$ 336	336 $\times$ 336

Table S8: Summary of key hyperparameters for all teacher and student models used in our experiments. All models use the same decoder, predictor, and loss weights.

Figure S3 : Slot visualization on DAVIS 2017. Best viewed in color.

Method	Dataset	Steps	GPU Hours
SlotMatch	MOVi-E	300K	$\approx$ 138
	YTVIS	100K	$\approx$ 39
	DAVIS	100K	$\approx$ 25

Table S9: Approximate training time per experiment on two A100 GPUs.

Run	Seed	Video FG-ARI $\uparrow$	Video mBO $\uparrow$
Run 1	42	37.2	33.1
Run 2	101	37.4	32.4
Run 3	2048	37.3	32.9
Mean	-	37.3	32.8
Std.	-	$\pm$ 0.06	$\pm$ 0.29

Table S10: Robustness evaluation of SlotMatch on the YTVIS dataset using three random seeds (42, 101, 2048). The last row reports the average performance across seeds. Consistent results across runs indicate stable training.

Video Object Segmentation

Video object segmentation assigns pixel-level masks to objects across frames while preserving their identities through motion, occlusion, and change.

See all 110 papers

Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning
Jun 10, 2026Sieu Tran, Duc Nguyen, Hao Vo +2Object-Centric Representation Learning Video Object-Centric Learning
University of Arkansas, Fayetteville, AR, USA
Reconstruction-Guided Slot Curriculum: Addressing Object Over-Fragmentation in Video Object-Centric Learning
Mar 24, 2026WonJun Moon, Hyun Seok Seong, Jae-Pil HeoVideo Object-Centric Learning Object-Centric Representation Learning
Sungkyunkwan University, South Korea
Semantic Slots for Video Object-Centric Learning
Aug 21, 2026Khalil Sabri, Guillaume-Alexandre Bilodeau, Nicolas Saunier +1Video Object-Centric Learning Object-Centric Representation Learning
Polytechnique Montréal Montréal, Canada · Université TÉLUQ Montréal, Canada

Jun 10, 2026J/K move · Enter open · S save

Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning

Sieu Tran, Duc Nguyen, Hao Vo, Khoa Vo+1

University of Arkansas, Fayetteville, AR, USA

Object-Centric Representation Learning Video Object-Centric Learning Slot Attention Self-Supervised Visual Representation Learning

Open paper AskPDF arXiv

Unsupervised video object-centric learning aims to decompose dynamic scenes into persistent, object-level representations without supervision. However, existing slot-based methods struggle to maintain stable object identity in challenging settings such as rapid motion and partial occlusion. First, they typically encode both the per-frame appearance of an object and its identity across frames in a single slot vector, creating an objective conflict that leads to slot swapping: reconstruction requires sensitivity to transient visual changes, whereas temporal consistency requires invariance to them. Second, the token renormalization used in Slot Attention can amplify weakly attending slots, allowing them to absorb tokens from other objects and destabilize slot-to-object correspondence. We propose Dual-State Slot Attention (DSSA), a fully self-supervised framework that addresses these limitations by separating appearance from identity and by reducing spurious updates from weakly matching slots. DSSA decomposes each slot into a local state for per-frame appearance and an identity state for temporally stable object information, thereby aligning reconstruction and temporal consistency with separate representations. The identity state is updated through a learned recurrent transition that acts as a temporal filter on the local state, while competition-modulated aggregation (CMA) down-weights updates from weakly matching slots and prevents them from absorbing tokens from other objects. Experiments on MOVi-C, MOVi-D, and YouTube-VIS demonstrate that DSSA consistently improves segmentation quality and temporal consistency over prior methods, while also yielding stronger downstream object recognition and video dynamics prediction. Code is available at uark-aicv.github.io/dssa.

Abstract

Video Object Segmentation

Explore similar work

Dual-State Slot Attention: Decoupling Appearance and Identity for Video Object-Centric Learning

Reconstruction-Guided Slot Curriculum: Addressing Object Over-Fragmentation in Video Object-Centric Learning

Semantic Slots for Video Object-Centric Learning