GeoCo-SAVi: Geometry-Consistent Slot Attention for Explicitly Editable Object Representations

Haoxiang Huang; Zhekai Wang; Xiang Liu; Shuwei Wang; Yiwen Sun; Zhiwei Yu; Jingheng Ma; Sen Cui

cs.CVSep 6, 2026

GeoCo-SAVi: Geometry-Consistent Slot Attention for Explicitly Editable Object Representations

Authors: Haoxiang Huang, Zhekai Wang, Xiang Liu, Shuwei Wang, Yiwen Sun, Zhiwei Yu, Jingheng Ma, Sen Cui,

Organizations: University of Science and Technology of China · Beijing Institute of Technology · Tsinghua University · Chinese Academy of Sciences ·

Abstract

Object-centric video models represent scenes with slots, yet exposed geometry can vary in meaning with appearance. In Invariant Slot Attention (ISA), explicit position and scale can disagree with the decoded center and extent; edits can yield unexpected motion or resizing, and replacing appearance can shift geometry. GeoCo-SAVi promotes geometric authority and semantic alignment. Its spatially equivariant, object-wise decoder makes position and scale effective commands: changing them moves or resizes the rendered support. Factual position alignment ties position to the decoded center, and normalized attention overlap discourages duplicate allocation. Appearance transplantation aligns geometry semantics across objects, so recipient geometry governs layout while donor appearance supplies shape. On Obj3D, GeoCo-SAVi matches ISA reconstruction, reduces latent-position-to-decoded-centroid error by nearly 90%, and reduces appearance-induced size variation while producing the expected translation and scale responses. On 250 MOVi-C videos, it improves reconstruction and instance grouping over both same-protocol references, and video editing over STAITUS. GeoCo-SAVi transforms explicit geometry into compositional control, making both position and scale more readable and editable. Project page: https://GeoCo-SAVi.github.io/.

Figures & tables

Figure 2: GeoCo-SAVi overview. A: ISA infers per-slot appearance and geometry; a spatially equivariant decoder renders RGB and alpha. B1: factual losses calibrate allocation and position. B2: transplants align recipient center and extent while retaining donor compactness.

Panel of Table 1 — Table 1: Single-frame Obj3D results. PSNR and $O_{\mathrm{attn}}$ cover the full validation set; $E_{p\rightarrow\hat{c}}$ and editing use the metric-specific scopes in Sec. 4 . Reference errors use 9,255 shared RGB pseudo-label objects. Spreads fix $s=0.2$ , with sample counts in the supplement; editing F1 is in percent. Best values are bold.

Method	FG-ARI	ARI	mIoU	mBO
STAITUS	0.1550	0.1114	0.1242	0.1307
STATM-SAVi	0.1619	0.4052	0.2035	0.2085
GeoCo-SAVi	0.1935	0.4610	0.2451	0.2507

Table 2: MOVi-C segmentation on 250 equally weighted validation videos. Best values are bold.

Method	PSNR $\uparrow$	$E_{p\rightarrow\hat{c}}\downarrow$	$E_{p\rightarrow c^{\star}}\downarrow$	$E_{\hat{c}\rightarrow c^{\star}}\downarrow$	$O_{\mathrm{attn}}\downarrow$	$\sigma_{r\mid g}\downarrow$	$\sigma_{A\mid g}\downarrow$
STATM-SAVi	21.314	N/A	N/A	N/A	0.0045	N/A	N/A
STAITUS	19.960	0.0868	0.3417	0.3128	0.0008	0.1756	0.1439
GeoCo-SAVi	22.808	0.0276	0.2403	0.2341	0.0011	0.0503	0.0191

Table 3: MOVi-C reconstruction, geometry, and editing. Position errors use 1,543 GT tracks with fixed pre-edit association; PSNR and

O_{\mathrm{attn}}

use all 250 videos; spreads fix

s=0.2

, with sample counts in the supplement. F1 (%) pools all 24 frames per video and averages 238 shared recipient/donor pairs. Other response metrics use objects within the canvas. N/A denotes an unsupported geometry interface; best values are bold.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Decoder	PSNR $\uparrow$	$\mathrm{PSNR}_{\mathrm{pos}}\uparrow$	$\mathrm{PSNR}_{\mathrm{scl}}\uparrow$	$\mathrm{PSNR}_{\mathrm{app}}\uparrow$	$F_{\mathrm{pos}}\uparrow$	$F_{\mathrm{scl}}\uparrow$	$F_{\mathrm{app}}\uparrow$
Scale-steered	44.210	39.914	40.176	35.728	93.29	92.72	91.09
Canonical-canvas	43.501	39.568	39.945	36.148	93.00	93.94	89.17

Appendix

Table 5: Obj3D decoder implementations within GeoCo-SAVi at 70k updates.

\mathrm{PSNR}_{\mathrm{pos/scl/app}}

measures full-image agreement with independent RGB edit targets (Sec. E.6 ). F-scores are percentages; PSNR is in dB.

		Radius			Coverage
Dataset	Method	Mean	Std. $\downarrow$	CV $\downarrow$	Mean	Std. $\downarrow$	CV $\downarrow$
Obj3D	ISA	0.15430	0.04294	0.27831	0.03656	0.01793	0.49034
	SlotAug	0.14300	0.04978	0.34815	0.03266	0.02145	0.65703
	GeoCo-SAVi	0.15521	0.02029	0.13075	0.03364	0.00836	0.24846
MOVi-C	STAITUS	0.25065	0.09912	0.49495	0.10652	0.07563	0.88901
	GeoCo-SAVi	0.14799	0.04616	0.32078	0.02830	0.01491	0.54265

Appendix

Table 6: Fixed-geometry stability with shared factual backgrounds at

s=0.2

. Each statistic is computed across 1,024 appearances within a background and averaged over eight backgrounds.

Dataset / method	$E_{p\rightarrow\hat{c}}$	$E_{p\rightarrow c^{\star}}$	$E_{\hat{c}\rightarrow c^{\star}}$
Obj3D / ISA	0.10135	0.10232	0.04204
Obj3D / SlotAug	0.04658	0.05794	0.03584
Obj3D / GeoCo-SAVi	0.01126	0.04361	0.03910
MOVi-C / STAITUS	0.08683	0.34170	0.31277
MOVi-C / GeoCo-SAVi	0.02761	0.24032	0.23406

Appendix

Table 7: External position evaluation in normalized-grid L2 units: Obj3D uses 9,255 RGB pseudo-reference objects from all 2,600 windows, and MOVi-C uses 1,543 visible GT tracks. Within-model distance is evaluated on the same associated objects.

Dataset / method	$E_{p\rightarrow\hat{c}}$	Defined	Edit objects
Obj3D / ISA	0.10489	9424	9424
Obj3D / SlotAug	0.04763	9357	9357
Obj3D / GeoCo-SAVi	0.01065	9320	9320
MOVi-C / STAITUS	0.06865	51962	51971
MOVi-C / GeoCo-SAVi	0.02567	51600	51629

Appendix

Table 8: All-slot position error in normalized-grid L2 units. Position uses original factual ownership; video editing uses a consistent decoder precision for factual reference and intervention, so valid-object counts can differ slightly.

Method / scope	$F_{\rm pos}$	$F_{\rm scl}$	$F_{\rm app}$	$E_{\Delta p}$	$D_{\rm app}$
ISA (all)	86.35	62.84	75.73	0.01012	0.05410
	[85.84, 86.83]	[62.40, 63.28]	[74.44, 76.93]
SlotAug (all)	90.78	82.19	78.20	0.00770	0.06012
	[90.34, 91.22]	[82.03, 82.34]	[76.79, 79.49]
GeoCo-SAVi (all)	93.29	92.72	91.09	0.00645	0.02083
	[93.03, 93.55]	[92.60, 92.83]	[90.43, 91.64]

Appendix

Table 9: Obj3D editing results. F1 (%) uses valid edits in each scope; interior uses factual/ideal feasibility. Brackets give 95% confidence intervals for the main-table F-scores.

E_{\Delta p}

is a diagonal fraction and

D_{\rm app}

is grid L2.

Method / scope	$\beta_{r}$	$\beta_{A}$	$\beta_{r}$ def.	$\beta_{A}$ def.
ISA (all)	0.09882	-0.01512	9344/9424	9370/9424
SlotAug (all)	0.54612	1.04735	9339/9357	9344/9357
GeoCo-SAVi (all)	1.03473	1.98611	9303/9320	9310/9320
ISA (interior)	0.09388	-0.00923	6928/6952	6939/6952
SlotAug (interior)	0.54118	1.05748	7310/7314	7311/7314
GeoCo-SAVi (interior)	1.03391	2.02246	7380/7386	7384/7386

Appendix

Table 10: Obj3D five-point scale response. Slopes are means over defined curves; counts are defined/total.

Method / scope	$N_{\rm pos}$	$N_{\rm scl}$	$N_{\rm app}$	$E_{\Delta p}$ def.	$D_{\rm app}$ def.
ISA (all)	37144	36631	7402	37511/37696	7417/7418
SlotAug (all)	36843	36619	7691	37068/37428	7689/7691
GeoCo-SAVi (all)	36616	36858	7397	37221/37280	7397/7397
ISA (interior)	30842	30357	4564	30831/30842	4564/4564
SlotAug (interior)	31390	31258	5293	31328/31390	5291/5293
GeoCo-SAVi (interior)	30319	30654	4698	30319/30319	4698/4698

Appendix

Table 11: Obj3D scope-specific F1 denominators and defined/total center-error counts. F1 retains all actual outputs for valid edits, including empty outputs.

Method / scope	Position	Scale	Transplant
ISA (within-canvas)	11/30842 (0.04)	69/30357 (0.23)	0/4564 (0.00)
SlotAug (within-canvas)	62/31390 (0.20)	8/31258 (0.03)	2/5293 (0.04)
GeoCo-SAVi (within-canvas)	0/30319 (0.00)	4/30654 (0.01)	0/4698 (0.00)
ISA (all)	79/37358 (0.21)	70/37696 (0.19)	1/7418 (0.01)
SlotAug (all)	78/37064 (0.21)	16/37428 (0.04)	2/7691 (0.03)
GeoCo-SAVi (all)	1/36805 (0.00)	12/37280 (0.03)	0/7397 (0.00)

Appendix

Table 12: Obj3D conditional output diagnostics, shown as count/eligible count (%). Disappearance counts empty outputs for positive-area ideal targets.

Method	$s$	$\bar{r}$	$\sigma_{r\mid g}$	$\mathrm{CV}_{r\mid g}$	$\sigma_{A\mid g}$	$N$
ISA	0.1	0.14826	0.03351	0.22600	0.01239	9424
ISA	0.2	0.15308	0.04405	0.28775	0.01833	9424
ISA	0.3	0.15779	0.07107	0.45039	0.02911	9424
SlotAug	0.1	0.12094	0.02476	0.20475	0.00874	9357
SlotAug	0.2	0.14279	0.05201	0.36426	0.02265	9357
SlotAug	0.3	0.10030	0.10388	1.03572	0.04049	9357

Appendix

Table 13: Obj3D pooled cross-appearance fixed-geometry statistics. The radius coefficient of variation is

\mathrm{CV}_{r\mid g}=\sigma_{r\mid g}/\bar{r}

. At

s=0.2

, GeoCo-SAVi and ISA have similar mean radii, with lower radius standard deviation and CV for GeoCo-SAVi.

N

counts all measurements.

Method	$F_{\rm pos}^{\rm noop}$	$F_{\rm scl}^{\rm noop}$	$F_{\rm app}^{\rm noop}$
ISA	34.62	62.54	69.51
SlotAug	33.56	63.54	77.83
GeoCo-SAVi	37.02	63.54	80.04

Appendix

Table 14: Obj3D no-op F1 (%): unchanged recipient masks scored against the requested edit targets, on the same valid edits without boundary filtering.

Method	$F_{\rm pos}\uparrow$	$F_{\rm scl}\uparrow$
ISA	88.65	63.85
SlotAug	91.98	82.62
GeoCo-SAVi	93.77	92.93

Appendix

Table 15: Obj3D command adherence on common reference object–command pairs. F1 is in percent; each column uses the same pairs for all methods.

	Full image			Editing region
Method	Position $\uparrow$	Scale $\uparrow$	Transplant $\uparrow$	Position $\uparrow$	Scale $\uparrow$	Transplant $\uparrow$
ISA	37.322	35.368	34.606	27.675	24.349	23.327
SlotAug	38.654	38.528	34.951	29.720	28.924	24.047
GeoCo-SAVi	39.914	40.176	35.728	30.883	30.889	24.428

Appendix

Table 16: Obj3D PSNR against independent RGB oracle edits (dB). Position and scale each contain 36,960 edits; transplant contains 9,240 pairs. Target images are identical across methods.

	RGB MAE $\times 10^{3}$		Changed pixels (%)
Method	Position $\downarrow$	Scale $\downarrow$	Position $\downarrow$	Scale $\downarrow$
ISA	0.8902	0.7724	2.1480	2.0424
SlotAug	0.8296	0.7756	2.1646	1.9778
GeoCo-SAVi	0.4939	0.6873	1.2360	1.6966

Appendix

Table 17: Obj3D RGB changes outside the source/target editing region. MAE is multiplied by

10^{3}

; changed-pixel rates are percentages.

Updates	Frames / clip	Active terms	Purpose
0–15k	2	RGB reconstruction; $s_{\mathrm{ref}}$ continuation	Stable slot formation with carry-forward
15–17k	2	$\mathcal{L}_{\mathrm{pos}}$ , $\mathcal{L}_{\mathrm{ov}}$ ramp	Factual geometry and allocation calibration
17–48k	2	$\mathcal{L}_{\mathrm{pos}}$ , $\mathcal{L}_{\mathrm{ov}}$	Core-only stabilization
48–50k	2	$\mathcal{L}_{\mathrm{geo}}$ ramp	Counterfactual geometry activation
50–70k	2	Reconstruction, factual and counterfactual objectives	Appearance-transplantation calibration
70–72k	4	Zero-residual STATM local LR warm-up	Temporal-module activation

Appendix

Table 18: Reported GeoCo-SAVi MOVi-C curriculum. All stages share one optimizer, scheduler, and AMP-scaler trajectory.

Method / scope	$F_{\rm pos}$	$F_{\rm scl}$	$F_{\rm app}$	$E_{\Delta p}$	$D_{\rm app}$
STAITUS (all)	75.96	76.86	60.37	0.02442	0.08065
GeoCo-SAVi (all)	83.01	83.11	72.04	0.01045	0.04083
STAITUS (interior)	74.25	76.28	58.33	0.01745	0.05848
GeoCo-SAVi (interior)	87.86	85.85	76.02	0.00667	0.03335

Appendix

Table 19: MOVi-C editing results. F1 (%) uses valid edits in each scope; interior uses factual/ideal feasibility.

E_{\Delta p}

is a diagonal fraction and

D_{\rm app}

is grid L2.

Method / scope	$\beta_{r}$	$\beta_{A}$	$\beta_{r}$ def.	$\beta_{A}$ def.
STAITUS (all)	0.70923	1.12247	50738/51971	50964/51971
GeoCo-SAVi (all)	0.85099	1.37988	50631/51629	50916/51629
STAITUS (interior)	0.73223	1.21219	12042/12159	12056/12159
GeoCo-SAVi (interior)	0.88920	1.61780	16796/16866	16802/16866

Appendix

Table 20: MOVi-C five-point scale response. Slopes are means over defined curves; counts are defined/total.

Method / scope	$N_{\rm pos}$	$N_{\rm scl}$	$N_{\rm app}$	$E_{\Delta p}$ def.	$D_{\rm app}$ def.
STAITUS (all)	204961	201269	44284	205891/207884	42904/45177
GeoCo-SAVi (all)	198500	189003	41954	198106/206516	43175/43950
STAITUS (interior)	72936	68173	7181	72030/72936	6764/7181
GeoCo-SAVi (interior)	113245	103197	12059	112427/113245	11839/12059

Appendix

Table 21: MOVi-C scope-specific F1 denominators and defined/total center-error counts. F1 retains all actual outputs for valid edits, including empty outputs.

Method / scope	Position	Scale	Transplant
STAITUS (within-canvas)	906/72936 (1.24)	792/68173 (1.16)	417/7181 (5.81)
GeoCo-SAVi (within-canvas)	818/113245 (0.72)	559/103197 (0.54)	220/12059 (1.82)
STAITUS (all)	1494/206317 (0.72)	1227/207586 (0.59)	2263/45098 (5.02)
GeoCo-SAVi (all)	4172/201674 (2.07)	901/206414 (0.44)	760/43820 (1.73)

Appendix

Table 22: MOVi-C conditional output diagnostics, shown as count/eligible count (%). Disappearance counts empty outputs for positive-area ideal targets.

Method	$s$	$\bar{r}$	$\sigma_{r\mid g}$	$\mathrm{CV}_{r\mid g}$	$\sigma_{A\mid g}$	$N$
STAITUS	0.1	0.17068	0.16820	0.98550	0.11655	51962
STAITUS	0.2	0.24798	0.17560	0.70813	0.14393	51962
STAITUS	0.3	0.31203	0.18789	0.60217	0.17095	51962
GeoCo-SAVi	0.1	0.08325	0.02485	0.29851	0.00625	51600
GeoCo-SAVi	0.2	0.15475	0.05029	0.32496	0.01908	51600
GeoCo-SAVi	0.3	0.22067	0.08241	0.37347	0.04146	51600

Appendix

Table 23: MOVi-C pooled cross-appearance fixed-geometry statistics. Radius CV is

\sigma_{r\mid g}/\bar{r}

. GeoCo-SAVi has lower radius standard deviation and CV at all three scales.

N

counts all measurements.

Method	$F_{\rm pos}^{\rm noop}$	$F_{\rm scl}^{\rm noop}$	$F_{\rm app}^{\rm noop}$
STAITUS	48.06	64.07	59.21
GeoCo-SAVi	29.48	65.69	60.18

Appendix

Table 24: MOVi-C no-op F1 (%): unchanged recipient masks scored against the requested edit targets, on the same valid edits without boundary filtering.

Method	$F_{\rm pos}$	$F_{\rm scl}$	$F_{\rm app}$
STAITUS	75.34	72.14	45.15
	[73.46, 77.12]	[70.14, 74.01]	[39.71, 50.41]
GeoCo-SAVi	86.51	84.93	74.01
	[85.68, 87.31]	[84.03, 85.82]	[70.48, 77.26]
STAITUS no-op	54.60	61.77	47.20
GeoCo-SAVi no-op	40.99	64.60	58.39

Appendix

Table 25: Whole-video tube F1 (%) on 238 common recipient–donor video pairs, with all 24 frames retained. Brackets give 95% confidence intervals for the main-table F-scores using the resampling procedure in Sec. E.5 .

	F1 (%) $\uparrow$		Oracle PSNR: full image $\uparrow$			Oracle PSNR: editing region $\uparrow$
Method	$F_{\rm pos}$	$F_{\rm scl}$	Position	Scale	Transplant	Position	Scale	Transplant
STAITUS	77.03	73.50	20.444	20.587	19.000	17.202	17.796	15.234
GeoCo-SAVi	85.37	84.65	21.929	21.983	19.932	18.607	19.145	15.887

Appendix

Table 26: MOVi-C object-matched editing and RGB preservation. F1 and changed-pixel rates are percentages; oracle PSNR is in dB. Each block uses common object–command pairs across methods.

Dataset	Method	$N$	Linear $R^{2}\uparrow$	Unit-slope log $R^{2}\uparrow$	$\mathrm{Std}[\log(r/s)]\downarrow$
Obj3D	ISA	9424	0.6637	0.5078	0.2093
	SlotAug	9357	0.8129	0.6399	0.1551
	GeoCo-SAVi	9320	0.7924	0.6873	0.1394
MOVi-C	STAITUS	51971	0.6444	0.6346	0.3164
	GeoCo-SAVi	51629	0.8589	0.8346	0.2041

Appendix

Table 27: Factual scale calibration on Obj3D and MOVi-C.

N

counts valid factual frame–slot observations. The two

R^{2}

columns use their respective response spaces.

Dataset	Edit	Native	Extended	Prob. RMSE	F1	F1 bound	$N_{\mathrm{F1}}$
		Raw-logit RMSE (%), $N$		Extended-canvas final outputs
Obj3D	Translation	0.0325 (10324)	0.0199 (10324)	0.00422	95.87	95.56	10324
	Scaling	2.0376 (10324)	0.9230 (10324)	0.01049	95.69	95.17	10324
MOVi-C	Translation	0.0515 (704)	0.0446 (712)	0.00042	99.86	99.79	711
	Scaling	2.4478 (689)	1.4237 (712)	0.01136	96.95	95.16	712

Appendix

Table 28: Finite-grid covariance and final-output diagnostics. Raw-logit columns report relative RMSE in percent, with the number of interventions having at least 16 measurement pixels in parentheses. Native uses the receptive-field-valid domain; extended uses the extended computational canvas. Final-output columns use the extended canvas: probability RMSE averages all 10,324 interventions per family on Obj3D and 712 on MOVi-C; F1 and its lower bound are means in percent over the

N_{\mathrm{F1}}

target-membership comparisons with

|A|+|I|>0

Object-Centric Representation Learning

Object-Centric Representation Learning studies visual models that organize scenes into distinct objects and their properties, positions, and relations.

See all 94 papers

Geometry-Instructed Video Editing
Jun 23, 2026Chirui Chang, Xiaoyang Lyu, Yi-Hua Huang +7Video Editing Instruction-Based Video Editing
The University of Hong Kong, Hong Kong · Kuaishou Technology, China
Semantic Slots for Video Object-Centric Learning
Aug 21, 2026Khalil Sabri, Guillaume-Alexandre Bilodeau, Nicolas Saunier +1Video Object-Centric Learning Object-Centric Representation Learning
Polytechnique Montréal Montréal, Canada · Université TÉLUQ Montréal, Canada
ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inference
Aug 12, 2026Youze Huang, Penghui Ruan, Bojia Zi +3Physical Consistency in Video Generation Controllable Video Generation
University of Electronic Science and Technology of China · The Hong Kong Polytechnic University · The Chinese University of Hong Kong +2

Jun 23, 2026J/K move · Enter open · S save

Geometry-Instructed Video Editing

Chirui Chang, Xiaoyang Lyu, Yi-Hua Huang, Haoru Tan+6

The University of Hong Kong, Hong Kong · Kuaishou Technology, China

Video Editing Instruction-Based Video Editing

Open paper AskPDF arXiv

Object-level geometric edits, including translating, rotating, scaling, duplicating, or removing an object, are routine operations in digital content creation (DCC) workflows, yet they remain unreliable in generative video editing. The key challenge lies in specifying the target object's 3D state change unambiguously across viewpoint and time, while consistently updating geometry-dependent secondary effects such as shadows and reflections. We introduce GIVE, a geometry-instructed video editing framework that represents edits through a unified object-state formulation. Two video-aligned geometry streams describe the target object before and after editing: a depth-box encoding coarse 3D placement and extent, and an orientation-box providing an appearance-agnostic orientation cue. Together, these streams provide a compact pre/post geometric specification for object-state transitions. To provide paired supervision for learning these edits, we build a scalable graphics-engine pipeline that executes object-level edit programs and renders controlled before/after pairs, isolating the intended geometric edit while keeping secondary effects consistent with the transformation. Experimental results demonstrate that GIVE produces faithful geometric edits with temporal coherence and consistent secondary effects across operators in a unified framework, and shows promising transfer to in-the-wild videos. Project page: https://geometry-instructed-video-editing.github.io/give/

Abstract

Object-Centric Representation Learning

Explore similar work

Geometry-Instructed Video Editing

Semantic Slots for Video Object-Centric Learning

ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inference