OccJEPA: Occlusion as Masking for Amodal Video Segmentation

Genpei Zhang

cs.CVOct 21, 2026

OccJEPA: Occlusion as Masking for Amodal Video Segmentation

Authors: Genpei Zhang

Organizations: Department of Computer Sciences, University of Wisconsin–Madison

Abstract

Amodal video segmentation asks for the complete mask of an object in every frame, including the parts hidden behind other objects. The best current methods generate these masks with fine-tuned video diffusion models, which need many denoising steps and an extra depth network, and give a different answer for every sample. Self-supervised video world models of the JEPA family are trained on a closely related task: predicting the representation of masked space-time regions from the visible ones. We therefore treat occlusion as masking. The tokens that the object's visible mask does not cover are dropped, the pre-trained V-JEPA 2.1 predictor fills them in, and a light head decodes the observed and predicted features into amodal masks. The resulting model, OccJEPA (Occlusion-as-masking JEPA), is deterministic and 33.5x faster than the diffusion-based state of the art, and more accurate on SAIL-VOS, on TAO-Amodal, including a held-out test split (+4.00 AP50) and visible masks tracked by SAM2, and on Kubric. Controlled experiments trace the gain to the predictor's pre-trained weights rather than to its capacity, for world models from 87M to 1.85B parameters. Code is available at https://github.com/zgp110/occjepa.

Figures & tables

Figure 1: Background and concept of OccJEPA. Top: a video world model such as V-JEPA is pre-trained without labels: the encoder $E$ reads the visible tokens of a clip and the predictor $P$ predicts the features of masked space-time blocks. Bottom: amodal segmentation asks the same question for an occluded object. OccJEPA treats occlusion as masking: the tokens that the object’s visible mask does not cover are masked, the predictor with the same pre-trained weights fills them in, and a light head turns the result into the complete mask (magenta: completed hidden part), without depth or sampling. Real data of a SAIL-VOS sequence; feature colours: first three principal components; the top block mask is illustrative.

Figure 2: Overall framework of OccJEPA. The clip is cropped around the object’s visible mask. Tokens whose visible-mask coverage $\rho_{i}$ is below $\tau$ are dropped; the encoder $E$ reads the kept tokens and the pre-trained V-JEPA 2.1 predictor $P$ predicts features at the dropped positions, giving the filled features $f$ ( Eq. 4 ). The same encoder also reads the full clip ( $z$ ); it stays frozen except for one LoRA adapter (bow tie: a low-rank bottleneck) shared by both passes. The mask head $H$ decodes $[z;f]$ , together with the RGB crop and the visible mask, into amodal masks, which are pasted back into the frames; at test time, a second pass re-crops objects that leave the first crop. Real data of a SAIL-VOS validation sequence: feature maps show the first three principal components of $z$ and $f$ (in $f$ , the black outline encloses the kept tokens; everything outside it is predicted); magenta: completed hidden part.

		SAIL-VOS val		TAO-Amodal val
Method	Type	mIoU	mIoU ${}_{\text{occ}}$	AP 25	AP 50	AP 75
Visible mask	–	67.89	n/a	93.73	82.22	63.12
Convex hull	–	63.18	27.54	93.73	82.22	63.12
Convex hull (refined)	–	71.21	34.27	93.73	82.22	63.12
PCNet-M [ 51 ]	image	74.20	42.52	94.89	85.11	65.97
AISFormer [ 42 ]	image	73.51	39.16	95.45	81.93	59.84

Table 1: Comparison with the state of the art on SAIL-VOS and TAO-Amodal (Diffusion-VAS protocols). Rows other than ours are reported by [ 9 ] (CVPR version); ours is scored with the official metric, which our re-implementation reproduces to within 0.01 on the released Diffusion-VAS outputs (77.08 / 55.12 vs. the reported 77.07 / 55.12). mIoU

{}_{\text{occ}}

of the visible mask is undefined (its occluded part is empty); convex hulls have the same box as the visible mask on TAO-Amodal. Bold : best, underline : second best. Every method takes the ground-truth visible masks; all are zero-shot on TAO-Amodal.

	SAIL-VOS	TAO val		TAO test
Method	mIoU ${}_{\text{occ}}^{\text{fixed}}$	AP 50	AP 75	AP 50	AP 75
Visible mask	0.00	82.22	63.12	83.08	62.42
Diffusion-VAS [ 9 ]	39.98	89.25	71.99	88.99	71.05
OccJEPA (ours)	43.68	93.36	76.84	92.99	76.40
$\Delta$ vs. Diffusion-VAS	+3.70	+4.11	+4.85	+4.00	+5.35
	[+1.38, +5.40]	[+3.02, +5.21]	[+3.39, +6.34]	[+2.99, +5.03]	[+4.15, +6.56]

Table 2: Controlled comparison : every method scored by us on the same frames with the same code. mIoU

{}_{\text{occ}}^{\text{fixed}}

: occluded IoU on the fixed set of 83,245 SAIL-VOS frames with an occluded region, a miss counting as 0. TAO test (48,675 frames, 772 videos) was never used for any choice; Diffusion-VAS on test is its released checkpoint run by us.

\Delta

: 95% CI, bootstrap over videos.

Panel of Figure 3 — Figure 3: Qualitative results , two frames far apart per sequence. Top: SAIL-VOS, per-frame occluded IoU. Bottom: real TAO-Amodal videos, which annotate only amodal boxes (values: box IoU). Magenta: completed hidden part; grey: outside the image. More examples in the supplementary material.

MOVi-D
Method	mIoU	mIoU ${}_{\text{occ}}$
cropped
EoRaS+SG [ 12 ]	74.10	38.33
C2F-Seg [ 13 ]	71.67	36.13
Amodal SAM [ 52 ]	73.06	39.21
Diffusion-VAS [ 9 ]	77.03	44.23

Table 3: Synthetic benchmarks. MOVi rows of other methods are copied from their papers (EoRaS counting). ∗ OccJEPA is zero-shot on MOVi-B, whereas EoRaS and Diffusion-VAS are trained on it. Kubric: the upper block lists values reported with TACO’s scorer; the lower block scores both models with our implementation of the metric (Diffusion-VAS: released SAIL-VOS checkpoint), and comparisons are made within a block. One OccJEPA model is used for all benchmarks. Bold : best; underline : second best within a group.

Figure 4: What the predictor fills in. Features of a clip shown as colours (first three principal components; one projection per model and example, shared by $z$ and $f$ ). $z$ : the encoder on all tokens, so the hidden part of the object shows the occluder. $f$ ( Eq. 4 ): the encoder on the kept tokens only (black boxes) and the predictor everywhere else. Inside the ground-truth amodal outline, $f$ continues the colour of the visible object behind the occluder: the predictor completes the object in feature space. The released V-JEPA 2.1 predictor already does this in part, without any training on amodal labels; in OccJEPA the completed region follows the object’s true extent, which the mask head turns into the amodal mask. Top: the SAIL-VOS sequence with our largest gain over Diffusion-VAS; bottom: the sequence at the median gain.

Completion		SAIL-VOS			TAO val
module	RC	mIoU	mIoU ${}_{\text{occ}}$	mIoU ${}_{\text{occ}}^{\text{fixed}}$	AP 50	AP 75
none		75.81	42.39	35.13	88.80	68.14
random init		76.81	44.81	36.94	89.34	69.83
pre-trained		78.73	51.52	43.32	92.05	75.63
none	✓	75.18	42.26	35.15	89.43	68.18
random init	✓	76.60	45.09	37.30	90.17	70.32

Table 4: Ablation of the completion module (ViT-G; identical data, schedule, head and LoRA). Random init: same predictor architecture with V-JEPA’s official initialisation. RC: test-time re-crop. Every difference in the last three rows has a 95% CI above zero (bootstrap over videos; supplementary).

pre-trained module (V-JEPA: predictor; VideoMAE: decoder)
			with module		gain
World model	Params	TFLOPs	SV	AP 50	SV	AP 50
V-JEPA 2.1 ViT-B ∗	0.11B	1.4	36.45	90.13	+8.67	+1.95
V-JEPA 2.1 ViT-L ∗	0.33B	3.6	39.15	91.09	+8.42	+2.73
V-JEPA 2.1 ViT-g	1.08B	10.6	41.88	90.96	+8.45	+2.93
V-JEPA 2.1 ViT-G	1.91B	17.8	43.32	92.05	+8.19	+3.24

Table 5: Completion modules of video self-supervised models , same recipe (standard crop, one pass). SV: SAIL-VOS mIoU

{}_{\text{occ}}^{\text{fixed}}

; AP 50 : TAO-Amodal val. Gain: over the same encoder without the module. Params and TFLOPs per 24-frame clip, measured. ∗ Distilled from ViT-G. Every gain of a pre-trained module, and every difference between pre-trained and random-init modules, has a 95% CI above zero (supplementary).

Figure 5: Analysis. (a) SAIL-VOS mIoU

{}_{\text{occ}}^{\text{fixed}}

against measured inference FLOPs per clip (log scale): our model with four V-JEPA world models (one pass), the final model with re-crop, the VideoMAE variant, and Diffusion-VAS. (b) Occluded IoU on SAIL-VOS by ground-truth occlusion ratio. (c) TAO-Amodal val AP 50 by ground-truth visibility, for frames whose amodal box lies inside the image. Bands: 95% bootstrap CIs over videos.

Method	Params	TFLOPs	Time (s)	Extra input	Output
Diffusion-VAS	2.59B	903	11.77	depth	sampled
OccJEPA	1.91B	35.6	0.35	–	deterministic

Table 6: Inference cost per object and clip (Diffusion-VAS: 25 frames at

256{\times}512

, 25 denoising steps plus depth estimation; ours: 24 frames at

256{\times}256

, two passes with re-crop). FLOPs counted on the actual inference code; time on the same otherwise idle GPU, averaged over 50 objects.

Method	mIoU	mIoU ${}_{\text{occ}}$	kept %	mIoU ${}_{\text{occ}}^{\text{fixed}}$	our ${}_{\text{occ}}$
Visible mask	67.89	n/a	0.0	0.00	0.00
Convex hull	63.22	27.54	46.2	20.75	24.80
Diffusion-VAS	77.08	55.12	44.5	39.98	36.05
OccJEPA	78.60	52.09	51.4	43.68	43.94
$\Delta$ ours $-$ Diffusion-VAS	+1.52 [+0.73, +2.33]	$-$ 3.03 [ $-$ 5.26, $-$ 0.74]		+3.70 [+1.38, +5.40]	+7.89 [+6.09, +9.32]

Table S1: SAIL-VOS val, all metrics , every row scored by us with the official metric (the values reported by Diffusion-VAS, Tab. 1 of the main paper, differ by at most 0.04). Kept: share of frames that enter the official mIoU

{}_{\text{occ}}

. our

{}_{\text{occ}}

: un-eroded occluded IoU on frames whose occluded area is at least

\max(16\,\text{px},\,5\%)

of the amodal area, misses counted as 0. Robustness of

\Delta

mIoU

{}_{\text{occ}}^{\text{fixed}}

to the resampling unit: sequences [+2.85, +4.60], 23 game missions [+0.78, +5.61], 15 video-name prefixes [+0.31, +5.70].

	all		out of frame		in-frame occl.		vis $<$ 0.5		fully visible
Method	AP 50	AP 75	AP 50	AP 75	AP 50	AP 75	AP 50	AP 75	AP 50	AP 75
Diffusion-VAS (val)	89.25	71.99	82.85	55.12	87.66	68.40	37.12	11.57	97.94	94.09
OccJEPA (val)	93.36	76.84	87.39	57.72	93.33	77.15	65.68	26.85	99.66	96.49
Diffusion-VAS (test)	88.99	71.05	80.25	53.54	87.86	65.58	43.75	19.58	98.26	92.45
OccJEPA (test)	92.99	76.40	85.61	57.35	94.27	75.20	73.19	31.64	98.81	95.43
Diffusion-VAS (test, SAM2)	81.27	61.26	77.49	50.27	75.52	51.12	28.86	9.96	89.96	80.64

Table S2: TAO-Amodal by frame type (val / test frames: all 34,800 / 48,675, out of frame 11,124 / 16,200, in-frame occl. 13,068 / 15,300, vis

<

0.5 1,754 / 2,048, fully visible 10,608 / 17,175). Out of frame: the ground-truth amodal box leaves the image. SAM2: visible masks tracked by SAM2 instead of annotated ( Tab. S3 ).

	SAM2 visible masks			annotated visible masks
Method	AP 25	AP 50	AP 75	AP 25	AP 50	AP 75
Visible-mask box	87.94	75.65	53.22	93.67	83.08	62.42
Diffusion-VAS	91.07	81.27	61.26	96.70	88.99	71.05
OccJEPA, one pass	92.01	84.84	66.29	97.53	92.10	76.09
OccJEPA	92.72	85.59	66.85	98.09	92.99	76.40
$\Delta$ OccJEPA $-$ Diffusion-VAS	+1.65	+4.32	+5.60	+1.38	+4.00	+5.35

Table S3: TAO-Amodal test split with visible masks tracked by SAM2 (48,675 frames, 772 videos; Sec. B ). The same masks are the input of every method and enter the metric’s union with the visible box. Differences: 95% CIs over videos.

	SAIL-VOS val		TAO-Amodal val
Method	mIoU	mIoU ${}_{\text{occ}}$	AP 25	AP 50	AP 75
pix2gestalt, one sample	54.83	26.59	80.73	57.50	28.95
pix2gestalt, best of 3	60.79	33.76	91.80	71.19	38.80
Diffusion-VAS, one sample	77.07	55.12	97.28	89.25	71.99
Diffusion-VAS, best of 3	79.23	59.69	98.31	92.46	77.48
OccJEPA, deterministic	78.60	52.09	98.68	93.36	76.84

Table S4: Best of three samples , as reported by Diffusion-VAS: for each frame, the sample closest to the ground truth among three. Rows other than ours are reported by Diffusion-VAS.

Difference (ViT-G)	mIoU	mIoU ${}_{\text{occ}}$	mIoU ${}_{\text{occ}}^{\text{fixed}}$	AP 50	AP 75
pre-trained $-$ none	+2.92 [+2.30, +3.65]	+9.13 [+7.51, +10.88]	+8.19 [+6.84, +9.72]	+3.24 [+2.50, +4.03]	+7.49 [+6.28, +8.83]
random $-$ none	+1.00 [+0.72, +1.29]	+2.42 [+1.89, +2.96]	+1.81 [+1.18, +2.43]	+0.53 [+0.17, +0.92]	+1.70 [+1.10, +2.32]
pre-trained $-$ random	+1.92 [+1.48, +2.45]	+6.71 [+5.39, +8.28]	+6.38 [+5.41, +7.57]	+2.71 [+2.12, +3.35]	+5.79 [+4.83, +6.84]
pre-trained $-$ none, RC	+3.42 [+2.77, +4.14]	+9.83 [+8.24, +11.64]	+8.53 [+7.16, +10.15]	+3.93 [+3.00, +4.84]	+8.66 [+7.39, +10.09]
random $-$ none, RC	+1.42 [+1.06, +1.79]	+2.84 [+2.20, +3.49]	+2.15 [+1.38, +2.86]	+0.74 [+0.34, +1.13]	+2.14 [+1.50, +2.82]
pre-trained $-$ random, RC	+2.00 [+1.50, +2.61]	+6.99 [+5.58, +8.90]	+6.38 [+5.38, +7.74]	+3.19 [+2.35, +4.01]	+6.52 [+5.47, +7.68]

Table S5: Completion-module ablation with 95% CIs (SAIL-VOS: first three columns; TAO-Amodal val: last two). RC: with re-crop.

Difference (standard crop, one pass)	mIoU	mIoU ${}_{\text{occ}}^{\text{fixed}}$	AP 50	AP 75
V-JEPA ViT-B: predictor $-$ none	+2.80 [+2.16, +3.65]	+8.67 [+7.41, +10.09]	+1.95 [+1.28, +2.62]	+5.57 [+4.48, +6.82]
V-JEPA ViT-L: predictor $-$ none	+2.76 [+2.00, +3.70]	+8.42 [+6.89, +10.18]	+2.73 [+2.05, +3.46]	+6.32 [+5.24, +7.50]
V-JEPA ViT-g: predictor $-$ none	+2.70 [+2.06, +3.49]	+8.45 [+7.29, +9.74]	+2.93 [+2.15, +3.75]	+6.96 [+5.82, +8.27]
V-JEPA ViT-G: predictor $-$ none	+2.92 [+2.30, +3.65]	+8.19 [+6.84, +9.72]	+3.24 [+2.50, +4.03]	+7.49 [+6.28, +8.83]
VideoMAE ViT-L: decoder $-$ none	+3.40 [+2.72, +4.22]	+9.83 [+8.59, +11.15]	+2.51 [+1.76, +3.27]	+5.49 [+4.47, +6.48]
V-JEPA ViT-g: pre-trained $-$ random predictor	+1.58 [+1.06, +2.15]	+5.72 [+4.83, +6.82]	+2.28 [+1.61, +2.93]	+5.43 [+4.50, +6.49]

Table S6: World-model and module comparisons with 95% CIs (SAIL-VOS: mIoU and mIoU

{}_{\text{occ}}^{\text{fixed}}

; TAO-Amodal val: AP).

Figure S1 : Predictor ablation on three random SAIL-VOS sequences : the pre-trained predictor (ours), the same predictor trained from a random initialisation, no predictor, and the ground truth, at the first, middle and last of the selected frames. Values: per-frame occluded IoU.

Figure S2 : Feature visualisation on the random sequences of Fig. S1 , middle frame, as in Fig. 4 of the main paper: first three principal components of the full-clip features $z$ and of the filled features $f$ , for the released V-JEPA 2.1 and for OccJEPA (legend as in the main paper).

Setting	Model	E: mIoU	E: mIoU ${}_{\text{occ}}$	A: mIoU	A: mIoU ${}_{\text{occ}}$	D: mIoU ${}_{\text{occ}}$
MOVi-D, cropped	no predictor	74.99	39.85	63.46	33.72	46.90
MOVi-D, cropped	ViT-g world model	75.21	39.94	63.65	33.80	46.84
MOVi-D, cropped	OccJEPA	76.47	42.05	64.71	35.58	51.08
MOVi-D, whole frame	no predictor	81.94	48.17	77.28	48.70	51.53
MOVi-D, whole frame	ViT-g world model	81.73	47.91	76.81	48.19	51.28
MOVi-D, whole frame	OccJEPA	84.66	52.00	79.79	52.16	57.36

Table S7: MOVi under three frame-counting rules. E (primary): EoRaS counting over the annotated box frames. A: all frames with a non-empty amodal mask, including fully hidden ones. D: the Diffusion-VAS SAIL-VOS rule (eroded occluded part, misses dropped). Whole frame: with re-crop.

	Kubric-Static		Kubric-Dynamic
Model	mIoU	mIoU ${}_{\text{occ}}$	mIoU	mIoU ${}_{\text{occ}}$
Diffusion-VAS (run by us)	92.13	48.69	89.67	44.33
no predictor, + re-crop	91.38	44.83	91.19	45.73
ViT-g world model, + re-crop	92.03	47.54	91.92	49.53
OccJEPA, one pass	92.02	47.75	92.95	51.78
OccJEPA, + re-crop	93.31	49.54	93.23	52.04

Table S8: Kubric with the occluded-region metric (A counting, our re-implementation of the TACO scorer,

256{\times}256

Component added	SAIL-VOS mIoU ${}_{\text{occ}}^{\text{fixed}}$	TAO AP 50	TAO AP 75	TAO out-of-frame AP 75
high-resolution refiner, with predictor	+0.37 [+0.12, +0.63]	$-$ 0.11 [ $-$ 0.45, +0.28]	$-$ 0.29 [ $-$ 0.86, +0.34]	$-$ 2.02 [ $-$ 3.45, $-$ 0.66]
high-resolution refiner, no predictor	+1.92 [+1.43, +2.42]	$-$ 0.96 [ $-$ 1.44, $-$ 0.48]	$-$ 0.97 [ $-$ 1.71, $-$ 0.25]	$-$ 3.35 [ $-$ 5.22, $-$ 1.59]
MOVi-D in training	+0.04 [ $-$ 0.30, +0.34]	$-$ 0.36 [ $-$ 0.74, $-$ 0.01]	+0.05 [ $-$ 0.51, +0.63]	$-$ 0.78 [ $-$ 2.21, +0.66]
re-crop, with predictor	+0.47 [+0.26, +0.68]	+1.24 [+0.67, +1.84]	+0.66 [+0.16, +1.18]	+1.13 [ $-$ 0.03, +2.38]
re-crop, no predictor	+0.26 [ $-$ 0.03, +0.57]	$-$ 0.11 [ $-$ 0.68, +0.53]	$-$ 0.55 [ $-$ 1.09, +0.00]	$-$ 0.53 [ $-$ 1.58, +0.55]

Table S9: Design components (ViT-g world model; difference made by adding the component, 95% CIs; TAO-Amodal val). Out of frame: frames whose ground-truth amodal box leaves the image.

Model	Total	Module	Head	LoRA	Trained	TFLOPs
ViT-B ∗	0.111	22.9	1.08	0.59	24.5	1.38
ViT-L ∗	0.330	23.0	1.14	1.57	25.7	3.64
ViT-g	1.076	55.4	3.36	3.77	62.5	10.57
ViT-G (OccJEPA)	1.914	59.4	3.88	5.70	69.0	17.78
VideoMAE-L	0.345	39.1	0.85	1.57	41.6	3.79

Table S10: Cost of each model (V-JEPA world models unless marked). Parameters in billions (total) or millions (module: predictor or decoder; trained: module, LoRA and head); TFLOPs per 24-frame

256{\times}256

window, one pass (re-crop doubles it), counted with PyTorch’s FLOP counter on the inference code. Diffusion-VAS: U-Net 1.52B, VAE 0.098B, image encoder 0.63B, depth network 0.34B; 903 TFLOPs per 25-frame

256{\times}512

clip. ∗ Distilled from ViT-G.

Occluded	OccJEPA	Diff.-VAS	random-init	no pred.	frames
0–10%	30.9	43.3	24.0	22.7	13,880
10–20%	50.7	52.2	40.8	38.2	9,014
20–30%	52.6	45.1	42.8	39.1	7,759
30–40%	54.4	48.1	46.4	43.4	7,094
40–50%	51.8	44.9	45.8	42.8	7,419
50–60%	48.0	40.1	42.0	38.3	7,200

Table S11: SAIL-VOS occluded IoU by occlusion ratio (fixed frame set; random-init: predictor trained from a random initialisation; no pred.: no predictor; values behind Fig. 5 b).

Visibility	$<$ 0.25	0.25–0.5	0.5–0.75	0.75–1	1
OccJEPA	41.5	77.7	91.9	99.0	99.7
Diffusion-VAS	24.5	43.4	89.0	97.1	97.9
random-init predictor	44.8	74.8	90.0	98.9	99.3
no predictor	45.8	76.5	89.0	98.7	98.8
frames	583	1,171	2,210	9,104	10,608

Table S12: TAO-Amodal val AP 50 by visibility (frames with the amodal box inside the image; values behind Fig. 5 c).

Method	$\|$ IoU $(P_{t},P_{t+1})-$ IoU $(A_{t},A_{t+1})\|$	IoU $(P_{t},P_{t+1})$
OccJEPA	7.15 [5.98, 8.35]	74.5 [69.9, 78.7]
Diffusion-VAS	3.96 [3.21, 4.76]	77.0 [70.9, 82.3]
no predictor	6.52 [5.70, 7.37]	75.9 [71.0, 80.1]
ground truth	0	80.0

Table S13: Temporal stability on SAIL-VOS (all sequences, 24 consecutive pairs each,

\times 100

; 95% CIs over videos). Lower deviation means frame-to-frame changes closer to those of the ground truth.

Figure S3 : SAIL-VOS: our largest gain over Diffusion-VAS (top) and the sequence at the median gain (bottom) , middle frame. These are the sequences of the feature visualisation (Fig. 4 of the main paper). Values: occluded IoU of the frame.

Figure S4 : SAIL-VOS: three random draws and our largest gain (last block), at the first, middle and last of the selected frames. Values: per-frame occluded IoU.

Figure S5 : SAIL-VOS: our two largest losses against Diffusion-VAS. In both sequences the hidden part is a small fraction of the object, the regime of the lowest occlusion bin in Tab. S11 . Values: per-frame occluded IoU.

Figure S6 : TAO-Amodal: random draws with the object leaving the frame (left two) and with in-frame occlusion (right two), at the first, middle and last of the selected frames. Values: box IoU; each crop follows the object, and grey marks the area outside the image.

Figure S7 : Kubric-Static (left two) and Kubric-Dynamic (right two): random draws , at the first, middle and last of the selected frames. Diffusion-VAS is run by us with the same inputs. Values: per-frame occluded IoU.

Amodal Segmentation

See all 12 papers

Amodal SAM: A Unified Amodal Segmentation Framework with Generalization
Apr 22, 2026Bo Zhang, Zhuotao Tian, Xin Tao +3Vision Foundation Model Adaptation Amodal Segmentation
Department of Computer Science, Harbin Institute of Technology at Shenzhen, Shenzhen 518057, China · Kuaishou Technology, Shenzhen, Guangdong, China
Mask-supervised Object-centric Representation Learning with LeJEPA
Jul 2, 2026Jakob Geusen, Ender KonukogluContrastive Learning Representation Learning
Biomedical Image Computing Group, ETH Zurich
Latent Video Prediction for World Modeling: An Evaluation Uncovering Intriguing Favorable Evidence
May 15, 2026Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam +2Video Representation Learning Video Prediction
The University of Melbourne · Monash University

Apr 22, 2026J/K move · Enter open · S save

Amodal SAM: A Unified Amodal Segmentation Framework with Generalization

Bo Zhang, Zhuotao Tian, Xin Tao, Songlin Tang+2

Department of Computer Science, Harbin Institute of Technology at Shenzhen, Shenzhen 518057, China · Kuaishou Technology, Shenzhen, Guangdong, China

Vision Foundation Model Adaptation Amodal Segmentation

Open paper AskPDF arXiv

Amodal segmentation is a challenging task that aims to predict the complete geometric shape of objects, including their occluded regions. Although existing methods primarily focus on amodal segmentation within the training domain, these approaches often lack the generalization capacity to extend effectively to novel object categories and unseen contexts. This paper introduces Amodal SAM, a unified framework that leverages SAM (Segment Anything Model) for both amodal image and amodal video segmentation. Amodal SAM preserves the powerful generalization ability of SAM while extending its inherent capabilities to the amodal segmentation task. The improvements lie in three aspects: (1) a lightweight Spatial Completion Adapter that enables occluded region reconstruction, (2) a Target-Aware Occlusion Synthesis (TAOS) pipeline that addresses the scarcity of amodal annotations by generating diverse synthetic training data, and (3) novel learning objectives that enforce regional consistency and topological regularization. Extensive experiments demonstrate that Amodal SAM achieves state-of-the-art performance on standard benchmarks, while simultaneously exhibiting robust generalization to novel scenarios. We anticipate that this research will advance the field toward practical amodal segmentation systems capable of operating effectively in unconstrained real-world environments.

Abstract

Amodal Segmentation

Explore similar work

Amodal SAM: A Unified Amodal Segmentation Framework with Generalization

Mask-supervised Object-centric Representation Learning with LeJEPA

Latent Video Prediction for World Modeling: An Evaluation Uncovering Intriguing Favorable Evidence