ROCOCO: Robust Outline Completion of Occluded Curvilinear Objects

Ken Zheng; Jaimyn Drake

cs.CVOct 21, 2026

ROCOCO: Robust Outline Completion of Occluded Curvilinear Objects

Authors: Ken Zheng, Jaimyn Drake

Organizations: Electrical Engineering & Computer Sciences (EECS), University of California, Berkeley

Abstract

We propose a learning-based generative model for completing curvilinear features under occlusion by environmental obstacles. To this end, we adopt a flow matching-based approach that supports the inpainting of occluded regions with the appropriate curvilinear features. To eliminate the reliance on privileged information about the occlusion a priori, we also train a UNet-based mask prediction model, which can identify regions for inpainting given only the occluded observation. The components of our approach are validated via experiments using procedurally-generated test cases, and the complete pipeline is validated using large public datasets for curvilinear objects. Against off-the-shelf inpainting baselines, ROCOCO is the strongest method by a wide margin when no occlusion mask is given, and is competitive with mask-supervised inpainting when one is, achieving best on full-image error and second only to LaMa on the occluded region, while using a completer of $7.7$ M parameters against LaMa's $51$ M and Stable Diffusion's $860$ M.

Figures & tables

Model	MSE $\downarrow$	ML1L $\downarrow$	MSE ${}_{M}\downarrow$	ML1L ${}_{M}\downarrow$	SSIM ${}_{M}\uparrow$
Naive UNet	0.0026	0.0170	0.0548	0.1354	0.764
Pure Flow	0.0028	0.0061	0.0731	0.1666	0.708
Flow + GT mask	0.0018	0.0036	0.0398	0.0892	0.851
Flow Blind (full pipeline)	0.0019	0.0038	0.0427	0.0922	0.844

Table 1: Ablation on procedurally generated data, averaged over

500

freshly sampled test cases. Because samples are generated at run time, no test case is seen during training. Subscript

M

denotes the occluded region.

Panel of Figure 1 — Figure 1: Qualitative comparison on the wire dataset for all baselines. Columns show: Ground Truth, Occluded Input, Occlusion Mask, SD-Inpaint, SD-Blind, LaMa, ROCOCO-GT, ROCOCO-Blind.

Dataset	MSE $\downarrow$	ML1L $\downarrow$	MSE ${}_{M}\downarrow$	ML1L ${}_{M}\downarrow$	SSIM ${}_{M}\uparrow$
Wires ( Zanella et al., 2021 )	0.0024	0.0064	0.0576	0.1670	0.655
Cracks	0.0022	0.0060	0.0696	0.1931	0.547

Table 2: ROCOCO (blind) on the held-out splits of both realistic datasets.

Figure 2: Blind completions on held-out images from both datasets: the model receives only the occluded input, and must localize the occluder and reconstruct the curvilinear structure behind it without being told where it is.

Method	Mask	Params	MSE $\downarrow$	ML1L $\downarrow$	MSE ${}_{M}\downarrow$	ML1L ${}_{M}\downarrow$	SSIM ${}_{M}\uparrow$
SD-Blind	no	860M	0.0474	0.1407	0.4416	0.5411	0.258
SD-Inpaint	GT	860M	0.0041	0.0199	0.0895	0.1923	0.634
LaMa	GT	51M	0.0037	0.0297	0.0560	0.1468	0.695
ROCOCO	GT	7.7 M	0.0032	0.0073	0.0782	0.1955	0.596
ROCOCO ( $n{=}8$ )	GT	7.7 M	0.0022	0.0060	0.0540	0.1615	0.664
ROCOCO	no	7.7 M	0.0035	0.0078	0.0831	0.2012	0.586

Table 3: Comparison against off-the-shelf inpainting on the held-out wire split (

62

images). “Mask” indicates whether the method is given the ground-truth occlusion mask; ROCOCO’s blind rows estimate it from the observation alone. Subscript

M

denotes the occluded region.

Figure 3: Completion quality against occluder size on the held-out wire split. Degradation is graceful: the occluded-region error rises by about a third as the occluded area grows more than tenfold.

Figure 4: Inference budget. Quality peaks at

T\approx 2

integration steps rather than the tens typically used for iterative samplers, and averaging independent completions is a more productive use of the same compute.

Figure 5: Example Generation Result: Inpainting with Ground Truth Mask

Figure 6: Example Generation Result: Inpainting with Mask Prediction

Figure 7: Example Generation Result: Pure Flow Model

Panel of Figure 8 — Figure 8: Example Generations: ROCOCO on cracks and wires

Figure 9: Example Baseline Comparison Result on the Wires Dataset

Figure 11: Training Losses, Mask Prediction Model

Figure 12: Training Losses, Flow Matching Inpainting Model

Image Inpainting

Image inpainting reconstructs missing or corrupted image regions using surrounding context, learned visual priors, and optional structural or reference guidance.

See all 92 papers

SERO: Semantic Prediction for Single-Step Object Removal
Oct 16, 2026Amreen Hasheem Siraj, Alireza Javanmardi, Pragati Jaiswal +2Object Removal Image Inpainting
Deutsches Forschungszentrum für Künstliche Intelligenz, Kaiserslautern, Germany
Integrating Multimodal Large Language Model Knowledge into Amodal Completion
Mar 30, 2026Heecheol Yun, Eunho YangImage Inpainting
KAIST · AITRICS
Observation-Aligned Mask Priors for Learning Physical Fields from Authentic Occlusions
May 16, 2026Chiyuan Ma, Zihan Zhou, Tianshu YuIncomplete Data Imputation Masked Image Modeling
School of Data Science, The Chinese University of Hong Kong, Shenzhen · Shanghai Artificial Intelligence Laboratory

Oct 16, 2026J/K move · Enter open · S save

SERO: Semantic Prediction for Single-Step Object Removal

Amreen Hasheem Siraj, Alireza Javanmardi, Pragati Jaiswal, Didier Stricker+1

Deutsches Forschungszentrum für Künstliche Intelligenz, Kaiserslautern, Germany

Object Removal Image Inpainting

Open paper AskPDF arXiv

Object removal and semantic inpainting of missing regions remain inherently ill-posed, as multiple plausible completions may exist for a given scene. Recent diffusion-based methods have demonstrated strong potential for high-quality object removal; however, they still face two key limitations. First, although they can synthesise sharp and high-resolution images, they often struggle to complete removed regions in a semantically consistent and contextually meaningful manner, where the generated content should agree with the surrounding scene layout and object relationships rather than merely appear visually realistic. Second, their iterative denoising process makes them computationally expensive, limiting their practicality for downstream applications. We propose SERO, a semantic prediction framework for single-step object removal. Our model predicts a hidden-region semantic prior in embedding space, which guides a latent bridge-matching generator toward plausible scene completion by providing both where to edit and what structure is likely to exist behind the removed object. This enables visually realistic and contextually coherent completions in one forward pass. Experiments on the RORD and OBER datasets show that SERO achieves competitive or state-of-the-art performance while requiring only a single generation step.

Abstract

Image Inpainting

Explore similar work

SERO: Semantic Prediction for Single-Step Object Removal

Integrating Multimodal Large Language Model Knowledge into Amodal Completion

Observation-Aligned Mask Priors for Learning Physical Fields from Authentic Occlusions