Amodal Segmentation

Latest papers

11 papers1 paper in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Sep 28, 2026

HyperDAM: Hyperspectral Distractor-Aware Memory with Amodal Expansion for SAM 3 Tracking

Hyperspectral video provides material cues that can disambiguate targets with similar false-color appearance, yet foundation-model trackers update memory primarily from spatial and appearance evidence. We present HyperDAM, a DAM4SAM3-based hyperspectral tracker with three principal contributions. First, HOTC2026-Modal adds human-verified frame-wise modal masks and mask-tight boxes to all 481 organizer-provided HOTC 2026 videos. Second, a frame-zero-calibrated HSI gate rejects spectrally inconsistent updates to the distractor-resolving memory (DRM) without altering the current prediction. Third, a causal spatiotemporal expander adds outward-only amodal corrections from frozen SAM features. Static-scene recovery and empty-mask RTS smoothing address target switches and full occlusion. Model selection prioritizes cross-domain robustness over leaderboard-specific optimization. The final system ranked second in HOTC 2026, achieving 68.0093% AUC and 87.7703% DP@20 in the organizer's private evaluation.

Ryoga Yuzawa, Tasuku Takagi

Visual Object Tracking Hyperspectral Imaging Amodal Segmentation

Ask PDF

Aug 5, 2026

Foreseeing the Invisible: Amodal Reconstruction of Leaf Fossil Images

Fossil leaves are rarely preserved whole -- sedimentary rock hides, breaks, and erodes the lamina, yet paleobotany depends on the complete shape and outline of the leaf. We cast the recovery of the missing tissue as amodal reconstruction and present AmodalDINO, a multi-head dense-prediction model that predicts four masks from a single RGB image: visible leaf, amodal complete leaf, amodal main vein, and fine veins. Unlike essentially all prior amodal work, AmodalDINO is given no visible mask. It predicts the visible and amodal regions jointly, so it needs no upstream instance segmenter at runtime. Two simple but effective changes adapt the model to the amodal segmentation task: fully fine-tune a DINOv3 ViT-L/16 at a small learning rate instead of freezing it, and attach auxiliary venation heads alongside the leaf heads. These two changes enable the model to learn the structural shape prior of leaves. Trained only on synthetic leaf fossil images, AmodalDINO reaches 95.0% Dice / 90.5% IoU on the validation set and transfers well to real fossil specimens. Stripped to two heads, the same recipe can run on two benchmark datasets, reaching 85.05 full mIoU / 66.65 occluded mIoU on KINS and 80.90 / 38.15 on COCOA-cls. The model is also practical: by quantizing to 4-bit weights, it runs entirely offline in a browser, matching the original model with an IoU of 0.910. We also add ruler-based calibration to estimate surface area, and a generative visualization of living leaves on local devices.

Liuxiang Yue, Ailin Zhang, Ziyue Zhao +1

Global College Shanghai Jiao Tong University Shanghai, China

Image Segmentation Image Reconstruction Vision Foundation Model Adaptation Amodal Segmentation

Ask PDF

May 23, 2026

Learnable Shape Prototypes with Occlusion-Geometry-Guided Injection for Amodal Instance Segmentation

Amodal instance segmentation aims to predict the complete object mask including occluded regions that lack pixel-level observations and must be inferred with the aid of shape priors. Existing methods acquire shape priors through fixed-capacity encoding spaces or expensive generative models, and inject them uniformly across all spatial positions without adapting to the varying prior demand between visible and occluded regions. In this paper, we propose a gated reliability-adaptive shape prior framework, which introduces a shape prior memory module that combines learnable prototypes via cross-attention to produce instance-adaptive shape priors through weighted prototype combination rather than generation. A spatial adaptive reliability gate then employs the signed distance field of the visible mask to modulate injection intensity at each position according to its occlusion depth, preserving reliable features in visible regions while directing shape compensation toward occluded areas. Experiments on two mainstream amodal instance segmentation benchmarks demonstrate that the proposed method outperforms existing approaches under multiple evaluation settings, improving the mean intersection-over-union over occluded regions by over 11 percentage points on one of the two benchmarks under the standard setting, while using approximately one-third of the total parameters. Linear probing analysis further reveals that the visible-mask cross-attention module implicitly encodes occlusion geometry into visual token representations, explaining the effectiveness of the proposed module decomposition.

Fufan Zhang, Jingxiang Wang, Xiangjie Ye

School of Mechanical Engineering and Automation, Northeastern University, Shenyang 110819, China · School of Information Science and Engineering, Northeastern University, Shenyang 110819, China

Representation Learning Occlusion Reasoning Instance Segmentation Prototype Learning Amodal Segmentation

Ask PDF

Apr 22, 2026

Amodal SAM: A Unified Amodal Segmentation Framework with Generalization

Amodal segmentation is a challenging task that aims to predict the complete geometric shape of objects, including their occluded regions. Although existing methods primarily focus on amodal segmentation within the training domain, these approaches often lack the generalization capacity to extend effectively to novel object categories and unseen contexts. This paper introduces Amodal SAM, a unified framework that leverages SAM (Segment Anything Model) for both amodal image and amodal video segmentation. Amodal SAM preserves the powerful generalization ability of SAM while extending its inherent capabilities to the amodal segmentation task. The improvements lie in three aspects: (1) a lightweight Spatial Completion Adapter that enables occluded region reconstruction, (2) a Target-Aware Occlusion Synthesis (TAOS) pipeline that addresses the scarcity of amodal annotations by generating diverse synthetic training data, and (3) novel learning objectives that enforce regional consistency and topological regularization. Extensive experiments demonstrate that Amodal SAM achieves state-of-the-art performance on standard benchmarks, while simultaneously exhibiting robust generalization to novel scenarios. We anticipate that this research will advance the field toward practical amodal segmentation systems capable of operating effectively in unconstrained real-world environments.

Bo Zhang, Zhuotao Tian, Xin Tao +3

Department of Computer Science, Harbin Institute of Technology at Shenzhen, Shenzhen 518057, China · Kuaishou Technology, Shenzhen, Guangdong, China

Vision Foundation Model Adaptation Amodal Segmentation

Ask PDF

Apr 22, 2026

MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimation

6D object pose estimation in cluttered scenes remains challenging due to severe occlusion and sensor noise. We propose MAPRPose, a two-stage framework that leverages mask-aware correspondences for pose proposal and amodal-driven Region-of-Interest (ROI) prediction for robust refinement. In the Mask-Aware Pose Proposal (MAPP) stage, we lift 2D correspondences into 3D space to establish reliable keypoint matches and generate geometrically consistent pose hypotheses based on correspondence-level scoring, from which the top-

K

candidates are selected. In the refinement stage, we introduce a tensorized render-and-compare pipeline integrated with an Amodal Mask Prediction and ROI Re-Alignment (AMPR) module. By reconstructing complete object geometry and dynamically adjusting the ROI, AMPR mitigates localization errors and spatial misalignment under heavy occlusion. Furthermore, our GPU-accelerated RGB-XYZ reprojection enables simultaneous refinement of all

N \times B

pose hypotheses in a single forward pass. Evaluated on the BOP benchmark, MAPRPose achieves a state-of-the-art Average Recall (AR) of 76.5%, outperforming FoundationPose by 3.1% AR while delivering a 43x speedup in multi-object inference.

Yang Luo, Yan Gong, Yongsheng Gao +2

aState Key Laboratory of Robotics and Systems, Harbin Institute of Technology, Harbin, 150001, China · bSchool of Civil Engineering, Harbin Institute of Technology, Harbin, 150001, China

Object Pose Estimation Amodal Segmentation

Ask PDF

Apr 13, 2026

Seeing Through the Tool: A Controlled Benchmark for Occlusion Robustness in Foundation Segmentation Models

Occlusion, where target structures are partially hidden by surgical instruments or overlapping tissues, remains a critical yet underexplored challenge for foundation segmentation models in clinical endoscopy. We introduce OccSAM-Bench, a benchmark designed to systematically evaluate SAM-family models under controlled, synthesized surgical occlusion. Our framework simulates two occlusion types (i.e., surgical tool overlay and cutout) across three calibrated severity levels on three public polyp datasets. We propose a novel three-region evaluation protocol that decomposes segmentation performance into full, visible-only, and invisible targets. This metric exposes behaviors that standard amodal evaluation obscures, revealing two distinct model archetypes: Occluder-Aware models (SAM, SAM 2, SAM 3, MedSAM3), which prioritize visible tissue delineation and reject instruments, and Occluder-Agnostic models (MedSAM, MedSAM2), which confidently predict into occluded regions. SAM-Med2D aligns with neither and underperforms across all conditions. Ultimately, our results demonstrate that occlusion robustness is not uniform across architectures, and model selection must be driven by specific clinical intent-whether prioritizing conservative visible-tissue segmentation or the amodal inference of hidden anatomy.

Nhan Ho, Luu Le, Thanh-Huy Nguyen +3

Stony Brook University, USA · AIMA Research Lab · Carnegie Mellon University, USA +2

Amodal Segmentation Image Segmentation Medical Imaging Foundation Models Polyp Segmentation

Ask PDF

Apr 13, 2026

AmodalSVG: Amodal Image Vectorization via Semantic Layer Peeling

We introduce AmodalSVG, a new framework for amodal image vectorization that produces semantically organized and geometrically complete SVG representations from natural images. Existing vectorization methods operate under a modal paradigm: tracing only visible pixels and disregarding occlusion. Consequently, the resulting SVGs are semantically entangled and geometrically incomplete, limiting SVG's structural editability. In contrast, AmodalSVG reconstructs full object geometries, including occluded regions, into independent, editable vector layers. To achieve this, AmodalSVG reformulates image vectorization as a two-stage framework, performing semantic decoupling and completion in the raster domain to produce amodally complete semantic layers, which are then independently vectorized. In the first stage, we introduce Semantic Layer Peeling (SLP), a VLM-guided strategy that progressively decomposes an image into semantically coherent layers. By hybrid inpainting, SLP recovers complete object appearances under occlusions, enabling explicit semantic decoupling. To vectorize these layers efficiently, we propose Adaptive Layered Vectorization (ALV), which dynamically modulates the primitive budget via an error-budget-driven adjustment mechanism. Extensive experiments demonstrate that AmodalSVG significantly outperforms prior methods in visual fidelity. Moreover, the resulting amodal layers enable object-level editing directly in the vector domain, capabilities not supported by existing vectorization approaches. Code will be released upon acceptance.

Juncheng Hu, Ziteng Xue, Guotao Liang +5

School of Software, Beihang University · Igarashi Lab, The University of Tokyo · Bambu Lab +1

Amodal Segmentation

Ask PDF

Mar 25, 2026

SemLayer: Semantic-aware Generative Segmentation and Layer Construction for Abstract Icons

Graphic icons are a cornerstone of modern design workflows, yet they are often distributed as flattened single-path or compound-path graphics, where the original semantic layering is lost. This absence of semantic decomposition hinders downstream tasks such as editing, restyling, and animation. We formalize this problem as semantic layer construction for flattened vector art and introduce SemLayer, a visual generation empowered pipeline that restores editable layered structures. Given an abstract icon, SemLayer first generates a chromatically differentiated representation in which distinct semantic components become visually separable. To recover the complete geometry of each part, including occluded regions, we then perform a semantic completion step that reconstructs coherent object-level shapes. Finally, the recovered parts are assembled into a layered vector representation with inferred occlusion relationships. Extensive qualitative comparisons and quantitative evaluations demonstrate the effectiveness of SemLayer, enabling editing workflows previously inapplicable to flattened vector graphics and establishing semantic layer reconstruction as a practical and valuable task. Project page: https://xxuhaiyang.github.io/SemLayer/

Haiyang Xu, Ronghuan Wu, Li-Yi Wei +5

UC San Diego · Adobe Research · City University of Hong Kong +1

Amodal Segmentation Semantic Segmentation

Ask PDF

Feb 27, 2026

GDA-YOLO11: Amodal Instance Segmentation for Occlusion-Robust Robotic Fruit Harvesting

Occlusion remains a critical challenge in robotic fruit harvesting, as undetected or inaccurately localised fruits often results in substantial crop losses. To mitigate this issue, we propose a harvesting framework using a new amodal segmentation model, GDA-YOLO11, which incorporates architectural improvements and an updated asymmetric mask loss. The proposed model is trained on a modified version of a public citrus dataset and evaluated on both the base dataset and occlusion-sensitive subsets with varying occlusion levels. Within the framework, full fruit masks, including invisible regions, are inferred by GDA-YOLO11, and picking points are subsequently estimated using the Euclidean distance transform. These points are then projected into 3D coordinates for robotic harvesting execution. Experiments were conducted using real citrus fruits in a controlled environment simulating occlusion scenarios. Notably, to the best of our knowledge, this study provides the first practical demonstration of amodal instance segmentation in robotic fruit harvesting. GDA-YOLO11 achieves a precision of 0.844, recall of 0.846, mAP@50 of 0.914, and mAP@50:95 of 0.636, outperforming YOLO11n by 5.1%, 1.3%, and 1.0% in precision, mAP@50, and mAP@50:95, respectively. The framework attains harvesting success rates of 92.59%, 85.18%, 48.14%, and 22.22% at zero to high occlusion levels, improving success by 3.5% under medium and high occlusion. These findings demonstrate that GDA-YOLO11 enhances occlusion robust segmentation and streamlines perception-to-action integration, paving the way for more reliable autonomous systems in agriculture.

Caner Beldek, Emre Sariyildiz, Son Lam Phung +1

School of Engineering, University of Wollongong, 2522, NSW, Australia

Amodal Segmentation Instance Segmentation Agricultural Robotics

Ask PDF

Aug 3, 2025

Shape Distribution Matters: Shape-Specific Mixture-of-Experts for Amodal Segmentation Under Diverse Occlusions

Amodal segmentation aims to predict complete object masks covering both visible and occluded regions, serving as a critical foundation for robust multimedia scene understanding and interactive media editing. This task poses significant challenges in complex visual media due to severe occlusions and extreme shape variation, ranging from rigid furniture to highly deformable clothing. Existing one-size-fits-all approaches struggle to reason about diverse amodal shapes in multimedia content due to limited representation capacity. While a mixture-of-experts framework is a natural solution, naively applying it without considering the underlying shape distribution leads to mismatched routing and redundant experts. To address these issues, we introduce ShapeMoE, a shape-specific sparse mixture-of-experts framework designed for complex visual scene parsing. The key idea is to learn a latent shape distribution space and dynamically route each object to a lightweight expert tailored to its geometric characteristics. Specifically, ShapeMoE encodes each object into a compact Gaussian embedding that captures key shape features. A Shape-Aware Sparse Router then maps the object to the most suitable expert, enabling precise shape-aware routing. Each expert is lightweight and specializes in recovering occluded regions for specific shape patterns. ShapeMoE is designed for shape-specific expert specialization while maintaining computational efficiency. Extensive experiments on three challenging benchmarks demonstrate that ShapeMoE consistently outperforms state-of-the-art methods. Beyond advancing fundamental visual reasoning, ShapeMoE provides a robust and efficient visual prior for downstream multimedia applications, particularly in occlusion-heavy immersive environments.

Zhixuan Li, Yujia Liu, Chen Hui +3

College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science, Peking University, Beijing, China · National Key Laboratory for Multimedia Information Processing, Peking University, Beijing, China +3

Amodal Segmentation Occlusion Reasoning Mixture-of-Experts Models Sparse Mixture-of-Experts

Ask PDF

Aug 3, 2025

Single Point, Full Mask: Velocity-Guided Level Set Evolution for End-to-End Amodal Segmentation

Amodal segmentation aims to recover complete object shapes, including occluded regions, serving as an essential technique for user-centric multimedia authoring and object-level visual manipulation. Existing methods typically rely on informative prompts, such as bounding boxes or dense visible masks, which heavily degrade the user experience and interaction efficiency in real-world multimedia applications. While recent interactive paradigms (e.g., the Segment Anything Model) support lightweight point-based interactions, they often perform direct mask regression. Crucially, the opaque nature of these direct-regression models offers no visual explainability regarding how occluded structures are inferred, conflicting with the growing demand for interpretable multimedia systems. To address these limitations, we propose VELA, an end-to-end VElocity-driven Level-set Amodal segmentation method that enables explicit and transparent contour evolution driven by simple point clicks. VELA constructs an initial level set function from visual features and the user's point input, which then progressively evolves into the final amodal mask under the guidance of a shape-specific motion field predicted by a fully differentiable network. This mechanism learns to generate evolution dynamics at each step, ensuring that the spatial reasoning process is geometrically grounded, topologically flexible, and visually explainable to the user. Extensive experiments on COCOA-cls, D2SA, and KINS benchmarks demonstrate that VELA outperforms existing methods that use bounding-box or dense visible-mask prompts while requiring only a single-point prompt, validating the effectiveness of explainable geometric modeling for interactive multimedia tasks.

Zhixuan Li, Yujia Liu, Chen Hui +2

Promptable Image Segmentation Amodal Segmentation

Ask PDF