SEP-YOLO: Fourier-Domain Feature Representation for Transparent Object Instance Segmentation

Fengming Zhang; Tao Yan; Jianchao Huang

cs.CVMar 3, 2026

SEP-YOLO: Fourier-Domain Feature Representation for Transparent Object Instance Segmentation

Authors: Fengming Zhang, Tao Yan, Jianchao Huang

Organizations: Jiangnan University, Wuxi, China

Abstract

Transparent object instance segmentation presents significant challenges in computer vision, due to the inherent properties of transparent objects, including boundary blur, low contrast, and high dependence on background context. Existing methods often fail as they depend on strong appearance cues and clear boundaries. To address these limitations, we propose SEP-YOLO, a novel framework that integrates a dual-domain collaborative mechanism for transparent object instance segmentation. Our method incorporates a Frequency Domain Detail Enhancement Module, which separates and enhances weak highfrequency boundary components via learnable complex weights. We further design a multi-scale spatial refinement stream, which consists of a Content-Aware Alignment Neck and a Multi-scale Gated Refinement Block, to ensure precise feature alignment and boundary localization in deep semantic features. We also provide high-quality instance-level annotations for the Trans10K dataset, filling the critical data gap in transparent object instance segmentation. Extensive experiments on the Trans10K and GVD datasets show that SEP-YOLO achieves state-of-the-art (SOTA) performance.

Figures & tables

Fig. 1 : SEP-YOLO framework based on YOLO11

Panel of Figure 2 — Fig. 2 : Architecture of the proposed modules: (a) FDDEM; (b) MS-GRB

Panel of Figure 3 — Fig. 3 : (a) LDConv; (b) DySample

Method	Venue	Box $mAP_{50}$		Box $mAP_{75}$		Mask $mAP_{50}$		Mask $mAP_{75}$		Params (M)	FPS
Method	Venue	Trans10k	GVD	Trans10k	GVD	Trans10k	GVD	Trans10k	GVD	Params (M)	FPS
Mask-RCNN [ 10 ]	ICCV 2017	0.773	0.797	0.691	0.704	0.771	0.791	0.687	0.679	44.5	27
Solov2 [ 24 ]	NeurIPS 2020	0.803	0.856	0.601	0.642	0.805	0.852	0.600	0.621	36.1	43
YOLOv10n [ 25 ]	NeurIPS 2024	0.794	0.812	0.689	0.725	0.798	0.804	0.691	0.701	2.64	87
YOLO11n	-	0.816	0.851	0.742	0.751	0.813	0.849	0.729	0.742	2.87	97
TrInSeg [ 17 ]	IROS 2024	0.810	0.851	0.739	0.753	0.805	0.847	0.737	0.739	55.6	21

TABLE I : Performance comparison of different methods on Trans10k and GVD datasets. The best, second best, and third best results are highlighted in red , blue , and green , respectively.

Panel of Figure 5 — Fig. 4 : Visual comparison of instance segmentation results on the GVD (first row) and Trans10K (second and third rows) datasets, respectively.

Method	Box		Mask
Method	$mAP_{50}$	$mAP_{75}$	$mAP_{50}$	$mAP_{75}$
Baseline(YOLO11)	0.816	0.742	0.813	0.729
+ FDDEM	0.836	0.755	0.833	0.741
+ FDDEM + MS-GRB	0.845	0.759	0.841	0.749
+ FDDEM + CA 2 -Neck	0.844	0.761	0.840	0.748
SEP-YOLO (ours)	0.852	0.774	0.851	0.754

TABLE II : Ablation study of SEP-YOLO components on the Trans10K dataset.

Method	Box		Mask
Method	$mAP_{50}$	$mAP_{75}$	$mAP_{50}$	$mAP_{75}$
Baseline(YOLO11)	0.851	0.761	0.849	0.742
+ FDDEM	0.864	0.773	0.861	0.769
+ FDDEM + MS-GRB	0.870	0.784	0.863	0.758
+ FDDEM + CA 2 -Neck	0.865	0.776	0.860	0.755
SEP-YOLO (ours)	0.882	0.796	0.872	0.769

TABLE III : Ablation study of SEP-YOLO components on the GVD dataset.

Instance Segmentation

Instance segmentation identifies each object and assigns it a separate pixel-level mask in images or video.

See all 159 papers

GHOST: Geometry-Guided Hallucination of Opaque Surface Textures
Jul 13, 2026Langxu Zhao, Zuan Gu, Tianhan GaoImage-to-Image Translation 3D Reconstruction
School of Software, Northeastern University, Shenyang, China.
From Transparent Labware Segmentation to Collision Avoidance: A Real-Time Edge-Aware Perception Pipeline
Aug 5, 2026Shijun Ding, Chen Qian, Weiwei Shang +1Image Segmentation Instance Segmentation
SeeClear: Reliable Transparent Object Depth Estimation via Generative Opacification
Mar 20, 2026Xiaoying Wang, Yumeng He, Jingkai Shi +4Depth Estimation Monocular Depth Estimation
University of California, Los Angeles. · University of Southern California. · University of Utah.

Jul 13, 2026J/K move · Enter open · S save

GHOST: Geometry-Guided Hallucination of Opaque Surface Textures

Langxu Zhao, Zuan Gu, Tianhan Gao

School of Software, Northeastern University, Shenyang, China.

Image-to-Image Translation 3D Reconstruction Depth Estimation

Open paper AskPDF arXiv

Transparent objects pose a fundamental challenge for depth estimation and 3D reconstruction due to their violation of Lambertian assumptions, leading to severe geometry degradation in downstream tasks. To address this, we propose a novel geometry-guided preprocessing framework GHOST that leverages visual foundation models to transform transparent regions into opaque, structurally consistent representations without requiring downstream model retraining. Specifically, our pipeline utilizes (1) TransDINO and (2) TransDecomp to disentangle masks and transparency physical properties, while (3) DAF-Net recovers surface normal priors to encode geometric curvature. Subsequently, (4) GeoSemTransNet integrates these multi-modal cues to synthesize a texture-rich opaque RGB image that preserves the transparent object's 3D structure. Extensive experiments demonstrate that our method significantly enhances the accuracy of state-of-the-art depth estimation and reconstruction models on transparent objects by restoring essential photometric cues.

Abstract

Instance Segmentation

Explore similar work

GHOST: Geometry-Guided Hallucination of Opaque Surface Textures

From Transparent Labware Segmentation to Collision Avoidance: A Real-Time Edge-Aware Perception Pipeline

SeeClear: Reliable Transparent Object Depth Estimation via Generative Opacification