cs.CVNov 9, 2026

What Breaks Scribble-Supervised Segmentation? An Audit-Informed Benchmark for Scribble Annotation Robustness

Authors: Minheng Wu, Di Wen, Ruiping Liu, Junwei Zheng, Yufan Chen, Kailun Yang, Kunyu Peng

Organizations: Karlsruhe Institute of Technology, Kaiserstr. 12, Karlsruhe, 76131, Baden-Württemberg, Germany. · Hunan University, No. 2 Lushan South Road, Changsha, 410082, Hunan, China.

Abstract

Scribble-supervised segmentation is typically evaluated with a fixed annotation set, leaving robustness to distinct semantic and geometric imperfections poorly understood. Motivated by an audit of 12,031 manual PASCAL Visual Object Classes (VOC) 2012 scribbles and prior work, we introduce six controlled perturbation operators. Class corruption, spatial offset, and directional perturbation directly operationalize audit signals. Stroke-length reduction is motivated by prior scribble studies, while random dropout and fragmentation serve as diagnostic controls. Each operator reports requested and achieved changes. We evaluate manual and automatic scribbles across two image domains, two principal backbone settings, and multiple training pipelines, with an additional MiT-B1 PFA anchor evaluation. In the five-seed Tree Energy Loss (TEL)-ResNet101 VOC study, mean intersection over union (mIoU) drops from 70.08 to 1.46 under class corruption, compared with 35.57 for spatial offset, 52.20 for directional perturbation, and 63.24 for stroke-length reduction at their strongest settings. At the evaluated endpoints, semantic-anchor corruption produces the largest degradation, while geometric and support responses depend on the pipeline and dataset. Our benchmark provides a reproducible protocol for diagnosing annotation robustness beyond clean-set accuracy. The operator code is available at https://github.com/Wmhiram99/scribble-operator-benchmark.

Figures & tables

Explore similar work

CardsList
  1. ZScribbleSeg: A comprehensive segmentation framework with modeling of efficient annotation and maximization of scribble supervision

    May 7, 2026Ke Zhang, Bomin Wang, Hangqi Zhou +1Image SegmentationWeakly Supervised Image Segmentation

  2. Bi-Level Collaborative Learning for Few-Shot Scribble-Supervised Medical Image Segmentation

    Jul 28, 2026Xiang-Xiang Su, Yufan Ye, Yihang Zheng +2Weakly Supervised Image SegmentationSemi-Supervised Medical Image Segmentation

  3. SCISSR: Scribble-Conditioned Interactive Surgical Segmentation and Refinement

    Mar 19, 2026Haonan Ping, Jian Jiang, Cheng Yuan +3Interactive Medical Image SegmentationMedical Image Segmentation