On the Relaxation of Conditional Independence Assumption for Image Segmentation

Zixun Wang; Ben Dai

cs.CVSep 30, 2026

On the Relaxation of Conditional Independence Assumption for Image Segmentation

Authors: Zixun Wang, Ben Dai

Organizations: Department of Statistics and Data Science The Chinese University of Hong Kong

Abstract

In semantic segmentation, a recent line of RankSEG methods directly optimizes Dice/IoU scores at inference time, improving alignment with evaluation metrics without modifying model training. Despite its theoretical and empirical success, RankSEG relies on the restrictive Conditional Independence Assumption (CIA), which ignores crucial label correlations and therefore degrades performance in ambiguous or low-contrast scenarios. However, accounting for full label dependence is computationally prohibitive, requiring $\mathcal{O}(d^3)$ time. To address this, we replace the CIA with a Spatially Localized Dependence (SLD) structure that captures local label correlations while keeping the dependence model tractable. We further overcome the remaining computational bottleneck via a Reciprocal Moment Approximation coupled with a novel fixed-point optimization strategy that eliminates exhaustive search. The proposed algorithm achieves a highly practical $\mathcal{O}(d \log d)$ complexity and consistently outperforms conventional argmax and CIA-based RankSEG across diverse segmentation benchmarks. Improvements are significant in low-contrast or small-object scenarios, where label dependence offers valuable signals complementary to image information for accurate segmentation. The code of experiments is available at https://github.com/ZixunWang/RankSEG-DEP.

Figures & tables

Panel of Figure 1 — Figure 1 : (a) The intensity contrast, and consequently the foreground probabilities, are uniformly low, so traditional thresholding barely detects any tumor and RankSEG almost entirely misses the tumor in the upper right. In contrast, our proposed method more fully recovers the tumor by leveraging label dependencies within the tumor region. (b) When predicting $Y_{j^{\prime}}$ , only information from $\bm{X}$ is used under CIA, whereas both $\bm{X}$ and the neighboring label $Y_{j}$ are utilized when the CIA is not assumed.

Figure 2 : Trade-off in dependence modeling. The two endpoints represent the extremes of CIA and Full dependence. By relaxing CIA to local dependence, SLD strikes a better trade-off. The graphs depict the label dependence (dep.) under each model: none under CIA, arbitrary under Full, and neighboring-only under SLD.

Model	Prediction	KiTS		LiTS
Model	Prediction	IoU	Dice	IoU	Dice
UNet	Argmax-prob	51.00	57.36	38.45	47.58
	CIA-RankSEG	53.54	60.07	40.70	50.07
	Ours	53.92	60.48	40.77	50.16
DeepLabV3+	Argmax-prob	54.19	61.16	38.34	47.38
	CIA-RankSEG	56.22	63.56	40.09	49.50

Table 1 : Comparison of performance (Dice and IoU %) in LiTS and KiTS.

Model	Prediction	ADE20K		Cityscapes		DeepGlobe
Model	Prediction	mIoU	mDice	mIoU	mDice	mIoU	mDice
PSPNet	Argmax-prob	51.32	58.66	73.07	80.45	57.24	66.23
	CIA-RankSEG	51.57	59.17	73.72	81.14	58.20	67.82
	Ours	51.92	59.49	73.84	81.27	58.47	68.14
DeepLabV3+	Argmax-prob	52.53	59.57	73.37	80.59	57.75	66.73
	CIA-RankSEG	52.64	59.95	73.92	81.24	58.38	67.94

Table 2 : Comparison of performance (mDice and mIoU %) in ADE20K, Cityscapes, and DeepGlobe.

Quantile	0.2	0.4	0.6
Argmax-prob	+14.37 %	+7.46 %	+3.97 %
CIA-RankSEG	+2.51 %	+1.04 %	+0.48 %

Table 3 : Relative Dice improvements over baselines on KiTS subsets, formed by the lower quantiles of Argmax-prob performance.

Group	small	medium	large
Argmax-prob	+5.47 %	+2.92 %	+2.01 %
CIA-RankSEG	+1.19 %	+0.82 %	+0.42 %

Table 4 : Relative Dice improvements over baselines on ADE20K across category groups, defined by average pixel area.

Panel of Figure 3 — Figure 3 : Our method consistently outperforms CIA-RankSEG across 10 retraining runs on (a) ADE20K, (b) Cityscapes, and (c) DeepGlobe, with paired $t$ -test $p$ -values $\ll 0.01$ confirming the statistical significance.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	LiTS	KiTS	ADE20K	Cityscapes	DeepGlobe
#Steps $<1.5$	63.36%	88.52%	9.30%	2.80%	40.49%
$1.5\leq$ #Steps $<2.5$	36.42%	11.41%	90.55%	97.20%	50.51%
$2.5\leq$ #Steps	0.22%	0.07%	0.15%	0.00%	0.00%

Appendix

Table 5 : Distribution of the number of iterations required for convergence across all test samples. On every dataset, the iteration terminates within

2.5

steps for the vast majority of samples, with only a negligible fraction requiring more.

Dataset	KiTS		LiTS
Metric	IoU	Dice	IoU	Dice
Fixed-point Optimization	56.70	64.07	40.48	49.88
Brute-force Search	56.73	64.10	40.48	49.88

Appendix

Table 6 : Comparison between the convergent point obtained by the fixed-point optimization and the global optimum obtained by brute-force search. The performance gap is at most

0.03

on both metrics, indicating that the iteration consistently recovers a near-optimal fixed point.

Method	CRF		Argmax-prob		CIA-RankSEG		Ours
Method	mIoU	mDice	mIoU	mDice	mIoU	mDice	mIoU	mDice
DeepGlobe	58.61	67.40	59.49	68.60	60.19	69.57	60.52	69.91
Cityscapes	73.26	80.35	75.66	82.61	76.17	83.21	76.25	83.31
ADE20K	56.05	62.61	56.94	63.98	57.67	64.92	57.94	65.19

Appendix

Table 7 : Performance comparison of CRF, Argmax-prob, CIA-RankSEG, and our method using UPerNet.

Figure 6 : Qualitative comparison of segmentation masks produced by different methods on LiTS.

Figure 7 : Qualitative comparison of segmentation masks produced by different methods on KiTS.

Figure 8 : Qualitative comparison on ADE20K. Each case spans two rows, where the first row shows the original image and the second row zooms in on the most distinctive region.

Figure 9 : Qualitative comparison on Cityscapes. Each case spans two rows, where the first row shows the original image and the second row zooms in on the most distinctive region.

Figure 10 : Qualitative comparison on DeepGlobe Land. Each case spans two rows, where the first row shows the original image and the second row zooms in on the most distinctive region.

Explore similar work

Aug 11, 2026cs.CV

SegPAR: Class-Centric Decision-Based Sparse Attack for Semantic Segmentation

Despite the practical relevance of sparse decision-based black-box threats, they have received limited attention in semantic segmentation. To bridge this gap, we adapt the most representative decision-based black-box sparse attacks from the classification domain to serve as baselines, establishing a rigorous benchmark for this underexplored setting. In this context, we demonstrate that one of the existing methods suffers from severe query inefficiency due to its image-centric pixel accumulation, which rapidly exhausts query budgets across the vast image space. To overcome this, we propose SegPAR, a novel decision-based framework that shifts to a class-centric exploration paradigm. Furthermore, to eliminate the misleading feedback generated by standard decision rewards during pixel accumulation, we introduce a novel discrepancy reward. Extensive experiments show that SegPAR significantly outperforms black-box baselines in sparsity efficiency and MIoU reduction, while remaining competitive with white-box sparse attacks. Code is available at https://github.com/KAU-QuantumAILab/SegPAR.

Dongsu Song, DaeYun GO, Boseung Seo +1

Interdisciplinary Program in Space Systems Engineering, Korea Aerospace University, South Korea · Department of Computer Science, Korea Aerospace University, South Korea · Department of Artificial Intelligence, Korea Aerospace University, South Korea

Semantic Segmentation Sparse Supervision White-Box Spectral-Subspace-Guided Attack

Ask PDF

Jul 22, 2026cs.CV

A Unified Variational Framework for Deep Weakly Supervised Image Segmentation

We propose a unified variational framework for image segmentation under sparse pixel-level supervision. Our method is based on a simplex-constrained Potts model with a smooth perimeter regularizer, yielding a convex, smooth energy functional that can be used as a training loss in weakly supervised deep learning paradigms or optimized efficiently using iterative methods. Sparse labels are incorporated into the data fidelity term by constructing a fuzzy membership function via a function extension problem in a Reproducing Kernel Hilbert Space (RKHS), which can effectively capture inhomogeneous intensity statistics. The derived discrete loss for training standard networks demonstrates robustness and consistent improvements over non-training and partial cross-entropy (PCE) baselines in experiments, achieving comparable performance without requiring ground-truth segmentation images.

Yin King Chu, Lingfeng Li, Sung Ha Kang +2

Department of Mathematics, The Hong Kong University of Science and Technology, Clear Water Bay, Kowloon, Hong Kong, China · Hetao Institute of Mathematics and Interdisciplinary Sciences, Shenzhen, Guangdong, China · School of Mathematics, Georgia Institute of Technology, Atlanta, GA, 30332, USA +2

Variational Spatial Photonic Ising Machines

Ask PDF

May 7, 2026cs.CV

Towards Fairness under Label Bias in Image Segmentation: Impact, Measurement and Mitigation

Labeled datasets reflect the biases of their annotation pipelines, which sometimes introduce label bias: group-conditional label errors that cause systematic performance disparities across demographic subgroups. Label bias in image segmentation remains underexplored, as even detecting it typically requires clean, unbiased annotations, which are not readily available. We present a data-centric adaptation of Confident Learning to segmentation, allowing detection of label bias directly in the training data without a clean, unbiased ground truth. By comparing the provided training labels to the model's confident predictions, we isolate directional errors that quantify the presence and nature of bias, where standard overlap metrics like Dice fail. We further show that label bias influences subgroup separability in the encoder's feature space, an artifact we leverage for bias mitigation rather than suppressing it. We evaluate three datasets, spanning from synthetic to real-life bias, showing how our framework reliably detects and mitigates bias without access to clean labels, achieving equitable performance across experimental conditions.

Aditya Parikh, Stella Frank, Sneha Das +1

Department of Applied Mathematics and Computer Science Technical University of Denmark

Biases Algorithmic Fairness

Ask PDF