Learning Class Difficulty via Dynamic Focal Attention for Histopathology Segmentation

Lakmali Nadeesha Kumari Rathukohe Mudiyanselage; Sen-Ching Samson Cheung

eess.IVApr 15, 2026

Learning Class Difficulty via Dynamic Focal Attention for Histopathology Segmentation

Authors: Lakmali Nadeesha Kumari Rathukohe Mudiyanselage, Sen-Ching Samson Cheung

Organizations: University of Kentucky, Lexington, KY 40506, USA

Abstract

Frequency-based loss reweighting, the standard remedy for imbalanced histopathology segmentation, implicitly assumes that rare classes are difficult. Yet difficulty also arises from morphological variability, boundary ambiguity, and contextual similarity, all largely orthogonal to class frequency. We propose Dynamic Focal Attention (DFA), a simple, efficient mechanism that learns class-specific difficulty directly within the cross-attention of query-based mask decoders. DFA adds a learnable per-class bias to the attention logits, reweighting representations before prediction rather than gradients after it. Initialised from a centred log-frequency prior to prevent gradient starvation and then optimised end-to-end, the bias adapts to difficulty signals as they emerge during training, unifying frequency- and difficulty-aware reweighting in a single attention-bias framework. On three benchmarks (BCSS, BDSA, CRAG), DFA consistently improves Dice and IoU, matching or exceeding a two-stage difficulty-aware baseline without a separate estimator or extra training stage. This shows that encoding difficulty at the representation level is a principled alternative to conventional loss reweighting.

Figures & tables

Figure 1: DFA integrated into cross-attention. A learnable class-specific bias $b_{c}$ is added to attention logits before softmax, initialised from a log-frequency prior and optimised end-to-end via $\mathcal{L}$ (Eq. 7 ). Here $\otimes$ denotes matrix multiplication and $\oplus$ broadcast addition of $\mathbf{b}\!\in\!\mathbb{R}^{C}$ across $N$ spatial positions.

Figure 2: Frequency–difficulty disconnect. Pixel frequency (hatched bars) vs. Dice of the focal-loss baseline (solid bars) per class across three datasets.

		Base w/o FL [ 22 ]		Base w/ FL [ 22 ]		CFFA [ 17 ]		HCFA (Ours)		DFA (Ours)
Dataset	Class	Dice	IoU	Dice	IoU	Dice	IoU	Dice	IoU	Dice	IoU
BCSS	Mean	0.7032	0.5535	0.7336	0.5882	0.7737	0.6382	0.7780	0.6434	0.7826	0.6488
	$\pm$ std	$\pm 0.003$	$\pm 0.004$	$\pm 0.003$	$\pm 0.003$	$\pm 0.002$	$\pm 0.003$	$\pm 0.002$	$\pm 0.003$	$\pm 0.002$	$\pm 0.002$
	Tumor	0.8298	0.7090	0.8410	0.7257	0.8662	0.7640	0.8674	0.7659	0.8684	0.7674
	Stroma	0.7521	0.6027	0.7816	0.6415	0.7968	0.6622	0.7944	0.6590	0.7969	0.6624
	Inflamm.	0.4974	0.3310	0.5537	0.3828	0.6228	0.4522	0.6392	0.4697	0.6493	0.4807

Table 1: Segmentation performance across three histopathology benchmarks. Each method occupies two columns: Dice (left) and IoU (right). Means and stds are computed over three independent training runs with different random seeds. Hard classes (baseline Dice below dataset mean) are shaded amber . Bold : best result per row.

Figure 3: Converged biases $\delta_{c}$ and difficulty correlation. Top (a–c): $\delta_{c}$ per class ( positive $\Leftrightarrow$ hard, negative $\Leftrightarrow$ easy). Bottom (d–f): $\delta_{c}$ vs. baseline Dice with fitted trend line; the strong negative correlation indicates $\delta_{c}$ tracks empirical difficulty rather than frequency.

Init	$\delta_{\text{Infl.}}$	$\delta_{\text{Oth.}}$	$\delta_{\text{Nec.}}$	$\delta_{\text{Str.}}$	$\delta_{\text{Tum.}}$	Dice ( $\pm$ std)
Cold ( $\delta_{c}^{(0)}\!=\!0$ )	$+0.010$	$+0.002$	$+0.001$	$-0.012$	$-0.019$	$0.7751_{\pm.005}$
Warm (Eq. 6 )	$+0.410$	$+0.253$	$+0.068$	$-0.308$	$-0.524$	$0.7826_{\pm.002}$

Table 2: Warm- vs. cold-start ablation on BCSS.

Panel of Figure 4 — Figure 4: Qualitative segmentation comparison across three pathology benchmarks. Each row shows a WSI patch with ground truth and predictions from all five methods. BCSS: x Inflamm. x Tumor x Stroma x Other x Necrosis CRAG: x Non-Gland x Gland BDSA: x BG x Gray M. x White M. x Leptom. x Superficial.

Histopathology Image Segmentation

Histopathology image segmentation identifies tissue regions, cells, and their boundaries in microscopic pathology images.

See all 53 papers

Aug 26, 2026

Difficulty-Aware Sample Allocation for Adaptive Data Augmentation in Semantic Segmentation

Data augmentation is a standard component of modern semantic segmentation pipelines, but most augmentation techniques allocate transformations uniformly across training samples or adapt to a single difficulty signal such as loss. This ignores the fact that segmentation difficulty is multi-factorial, since ambiguous predictions, persistent optimization errors, rare classes, and complex object boundaries can each make a sample informative in different ways. This paper introduces Difficulty-Aware Sample Allocation (DASA), an architecture-agnostic framework that assigns stronger augmentation to samples estimated to be more difficult. DASA combines prediction ambiguity, training loss, class rarity, and boundary complexity into a normalized difficulty score, then maps that score to sample-specific augmentation strength during iterative training. Experiments on Oxford-IIIT Pet and binary Pascal VOC segmentation with U-Net, DeepLabV3, and SegFormer-B0 show that DASA improves over standard training and is competitive with or stronger than single-signal adaptive baselines. On Oxford-IIIT Pet, DASA improves DeepLabV3 from 0.633 to 0.740 mIoU. On binary Pascal VOC, DASA obtains the best foreground IoU for all three evaluated architectures. These results attest to the value of multi-factor difficulty estimation as a practical mechanism for directing augmentation where it is most useful.

Olasimbo Ayodeji Arigbabu, Abimbola Ismail Arigbabu

Independent Researcher, Germany. · Department of Science and Technology Education, Olabisi Onabanjo University, Ago-Iwoye, Ogun State, Nigeria.

Data Augmentation Semantic Segmentation

Ask PDF

Apr 27, 2026

Instance Awareness of Multi-class Semantic Segmentation Loss Functions

Instance-sensitive losses for semantic segmentation such as blob loss and CC loss were designed to address instance imbalance, ensuring small lesions generate the same gradient as large ones, but operate only on single-class segmentation. In multi-class settings, class imbalance poses an additional problem: rare classes with few instances receive a disproportionately small share of the training signal. We show that extending instance-sensitive losses to multi-class segmentation via a one-vs-rest class decomposition repurposes them to also address class imbalance, as uniform averaging over classes ensures each class contributes equally regardless of frequency. We further show that inverse-size weighting, which destabilizes training when applied globally due to weight imbalances across rare and common classes, becomes effective when integrated within the per-component loss, confining the reweighting to each component's spatial context. On the BraTS-METS 2025 dataset (260 test cases), multi-class CC loss improves foreground Dice (0.64 +/- 0.26 vs. 0.59 +/- 0.27 baseline) and rare-class Dice, while maintaining Panoptic Quality at DSC threshold 0.5. Multi-class blob loss achieves the best Panoptic Quality at threshold 0.5 (0.40 +/- 0.24 vs. 0.38 +/- 0.25 baseline) and recognition quality (0.53 +/- 0.29 vs. 0.49 +/- 0.30). Integrating inverse-size weighting within the per-component loss increases rare-class Dice to 0.44 +/- 0.36 at the cost of reduced detection quality.

Soumya Snigdha Kundu, Florian Kofler, Marina Ivory +3

1King’s College London · University of Tübingen · 3Technical University of Munich +1

Class-Imbalanced Learning Image Segmentation Medical Image Segmentation 3D Medical Image Segmentation Semantic Segmentation

Ask PDF

Aug 12, 2026

ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation

Weakly supervised semantic segmentation enables histopathology tissue segmentation from image-level annotations, avoiding costly pixel-level labeling by expert pathologists. However, CAM-based methods often localize only highly discriminative regions and remain unreliable near tissue interfaces. We propose ProBAG, a stage-1 pseudo-mask generator that combines dataset-specific visual prototypes with pathology-aligned CONCH text prototypes over multi-scale frozen UNI features. ProBAG introduces two complementary mechanisms: class-wise power recalibration that reshapes inter-class competition while preserving the total foreground activation mass at each pixel, and one-step graph diffusion in which feature affinities are penalized by a late-transformer attention-context discrepancy used as a soft structural boundary cue. The resulting stage-1 pseudo-masks require neither CRF nor an external segmentation model; for complete two-stage comparison, they additionally supervise a downstream Phikon-FPN segmenter. Experiments on BCSS-WSSS and LUAD-HistoSeg show consistent gains over recent WSSS approaches, while ablations indicate that pathology-aligned text semantics provide the largest improvement and graph refinement provides a smaller complementary gain. The code is available at: https://github.com/wterrr/WSSS

Duy-Dong Nguyen, Le-Van Thai, Hoai Nhan Pham +3

AI VIETNAM Lab, Vietnam · Department of Mechanical System Engineering, Jeonbuk National University, Republic of Korea · Washington University School of Medicine, Saint Louis, MO, USA +1

Histopathology Image Segmentation Weakly Supervised Semantic Segmentation Boundary-Aware Segmentation Semantic Segmentation Graph Diffusion

Ask PDF

Abstract

Histopathology Image Segmentation

Explore similar work

Difficulty-Aware Sample Allocation for Adaptive Data Augmentation in Semantic Segmentation

Instance Awareness of Multi-class Semantic Segmentation Loss Functions

ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation