Learning Class Difficulty via Dynamic Focal Attention for Histopathology Segmentation

Lakmali Nadeesha Kumari Rathukohe Mudiyanselage; Sen-Ching Samson Cheung

eess.IVApr 15, 2026

Learning Class Difficulty via Dynamic Focal Attention for Histopathology Segmentation

Authors: Lakmali Nadeesha Kumari Rathukohe Mudiyanselage, Sen-Ching Samson Cheung

Organizations: University of Kentucky, Lexington, KY 40506, USA

Abstract

Frequency-based loss reweighting, the standard remedy for imbalanced histopathology segmentation, implicitly assumes that rare classes are difficult. Yet difficulty also arises from morphological variability, boundary ambiguity, and contextual similarity, all largely orthogonal to class frequency. We propose Dynamic Focal Attention (DFA), a simple, efficient mechanism that learns class-specific difficulty directly within the cross-attention of query-based mask decoders. DFA adds a learnable per-class bias to the attention logits, reweighting representations before prediction rather than gradients after it. Initialised from a centred log-frequency prior to prevent gradient starvation and then optimised end-to-end, the bias adapts to difficulty signals as they emerge during training, unifying frequency- and difficulty-aware reweighting in a single attention-bias framework. On three benchmarks (BCSS, BDSA, CRAG), DFA consistently improves Dice and IoU, matching or exceeding a two-stage difficulty-aware baseline without a separate estimator or extra training stage. This shows that encoding difficulty at the representation level is a principled alternative to conventional loss reweighting.

Figures & tables

Figure 1: DFA integrated into cross-attention. A learnable class-specific bias $b_{c}$ is added to attention logits before softmax, initialised from a log-frequency prior and optimised end-to-end via $\mathcal{L}$ (Eq. 7 ). Here $\otimes$ denotes matrix multiplication and $\oplus$ broadcast addition of $\mathbf{b}\!\in\!\mathbb{R}^{C}$ across $N$ spatial positions.

Figure 2: Frequency–difficulty disconnect. Pixel frequency (hatched bars) vs. Dice of the focal-loss baseline (solid bars) per class across three datasets.

		Base w/o FL [ 22 ]		Base w/ FL [ 22 ]		CFFA [ 17 ]		HCFA (Ours)		DFA (Ours)
Dataset	Class	Dice	IoU	Dice	IoU	Dice	IoU	Dice	IoU	Dice	IoU
BCSS	Mean	0.7032	0.5535	0.7336	0.5882	0.7737	0.6382	0.7780	0.6434	0.7826	0.6488
	$\pm$ std	$\pm 0.003$	$\pm 0.004$	$\pm 0.003$	$\pm 0.003$	$\pm 0.002$	$\pm 0.003$	$\pm 0.002$	$\pm 0.003$	$\pm 0.002$	$\pm 0.002$
	Tumor	0.8298	0.7090	0.8410	0.7257	0.8662	0.7640	0.8674	0.7659	0.8684	0.7674
	Stroma	0.7521	0.6027	0.7816	0.6415	0.7968	0.6622	0.7944	0.6590	0.7969	0.6624
	Inflamm.	0.4974	0.3310	0.5537	0.3828	0.6228	0.4522	0.6392	0.4697	0.6493	0.4807

Table 1: Segmentation performance across three histopathology benchmarks. Each method occupies two columns: Dice (left) and IoU (right). Means and stds are computed over three independent training runs with different random seeds. Hard classes (baseline Dice below dataset mean) are shaded amber . Bold : best result per row.

Figure 3: Converged biases $\delta_{c}$ and difficulty correlation. Top (a–c): $\delta_{c}$ per class ( positive $\Leftrightarrow$ hard, negative $\Leftrightarrow$ easy). Bottom (d–f): $\delta_{c}$ vs. baseline Dice with fitted trend line; the strong negative correlation indicates $\delta_{c}$ tracks empirical difficulty rather than frequency.

Init	$\delta_{\text{Infl.}}$	$\delta_{\text{Oth.}}$	$\delta_{\text{Nec.}}$	$\delta_{\text{Str.}}$	$\delta_{\text{Tum.}}$	Dice ( $\pm$ std)
Cold ( $\delta_{c}^{(0)}\!=\!0$ )	$+0.010$	$+0.002$	$+0.001$	$-0.012$	$-0.019$	$0.7751_{\pm.005}$
Warm (Eq. 6 )	$+0.410$	$+0.253$	$+0.068$	$-0.308$	$-0.524$	$0.7826_{\pm.002}$

Table 2: Warm- vs. cold-start ablation on BCSS.

Panel of Figure 4 — Figure 4: Qualitative segmentation comparison across three pathology benchmarks. Each row shows a WSI patch with ground truth and predictions from all five methods. BCSS: x Inflamm. x Tumor x Stroma x Other x Necrosis CRAG: x Non-Gland x Gland BDSA: x BG x Gray M. x White M. x Leptom. x Superficial.

Histopathology Image Segmentation

Histopathology image segmentation identifies tissue regions, cells, and their boundaries in microscopic pathology images.

See all 53 papers

Difficulty-Aware Sample Allocation for Adaptive Data Augmentation in Semantic Segmentation
Aug 26, 2026Olasimbo Ayodeji Arigbabu, Abimbola Ismail ArigbabuData Augmentation Semantic Segmentation
Independent Researcher, Germany. · Department of Science and Technology Education, Olabisi Onabanjo University, Ago-Iwoye, Ogun State, Nigeria.
Instance Awareness of Multi-class Semantic Segmentation Loss Functions
Apr 27, 2026Soumya Snigdha Kundu, Florian Kofler, Marina Ivory +3Class-Imbalanced Learning Image Segmentation
1King’s College London · University of Tübingen · 3Technical University of Munich +1
ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation
Aug 12, 2026Duy-Dong Nguyen, Le-Van Thai, Hoai Nhan Pham +3Histopathology Image Segmentation Weakly Supervised Semantic Segmentation
AI VIETNAM Lab, Vietnam · Department of Mechanical System Engineering, Jeonbuk National University, Republic of Korea · Washington University School of Medicine, Saint Louis, MO, USA +1

Aug 26, 2026J/K move · Enter open · S save

Difficulty-Aware Sample Allocation for Adaptive Data Augmentation in Semantic Segmentation

Olasimbo Ayodeji Arigbabu, Abimbola Ismail Arigbabu

Independent Researcher, Germany. · Department of Science and Technology Education, Olabisi Onabanjo University, Ago-Iwoye, Ogun State, Nigeria.

Data Augmentation Semantic Segmentation

Open paper AskPDF arXiv

Data augmentation is a standard component of modern semantic segmentation pipelines, but most augmentation techniques allocate transformations uniformly across training samples or adapt to a single difficulty signal such as loss. This ignores the fact that segmentation difficulty is multi-factorial, since ambiguous predictions, persistent optimization errors, rare classes, and complex object boundaries can each make a sample informative in different ways. This paper introduces Difficulty-Aware Sample Allocation (DASA), an architecture-agnostic framework that assigns stronger augmentation to samples estimated to be more difficult. DASA combines prediction ambiguity, training loss, class rarity, and boundary complexity into a normalized difficulty score, then maps that score to sample-specific augmentation strength during iterative training. Experiments on Oxford-IIIT Pet and binary Pascal VOC segmentation with U-Net, DeepLabV3, and SegFormer-B0 show that DASA improves over standard training and is competitive with or stronger than single-signal adaptive baselines. On Oxford-IIIT Pet, DASA improves DeepLabV3 from 0.633 to 0.740 mIoU. On binary Pascal VOC, DASA obtains the best foreground IoU for all three evaluated architectures. These results attest to the value of multi-factor difficulty estimation as a practical mechanism for directing augmentation where it is most useful.

Abstract

Histopathology Image Segmentation

Explore similar work

Difficulty-Aware Sample Allocation for Adaptive Data Augmentation in Semantic Segmentation

Instance Awareness of Multi-class Semantic Segmentation Loss Functions

ProBAG: Prototype-Guided Boundary-Aware Graph Diffusion for Weakly Supervised Histopathology Segmentation