Anatomy-Change-Aware Bidirectional Selective State-Space Memory for Clinically Deployed Thoracic Radiotherapy Auto-Contouring

Galib Ahmed; Istiak Ahmed; Aritra Islam Saswato; Asib Mostakim Fony; Kazi Shahriar Sanjid; Md. Tanzim Hossain; Md. Anwarul Islam; Md. Nishan Khan

eess.IVSep 11, 2026

Anatomy-Change-Aware Bidirectional Selective State-Space Memory for Clinically Deployed Thoracic Radiotherapy Auto-Contouring

Authors: Galib Ahmed, Istiak Ahmed, Aritra Islam Saswato, Asib Mostakim Fony, Kazi Shahriar Sanjid, Md. Tanzim Hossain, Md. Anwarul Islam, Md. Nishan Khan,

Organizations: Department of Mathematics and Physics, North South University, Dhaka, 1229, Bangladesh · Big-Matrix Lab, Department of Mathematics and Physics, North South University, Dhaka, 1229, Bangladesh · Department of Electrical and Computer Engineering, North South University, Dhaka, 1229, Bangladesh · Department of Radiation Oncology, Dhaka, National Institute of Cancer Research & Hospital, 1212, Bangladesh · Department of Data Science, Erlangen, Friedrich-Alexander University, 91054, Germany · Square Cancer Centre, Square Hospitals Limited,, Dhaka 1205, Bangladesh · Department of Computer Science & Engineering,, Louisiana Tech University, 201 Mayfield Ave, Ruston, LA 71272, USA · Department of Radiation Oncology, Labaid Cancer Hospital & Super Speciality Center,, Dhaka 1205, Bangladesh · Department of Oncology & Radiotherapy, Bangladesh Specialized Hospital Limited,, Dhaka 1207, Bangladesh · Department of Clinical Oncology,, Bangladesh Medical University, Dhaka 1000, Bangladesh

Abstract

We developed DAMM-Net++, a 2.5D architecture for thoracic OAR and target volume segmentation that addresses three persistent challenges in radiotherapy auto-contouring: inter-slice surface incoherence, systematic failure on small low-contrast targets, and the absence of per-case reliability signals. The central component is an anatomy-change-aware bidirectional selective state-space memory that models through-plane anatomical change and selectively propagates context along the axial slice sequence. A boundary-aware decoder sharpens near-surface predictions, and an uncertainty head provides calibrated per-voxel confidence for clinical triage. We evaluated 2,146 patients across four centers, an independent external cohort of 112 patients, and a multicenter reader study involving 17 radiation oncologists on 305 cases. The model achieves a mean Dice of 0.955 and HD95 of 3.78 mm, with the largest gains on low-contrast organs-at-risk (OARs) and target volumes where through-plane context is most critical. The uncertainty head is well-calibrated and supports case-level triage. In the reader study, AI assistance reduced contouring time by 75-80% across experience levels and raised junior-reader IoU from 0.861 to 0.925, matching the unedited model. External validation showed a modest internal-to-external drop (< 5%) with calibrated uncertainty transferring without recalibration. The complete deployment pipeline from DICOM ingestion to TPS-compatible RTSTRUCT export has been integrated into the clinical workflow at a partner hospital, where it is used to assist with contouring. These results suggest that anatomically motivated inter-slice memory, paired with uncertainty-guided review, offers a clinically viable path for thoracic auto-contouring.

Figures & tables

Institution	Patients	Proportion (%)	Acquisition Years	Role
Internal Cohort
Bangladesh Medical University (BMU)	1,447	67.4	2020–2026	Internal
Square Hospital Limited (SHL)	309	14.4	2018–2026	Internal
Labaid Hospital (LH)	278	13.0	2019–2025	Internal
External Validation Cohort
United Hospital (UHL)	112	N/A	2026	External Validation

Table 1: Institutional distribution of the multicenter thoracic radiotherapy planning cohort. BMU, SHL, and LH comprised the internal cohort split 70/15/15; UHL was reserved exclusively for external validation.

Fig. 1 : Overview of the proposed DAMM-Net++ architecture. The input is a sequence of $T$ consecutive axial CT slices, each passed through a weight-shared ConvNeXt encoder. The Multi-Scale Inter-Slice Transition branch computes inter-slice feature differences; the Bidirectional Selective State-Space Memory accumulates anatomy-change-informed context in both through-plane directions; and the Memory-Guided Boundary-Aware Decoder reconstructs full-resolution predictions. The network produces segmentation, boundary, signed distance map, presence, and uncertainty outputs.

Fig. 2 : Detailed architecture of three core modules. (a) Multi-Scale Inter-Slice Transition Branch: multi-scale difference processing with channel and spatial attention. (b) Bidirectional Selective State-Space Memory: forward and backward through-plane sweeps with gated fusion. (c) Memory-Guided Cross-Attention: querying current features against memory context.

Model	Transition signal	Selective SSM	Dice $\uparrow$	$\Delta$ vs SSM only
Base	N/A	×	0.9047	N/A
SSM only	×	✓	0.9251	N/A
Difference + SSM	Real	✓	$\bm{0.9350}$	$\uparrow\,0.0099^{**}$
Randomized + SSM	Random	✓	0.9254	$\downarrow\,0.0003^{\text{ns}}$
Shuffled + SSM	Corrupted	✓	$\bm{0.9282}$	$\uparrow\,0.0031^{*}$
Full model	Real	✓	$\bm{0.9545}$	$\bm{\uparrow\,0.0294^{**}}$

Table 3: Controlled ablation isolating the inter-slice transition signal. All variants include the ConvNeXt encoder and MGBA decoder. The selective SSM is disabled in the Base condition. Blue and rose mark the best and second-best values, respectively; green arrows indicate improvement relative to the SSM-only baseline; red arrows indicate regression. Critical comparison: real transition vs random/corrupted controls.

Structure	Base	+Mem	Full	$\Delta$ (Full $-$ Base)
Spinal cord	0.8901	$\bm{0.9204}$	$\bm{0.9350}$	$\uparrow\,0.0449$
Esophagus	0.8210	$\bm{0.8788}$	$\bm{0.9058}$	$\uparrow\,0.0848$
Heart	0.9601	$\bm{0.9668}$	$\bm{0.9699}$	$\uparrow\,0.0098$
Lung (R)	0.9704	$\bm{0.9741}$	$\bm{0.9764}$	$\uparrow\,0.0060$
Lung (L)	0.9758	$\bm{0.9788}$	$\bm{0.9810}$	$\uparrow\,0.0052$
Trachea	0.8642	$\bm{0.9187}$	$\bm{0.9425}$	$\uparrow\,0.0783$

Table 4: Per-structure Dice. Base removes the inter-slice transition and memory modules; +Mem restores memory only; Full is DAMM-Net++. Gains concentrate on structures with high inter-slice variability. Blue and rose mark the best and second-best configuration per structure; green values show the Full-minus-Base improvement.

Variant	Dice $\uparrow$	ACE $\downarrow$	NLL $\downarrow$	AUROC $\uparrow$
Softmax-entropy proxy	0.9502	0.081	0.214	0.712
Variance head, std. loss	$\bm{0.9518}$	$\bm{0.052}$	$\bm{0.176}$	$\bm{0.784}$
Unc. head + UA loss (ours)	$\bm{0.9545}$	$\bm{0.031}$	$\bm{0.142}$	$\bm{0.861}$

Table 5: Uncertainty quality. Top : confidence calibration and error-detection performance. ACE is the absolute calibration error between predicted confidence and observed IoU. Bottom : Dice on the retained region as low-confidence voxels are routed to review. Blue and rose mark the best and second-best values; arrows quantify the gain over accepting all voxels.

Fig. 3 : Per-patient DSC distributions (

\uparrow

higher is better) for DAMM-Net++ and four baselines across nine thoracic structures. Paired violin plots show individual trajectories, box-and-whisker summaries, and paired Wilcoxon brackets (red:

p<0.05

; gray: ns) with Hedges’

g

annotations. DAMM-Net++ achieves significantly higher DSC on eight of nine structures; Heart is the sole exception.

Struc- ture	Metric	Ours	nnUNet	nnMamba	UNETR++	Swin- UNETR
SC	DSC	$\bm{0.935\pm 0.014}$	$0.912\pm 0.015^{\\|}$	$0.917\pm 0.012^{\\|}$	$0.918\pm 0.010^{\\|}$	$0.914\pm 0.013^{\\|}$
	IoU	$\bm{0.878\pm 0.024}$	$0.841\pm 0.022^{\\|}$	$0.847\pm 0.021^{\\|}$	$0.846\pm 0.018^{\\|}$	$0.843\pm 0.021^{\\|}$
	HD95	$\bm{1.585\pm 0.504}$	$7.523\pm 8.50^{\\|}$	$7.721\pm 9.00^{\\|}$	$7.234\pm 8.20^{\\|}$	$7.698\pm 8.80^{\\|}$
	ASD	$\bm{0.671\pm 0.217}$	$6.321\pm 8.00^{\\|}$	$6.497\pm 8.50^{\\|}$	$5.912\pm 7.80^{\\|}$	$6.654\pm 8.40^{\\|}$
	NSD	$\bm{0.995\pm 0.007}$	$0.977\pm 0.036^{\\|}$	$0.978\pm 0.035^{\\|}$	$0.979\pm 0.035^{\\|}$	$0.976\pm 0.037^{\\|}$
ESO	DSC	$\bm{0.906\pm 0.020}$	$0.908\pm 0.016$	$0.903\pm 0.017^{\ddagger}$	$0.902\pm 0.018^{\ddagger}$	$0.907\pm 0.016$

Table 6: Per-structure, per-metric comparison of five models on the multi-institutional thoracic dataset (background excluded). Values are mean

\pm

standard deviation over the test set; bold indicates the best result. Superscripts report paired Wilcoxon tests against DAMM-Net++, Holm-adjusted within each 36-comparison metric family:

{}^{\|}\,p_{\text{adj}}<0.001

{}^{\ddagger}\,p_{\text{adj}}<0.01

{}^{\dagger}\,p_{\text{adj}}<0.05

, and

{}^{\text{ns}}

not significant. Effect sizes use Hedges’

g

with BCa

95\%

CIs from

1{,}000

resamples. Structure abbreviations: SC, spinal cord; ESO, esophagus; HRT, heart; RL, right lung; LL, left lung; TRA, trachea; BDY, body; GTV, gross tumor volume; CTV, clinical target volume.

Fig. 4 : Through-plane consistency of the proposed model’s predictions on a representative thoracic case. Rows show axial (top), coronal (middle), and sagittal (bottom) planes reformatted from the same volumetric prediction. Columns show (1) original CT, (2) ground-truth overlay, and (3) DAMM-Net++ prediction overlay.

Fig. 5 : Contour overlay comparison on a representative GTV axial slice. Left: CT with ground truth and predictions; legend values report per-case IoU. Right: magnified boundary view. Color coding: Ground Truth (green), DAMM-Net++ (red), nnUNet (royal Blue), UNETR++ (yellow), SwinUNETR (magenta), nnMamba (cyan).

Fig. 6 : Voxel-level error maps for all five models. Rows: DAMM-Net++, nnUNet, UNETR++, SwinUNETR, nnMamba. Columns: (1) original CT, (2) ground truth, (3) prediction, (4) error overlay, (5) zoomed boundary error. Green, blue, and pink denote body, CTV, and GTV, respectively; red indicates prediction–reference disagreement.

Metric	Expert ( $n=5$ )	Senior ( $n=5$ )	Junior ( $n=7$ )	Model (unedited)	$p^{\dagger}$ Exp vs Jun	$p^{\dagger}$ Sen vs Jun	$p^{\dagger}$ Model vs Jun
IoU: No-AI
Mean $\pm$ SD	0.9490 $\pm$ 0.0058	0.9047 $\pm$ 0.0092	0.8606 $\pm$ 0.0133	N/A	$<$ 0.0001	$<$ 0.0001	N/A
Median [IQR]	0.9493 [0.9450–0.9530]	0.9046 [0.8981–0.9107]	0.8606 [0.8515–0.8702]	N/A
IoU: AI-Assisted
Mean $\pm$ SD	0.9702 $\pm$ 0.0038	0.9487 $\pm$ 0.0062	0.9245 $\pm$ 0.0091	0.9226 $\pm$ 0.0070	0.0021	$<$ 0.0001	$<$ 0.0001
Median [IQR]	0.9702 [0.9676–0.9727]	0.9486 [0.9447–0.9531]	0.9246 [0.9184–0.9306]	N/A

Table 7: Reader study summary. Values are mean

\pm

SD unless stated otherwise. IoU: Intersection over Union. Model (unedited): DAMM-Net++ predictions without reader editing. Expert

\geq

20 years (

n=5

), Senior 10–15 years (

n=5

), Junior

<

10 years (

n=7

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 796 papers

BAT-RM: A Boundary-Aware Transformer with Region-Aware Multi-Directional Mamba for Clinically Deployed Cervical Cancer Radiotherapy Auto-Contouring
Jul 11, 2026Istiak Ahmed, Kazi Shahriar Sanjid, Galib Ahmed +16Visual SSMs Medical Image Segmentation
Department of Electrical and Computer Engineering, North South University, Dhaka, 1229, Bangladesh · Big-Matrix Lab, Department of Mathematics and Physics, North South University, Dhaka, 1229, Bangladesh · Department of Data Science, Erlangen, Friedrich-Alexander University, 91054, Germany +8
Parameter-Efficient Adaptation of SAM 3 for Automated ITV Generation from 4DCT Images
Jun 14, 2026Changwoo SongVision Foundation Model Adaptation Medical Image Segmentation
Oncosoft Inc., Seoul, South Korea · Department of Computer Science & Engineering, Chungnam National University, Daejeon, South Korea
A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery
Aug 12, 2026Rafi Ibn Sultan, Chengyin Li, Yiannos Demetriou +6Medical Image Segmentation 3D Medical Image Segmentation
Department of Computer Science, Wayne State University, Detroit, MI, 48202, USA · Department of Radiation Oncology, Henry Ford Health, Detroit, MI, 48202, USA · Department of Clinical Oncology, Alexandria University, Alexandria, 21526, Egypt +4

Jul 11, 2026J/K move · Enter open · S save

BAT-RM: A Boundary-Aware Transformer with Region-Aware Multi-Directional Mamba for Clinically Deployed Cervical Cancer Radiotherapy Auto-Contouring

Istiak Ahmed, Kazi Shahriar Sanjid, Galib Ahmed, Md. Tanzim Hossain+15

Department of Electrical and Computer Engineering, North South University, Dhaka, 1229, Bangladesh · Big-Matrix Lab, Department of Mathematics and Physics, North South University, Dhaka, 1229, Bangladesh · Department of Data Science, Erlangen, Friedrich-Alexander University, 91054, Germany+8

Visual SSMs Medical Image Segmentation Oncology Boundary-Aware Segmentation

Open paper AskPDF arXiv

We present a clinically deployed end-to-end auto-contouring system for cervical cancer radiotherapy planning, anchored by the Boundary-Aware Transformer with Region-Aware Mamba (BAT-RM), a hybrid architecture that integrates Sobel-gated boundary attention, a linear-time, multi-directional Mamba module for long-range context, and a boundary-skeleton-guided fusion gate. This design achieves linear-time complexity for long-range context modeling, avoiding the quadratic cost of full spatial self-attention. The full pipeline spans multi-institutional data collection, rigorous inter-rater quality assurance, external validation in an independent cohort, and a web-based clinical interface natively compatible with Varian, RayStation, and Monaco. Against four baselines, BAT-RM achieves superior performance across seven anatomical classes, with statistically significant improvements in target volumes, including GTV and CTV, and in organs at risk such as the rectum and bladder. A prospective multi-center reader study involving 13 radiation oncologists demonstrated that AI assistance elevates junior oncologists' IoU from 0.899 to 0.965, approaching senior-level accuracy, while reducing contouring time by more than 80%. The system also reduced expert consultation rates and improved inter-reader consistency, reflecting gains in both efficiency and quality assurance. Following clinical deployment at a partner hospital, the system reduced patient wait times from days to hours without additional staffing, enabling same-day or next-day initiation of treatment for routine cases. BAT-RM demonstrates that a rigorous research pipeline, from data curation to clinical deployment, can translate directly into measurable patient benefit in resource-constrained settings where the demand for radiotherapy far exceeds specialist capacity.

Abstract

Medical Image Segmentation

Explore similar work

BAT-RM: A Boundary-Aware Transformer with Region-Aware Multi-Directional Mamba for Clinically Deployed Cervical Cancer Radiotherapy Auto-Contouring

Parameter-Efficient Adaptation of SAM 3 for Automated ITV Generation from 4DCT Images

A Neighborhood Attention Transformer Network for Enhanced 3D Segmentation of the Left Anterior Descending Artery