Cross-Spectral Feature Fusion for Multispectral Document Binarization

Yasas Savinda; Maheshi Buddhinee Dissanayake

cs.CVOct 22, 2026

Cross-Spectral Feature Fusion for Multispectral Document Binarization

Authors: Yasas Savinda, Maheshi Buddhinee Dissanayake

Organizations: Department of Electrical and Electronic Engineering, University of Peradeniya, Peradeniya, Sri Lanka.

Abstract

Historical document binarization remains challenging when faded ink, bleed-through, and background degradation exhibit wavelength-dependent behavior across visible and near-infrared observations. This study investigates whether adaptive cross-spectral feature fusion and targeted structural guidance can improve multispectral document binarization beyond fixed fusion and generic attention mechanisms. Four registered SMADI bands at 500, 600, 700, and 1000 nm are processed using a multi-branch conditional generative adversarial network (cGAN). Visible, NIR, and structural feature streams are fused at encoder depth L = 3 using spatial- and channel-wise softmax gating, while auxiliary edge maps derived from selected VIS-NIR band pairs provide targeted structural guidance. Nine controlled fusion and structural configurations are evaluated using five-fold document-level cross-validation, DIBCO-standard metrics, and Holm-corrected statistical testing. The F3 + NIR edge-guided configuration provides the strongest overall performance among the evaluated models, achieving mean F1 = 0.7916, p-FM = 0.8044, and IoU = 0.6654. Relative to the fixed Mid AVG baseline, it improves F1 by 0.0269, p-FM by 0.0360, and IoU by 0.0321, with all three gains remaining statistically significant after Holm correction. Mean NRM and DRD are also reduced, although these reductions are not statistically significant after correction. Targeted cross-spectral structural guidance provides an effective mechanism for regulating intermediate multispectral fusion, improving primary foreground-overlap performance while retaining the complete four-band observation.

Figures & tables

No.	Configuration	Structural source	$\boldsymbol{\lambda_{\mathrm{edge}}}$	Primary experimental objective
1	Mid AVG	None	0	Fixed intermediate fusion baseline (equal-weight feature averaging).
2	Visible/NIR gated fusion	None	0	Evaluates unguided, learned modality-dependent gating.
3	Four-band edge-guided gate	$F2+F3+F4+F7$	0	Tests structural input conditioning without edge-loss supervision.
4	Four-band edge-guided gate	$F2+F3+F4+F7$	5	Evaluates four-band structural guidance with explicit edge supervision.
5	CBAM fusion	None	0	Baseline for domain-agnostic spatial and channel attention Woo et al. (2018) .
6	SE fusion	None	0	Baseline for domain-agnostic channel recalibration Hu et al. (2018) .

Table 1: Comparative architectural study (

L=3

, evaluated under 5-fold document-level cross-validation).

Architecture	F1 $\uparrow$	p-FM $\uparrow$	IoU $\uparrow$	NRM $\downarrow$	DRD $\downarrow$
Mid AVG ( $L=3$ )	$0.7647\pm 0.1232$	$0.7684\pm 0.1134$	$0.6333\pm 0.1486$	$0.1285\pm 0.0871$	$5.9281\pm 3.1901$
Visible/NIR Gated ( $L=3$ )	$0.7651\pm 0.1694$	$0.7663\pm 0.1630$	$0.6414\pm 0.1679$	$0.1162\pm 0.1017$	$6.0284\pm 3.7699$
Edge4Band Gate ( $\lambda_{\mathrm{edge}}=0$ )	$0.7616\pm 0.1416$	$0.7694\pm 0.1315$	$0.6330\pm 0.1641$	$0.1307\pm 0.0946$	$5.9912\pm 3.4993$
Edge4Band Gate ( $\lambda_{\mathrm{edge}}=5$ )	$0.7853\pm 0.1174$	$0.8021\pm 0.1029$	$0.6600\pm 0.1461$	$0.1227\pm 0.0855$	$\mathbf{5.2537\pm 2.7648}$
CBAM Fusion ( $L=3$ )	$0.7502\pm 0.1750$	$0.7557\pm 0.1626$	$0.6247\pm 0.1826$	$0.1318\pm 0.1110$	$6.0324\pm 3.3924$
SE Fusion ( $L=3$ )	$0.7652\pm 0.1647$	$0.7737\pm 0.1535$	$0.6420\pm 0.1751$	$0.1260\pm 0.1029$	$5.8053\pm 3.8226$

Table 2: Full-page binarization performance of the nine fusion architectures over 30 held-out SMADI documents. Values are reported as mean

\pm

standard deviation. Best values for each metric are shown in bold.

Structural configuration	F1 $\uparrow$	p-FM $\uparrow$	IoU $\uparrow$	NRM $\downarrow$	DRD $\downarrow$
Four-band, $\lambda_{\mathrm{edge}}=0$	0.7616	0.7694	0.6330	0.1307	5.9912
Four-band, $\lambda_{\mathrm{edge}}=5$	0.7853	0.8021	0.6600	0.1227	5.2537
$F2+F7$ , $\lambda_{\mathrm{edge}}=5$	0.7842	0.7935	0.6568	0.1064	6.3810
$F3+F7$ , $\lambda_{\mathrm{edge}}=5$	0.7916	0.8044	0.6654	0.1115	5.7007
$F4+F7$ , $\lambda_{\mathrm{edge}}=5$	0.7746	0.7866	0.6495	0.1259	5.4313

Table 3: Effect of structural-source selection and edge-aware supervision. All models retain the complete

F2

–

F3

–

F4

–

F7

multispectral input, where

F2

F3

, and

F4

denote the 500 nm, 600 nm, and 700 nm visible bands, respectively, and

F7

denotes the 1000 nm near-infrared (NIR) band. Only the source of the auxiliary structural representation and the use of the edge-consistency loss are varied.

Metric	Mid AVG	$F3+$ NIR	$\Delta$	Wins	$W$	$r_{\mathrm{rb}}$	$p$	$p_{\mathrm{Holm}}$
F1	0.7647	0.7916	+0.0269	24/30	89	0.617	0.00237	0.01894
p-FM	0.7684	0.8044	+0.0360	23/30	94	0.596	0.00348	0.02780
IoU	0.6333	0.6654	+0.0321	24/30	89	0.617	0.00237	0.01894
NRM	0.1285	0.1115	+0.0170	21/30	122	0.475	0.02210	0.15469
DRD	5.9281	5.7007	+0.2274	20/30	132	0.432	0.03842	0.29148

Table 4: Paired statistical comparison of the selected

F3+\text{NIR}

edge-guided model against the Mid AVG baseline across 30 held-out test documents. Differences (

\Delta

) and rank-biserial correlations (

r_{\mathrm{rb}}

) are oriented such that positive values indicate improvement.

Fold	Window size ( $w$ )	$k$
1	15	0.2
2	25	0.2
3	15	0.1
4	15	0.2
5	15	0.2

Table 5: Fold-specific Sauvola parameters optimized strictly on validation partitions.

Method	F1 $\uparrow$	IoU $\uparrow$	NRM $\downarrow$
Otsu	0.5400	0.4076	0.1175
Sauvola	0.7436	0.6087	0.1178
Mid AVG ( $L=3$ )	0.7647	0.6333	0.1285
$F3+$ NIR Edge Gate (Proposed)	0.7916	0.6654	0.1115

Table 6: Performance comparison with classical binarization methods across 30 held-out test document pages.

Overlap	F1 $\uparrow$	p-FM $\uparrow$	IoU $\uparrow$	NRM $\downarrow$	DRD $\downarrow$
0%	0.7617	0.7783	0.6275	0.1382	6.1510
10%	0.7885	0.7995	0.6617	0.1141	5.7555
20%	0.7916	0.8044	0.6654	0.1115	5.7007
30%	0.7957	0.8053	0.6717	0.1078	5.5863

Table 7: Reconstruction-overlap sensitivity of the selected

F3+

NIR edge-guided model. No model retraining is performed.

Measure	Value
Trainable parameters	56.31 M
FP32 parameter memory	214.82 MiB
Checkpoint size	214.90 MiB
Patch complexity (MACs)	20.68 GMACs
Patch complexity (FLOPs)	41.35 GFLOPs
Mean patch latency	8.73 ms

Table 8: Computational characteristics of the selected

F3+

NIR edge-guided generator. Runtime measurements were obtained on an NVIDIA Tesla T4 GPU.

Panel of Figure 1 — Figure 1: Qualitative comparison of baseline, gated, and channel-attention architectures on held-out documents ( $z27$ , $z35$ , and $z80$ ), using validation-selected thresholds.

Panel of Figure 2 — Figure 2: Qualitative comparison of channel-attention and wavelength-specific edge-guided models ( $F2$ – $F4+\text{NIR}$ ). The proposed $F3+$ NIR model is highlighted in bold.

Feature Fusion

Feature fusion combines complementary representations across modalities, scales, layers, or feature types to improve prediction and perception.

See all 119 papers

DR-Mamba: Automatic Inference-Time Domain Adaptation for Document Image Binarization via Sample-Conditioned Detail-Background Suppression
Jun 21, 2026Sheng-Wei Chan, Jen-Shiun ChiangMamba Test-Time Adaptation
Department of Electrical and Computer Engineering, Tamkang University, New Taipei City, Taiwan
DeepMine-Mamba: Mitigating Information Dilution in Mamba-Based State Space Models for Document Image Binarization
Jun 7, 2026Sheng-Wei Chan, Yung-Che Wang, Hsin-Jui Pan +2Mamba Document Image Understanding
Department of Electrical and Computer Engineering, Tamkang University, No.151, Yingzhuan Rd., Tamsui Dist., New Taipei City, 251301, Taiwan
Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging
Aug 28, 2026Eric L. Wisotzky, Jost Triller, Simon W. Härtl +3Image Matching Cross-Modal Feature Matching
Vision & Imaging Technologies, Fraunhofer Heinrich Hertz Institute HHI, Berlin, Germany · Visual Computing, Humboldt University Berlin, Germany · Functional Imaging in Surgical Oncology, National Center for Tumordiseases, Dresden, Germany

Jun 21, 2026J/K move · Enter open · S save

DR-Mamba: Automatic Inference-Time Domain Adaptation for Document Image Binarization via Sample-Conditioned Detail-Background Suppression

Sheng-Wei Chan, Jen-Shiun Chiang

Department of Electrical and Computer Engineering, Tamkang University, New Taipei City, Taiwan

Mamba Test-Time Adaptation Image Corruption Robustness Document Image Understanding

Open paper AskPDF arXiv

Degraded document image binarization is sensitive to domain shifts caused by paper aging, bleed-through, stains, shadows, and uneven illumination, and the foreground-background separation of recent learning-based methods can become unstable on unseen degradation domains. We propose DR-Mamba, a sample-conditioned detail-background suppression framework that performs automatic inference-time domain adaptation for document image binarization. Unlike test-time adaptation methods that require gradient updates or auxiliary data at inference, DR-Mamba adapts to each input document through input-dependent gates within a single forward pass, requiring no target-domain labels, no fine-tuning, and no test-time parameter updates. Instead of using Mamba-style selective scanning as a single generic feature path, DR-Mamba reinterprets it as fast-slow route modeling: a fast detail route captures local stroke structures, while a slow background route accumulates spatially persistent degradation responses. The two routes are integrated through an input-dependent subtractive gate that explicitly suppresses background interference rather than fusing features by addition or concatenation. We further add full-resolution detail-guided reconstruction and thin-stroke-aware supervision to recover fine strokes lost during downsampling. Evaluated under a leave-one-year-out protocol on DIBCO-style benchmarks, where each held-out year is treated as an unseen degradation domain, DR-Mamba shows that per-document, per-location subtractive suppression improves cross-domain robustness, with particularly strong performance on the most severely degraded held-out fold.

Abstract

Feature Fusion

Explore similar work

DR-Mamba: Automatic Inference-Time Domain Adaptation for Document Image Binarization via Sample-Conditioned Detail-Background Suppression

DeepMine-Mamba: Mitigating Information Dilution in Mamba-Based State Space Models for Document Image Binarization

Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging