GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection

Durgesh Ameta; Ujjwal Mishra; Praful Hambarde; Amit Shukla

cs.CVMar 1, 2026

GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection

Authors: Durgesh Ameta, Ujjwal Mishra, Praful Hambarde, Amit Shukla

Organizations: School of Computing, IIIT Una HP-177209, India · Center of Artificial Intelligence and Robotics (CAIR), IIT Mandi, Mandi, HP 175005, India

Abstract

Change detection (CD) in remote sensing aims to identify semantic differences between satellite images captured at different times. While deep learning has significantly advanced this field, existing approaches based on convolutional neural networks (CNNs), transformers and Selective State Space Models (SSMs) still struggle to precisely delineate change regions. In particular, traditional transformer-based methods suffer from quadratic computational complexity when applied to very high-resolution (VHR) satellite images and often perform poorly with limited training data, leading to under-utilization of the rich spatial information available in VHR imagery. We present GRAD-Former, a novel framework that enhances contextual understanding while maintaining efficiency through reduced model size. The proposed framework consists of a novel encoder with Adaptive Feature Relevance and Refinement (AFRAR) module, fusion and decoder blocks. AFRAR integrates global-local contextual awareness through two proposed components: the Selective Embedding Amplification (SEA) module and the Global-Local Feature Refinement (GLFR) module. SEA and GLFR leverage gating mechanisms and differential attention, respectively, which generates multiple softmax heaps to capture important features while minimizing the captured irreverent features. Multiple experiments across three challenging CD datasets (LEVIR-CD, CDD, DSIFN-CD) demonstrate GRAD-Former's superior performance compared to existing approaches. Notably, GRAD-Former outperforms the current state-of-the-art models across all the metrics and all the datasets while using fewer parameters. Our framework establishes a new benchmark for remote sensing change detection performance. Our code will be released at: https://github.com/Ujjwal238/GRAD-Former

Figures & tables

Fig. 1: Example images showing different challenges in the CD task, along with their ground-truth change masks. It is especially difficult to ignore irrelevant changes (marked in red boxes), such as (a) shadows and lighting differences, (b) seasonal variations, and (c) moving cars and roof changes, while accurately detecting (d) both large and small meaningful changes (marked in the green box).

Fig. 2: The overall design of the proposed GRAD-Former is depicted as follows: (a) The full network structure is outlined, showcasing the pre- and post-change input images processed through a shared encoder. This encoder extracts features across four stages, which are then combined using DA module. At each stage, the fused feature maps are merged within the decoder, which employs multiple convolutional and transposed convolution layers to generate upsampled features. These upsampled feature maps are ultimately utilized in the prediction layer to produce the final change map. (b) Detailed architecture of the decoder, it processes the fused feature maps $\hat{F}_{fuse}^{i}$ extracted from the four stages ( $i=1,2,3,4$ ) by concatenating them along the channel dimension. It then applies a $1\times 1$ convolution to project the features. Subsequently, upsampling is performed twice to restore the spatial dimensions to match the model input, utilizing a cascaded transpose convolution followed by a residual block consisting of two convolutional layers. Finally, a convolutional layer is employed to generate the prediction scores. (c) illustrates the encoder block structure, incorporating the novel Adaptive Feature Relevance and Refinement (AFRAR) module alongside a convolutional MLP layer. Instance normalization is applied across channels before input is fed into the AFRAR and MLP layers. (d) The detailed architecture of the AFRAR module is shown, which first splits the input ${\mathcal{F}}_{norm}^{i}$ along the channel and combines SEA and GLFR modules to extract relevant features, whose outputs are concatenated to form $\bar{\mathcal{F}}_{norm}^{i}$ .

Fig. 3: Detailed illustration of the proposed Difference Amalgamation (DA) module that captures semantic and differential features by concatenating

\hat{\mathcal{F}}_{pre}^{i}

and

\hat{\mathcal{F}}_{post}^{i}

along the channel dimension, along with their difference. Next, a convolutional layer is applied to reduce the channel count, followed by an activation layer within the DA module, producing

\hat{\mathcal{F}}_{fuse}^{i}

, where

i\in[1,2,3,4]

.)).

Fig. 4: (a) Overview of the proposed Global-Local Feature Refinement (GLFR) Module, An attention mechanism designed to overcome the diffused focus problem in transformers by implementing differential multi-head attention. The module generates

Q

K

V

matrices from input

\mathcal{F}^{i}_{\text{GLFR}}

and splits

Q

K

into pairs to calculate dual softmax attention maps (

A_{1}

A_{2}

). By computing the difference

A=A_{1}-\lambda\cdot A_{2}

with learnable scaling factor

\lambda

, the module creates sparse attention patterns that focus exclusively on relevant features while filtering out noise. The resulting attention output is combined with local features, providing an efficient balance between global context and local detail without the computational burden typical of transformer architectures. (b) depicts the proposed Selective Embedding Amplification (SEA) module that selectively amplifies important features through gating mechanism. The input features

\mathcal{F}^{i}_{\text{SEA}}

undergo

L2

normalization and multiplication by learnable parameter

\alpha

. Using another learnable parameter

\gamma

, a normalization factor is computed. The gating function (

G(x)=1+\tanh(x+\beta)

) then adaptively weights each channel based on its importance where

\beta

is another learnable scalar. Then the final output

\bar{\mathcal{F}}^{i}_{\text{SEA}}=\mathcal{F}^{i}_{\text{SEA}}\cdot G

enhances relevant features while suppressing noise, making the model robust against the sparsity of information in high-resolution satellite imagery.

Type	Method	Publication	CDD			DSIFN-CD			LEVIR-CD
Type	Method	Publication	$F_{1}$	IoU	OA(%)	$F_{1}$	IoU	OA(%)	$F_{1}$	IoU	OA(%)
CNN-based	FC-EF [ 19 ]	ICIP-2018	65.38	48.57	93.07	64.38	47.47	87.55	81.16	68.29	98.12
	FC-Sima-diff [ 19 ]	ICIP-2018	70.37	54.29	94.31	62.03	44.96	88.17	83.77	72.08	98.44
	FC-Sima-conc [ 19 ]	ICIP-2018	72.65	57.05	94.47	65.42	48.61	87.60	86.26	75.85	98.60
	DTCDSCN [ 64 ]	LGRS-2021	94.14	88.93	98.63	87.21	76.58	95.59	89.75	81.41	98.98
	SNUNet [ 30 ]	LGRS-2022	95.34	91.11	98.90	79.64	66.17	93.48	88.59	79.51	98.85

TABLE I: Performance comparison of different methods on CDD, DSIFN-CD, and LEVIR-CD datasets. Best results are in green and second-best results are in red .

Fig. 5: Qualitative comparisons on the DSIFN-CD dataset. We show the comparison with the eight best existing change detection approaches in the literature, whose codebases are publicly available.(a) Pre-Change image (A), (b) Post-Change image (B), (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DTCDSCN [ 64 ] , (g) ChangeMamba [ 52 ] , (h) BIT [ 36 ] , (i) ChangeFormer [ 36 ] , (j) ELGC-Net [ 42 ] , (k) GRAD-Former (OURS) and (l) Ground truth. Color scheme: white represents TP (i.e., “changed”), black corresponds to TN (i.e., “unchanged”), red indicates FP, and green denotes FN.

Fig. 6: Qualitative comparisons on the LEVIR-CD dataset. We show the comparison with the eight best existing change detection approaches in the literature, whose codebases are publicly available.(a) Pre-Change image (A), (b) Post-Change image (B), (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DTCDSCN [ 64 ] , (g) ChangeMamba [ 52 ] , (h) BIT [ 36 ] , (i) ChangeFormer [ 36 ] , (j) ELGC-Net [ 42 ] , (k) GRAD-Former (OURS) and (l) Ground truth. Color scheme: white represents TP (i.e., “changed”), black corresponds to TN (i.e., “unchanged”), red indicates FP, and green denotes FN.

Fig. 7: Qualitative comparisons on the CDD dataset. We show the comparison with the eight best existing change detection approaches in the literature, whose codebases are publicly available.(a) Pre-Change image (A), (b) Post-Change image (B), (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DTCDSCN [ 64 ] , (g) ChangeMamba [ 52 ] , (h) BIT [ 36 ] , (i) ChangeFormer [ 36 ] , (j) ELGC-Net [ 42 ] , (k) GRAD-Former (OURS) and (l) Ground truth. Color scheme: white represents TP (i.e., “changed”), black corresponds to TN (i.e., “unchanged”), red indicates FP, and green denotes FN.

Method	Params(M)	GFLOPs	$F_{1}$
FC-EF [ 19 ]	1.35	14.31	64.38
FC-EF-conc [ 19 ]	1.54	21.32	62.03
FC-EF-diff [ 19 ]	1.35	18.91	65.42
DTCDSCN [ 64 ]	31.25	52.89	87.21
SNUNet [ 30 ]	12.03	21.33	79.64
ISNet [ 65 ]	14.36	23.58	78.42

TABLE II: Comparison of different methods by model size, computational requirements, and performance metrics. Our proposed method achieves the highest

F_{1}

score (93.14%) with a moderate parameter count.

Module			Params (M)	IoU	$F_{1}$	OA(%)
SEA	GLFR	DA	Params (M)	IoU	$F_{1}$	OA(%)
$\checkmark$	$\times$	$\times$	9.99	79.69	87.93	96.98
$\times$	$\times$	$\checkmark$	10.72	80.30	88.9	97.09
$\checkmark$	$\times$	$\checkmark$	10.71	81.22	88.99	97.80
$\times$	$\checkmark$	$\times$	10.19	82.92	90.63	97.60
$\times$	$\checkmark$	$\checkmark$	10.20	83.10	90.79	99.08

TABLE III: Ablation study of combinations of Selective Embedding Amplification (SEA) module, Global-Local Feature Refinement (GLFR) module and the Difference Amalgamation (DA) module and their performance metrics.

Attention Type	Params (M)	$F_{1}$	IoU	OA(%)
Simple Attention	10.98	84.32	72.89	98.18
Pooled-Transpose Attention	10.83	91.08	83.62	99.1
Differential Attention	10.90	91.52	84.36	99.14

TABLE IV: Comparison of different attention mechanisms in the GLFR module on LEVIR-CD dataset.

Loss Function	$F_{1}$	IoU	OA(%)
Focal Loss	84.32	72.89	98.18
mIoU Loss	48.93	47.90	95.80
Cross-Entropy Loss	91.52	84.36	99.14

TABLE V: Comparative analysis with different loss functions applied during the model training on LEVIR-CD dataset, with optimal performance results emphasized in bold.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Method	Params(M)	Publication	WHU-CD
Type	Method	Params(M)	Publication	$F_{1}$	IoU	OA(%)
CNN-based	FC-EF [ 19 ]	1.35	ICIP-18	80.64	67.56	98.12
	FC-Sima-diff [ 19 ]	1.54	ICIP-18	67.64	51.1	95.82
	FC-Sima-conc [ 19 ]	1.35	ICIP-18	64.66	47.78	95.32
	DTCDSCN [ 64 ]	31.25	LGRS-21	87.79	78.23	98.89
	SNUNet [ 30 ]	12.03	LGRS-22	90.29	82.3	99.05

Appendix

Table 1: Comparison of different methods for remote sensing change detection on the WHU dataset over

F_{1}

score, IoU and Overall Accuracy(OA (%)) where

F_{1}

and IoU are calculated over the change class. The highest and second highest values in each metric column are highlighted.

Figure 1: Performance comparison of change detection models as per Table 1 . The plot illustrates the trade-off between model size (parameters in millions) and performance ( $F_{1}$ score over WHU-CD dataset). Bubble color indicates computational complexity (GFLOPs). The dashed blue line represents the Pareto frontier of optimal models. Lying on the Pareto’s front our model achieves state-of-the-art $F_{1}$ score (91.29%),while maintaining reasonable computational efficiency (129.5 GFLOPs).

Figure 2: A visual comparison of various state-of-the-art change detection methods applied to the WHU-CD dataset: (a) pre-change image, (b) post-change image, (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DT-SCN [ 64 ] , (g) BIT [ 36 ] , (h) ChangeFormer [ 68 ] , (i) GRAD-Former, and (j) the ground truth. In the results, white pixels represent true positives (areas of change), black pixels represent true negatives (unchanged areas), and red pixels denote both false positives and false negatives.

Explore similar work

Beyond Quadratic: Linear-Time Change Detection with RWKV
Mar 20, 2026Zhenyu Yang, Gensheng Pei, Tao Chen +4Remote Sensing Change Detection
School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery, China · Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea
NeXt2Former-CD: Efficient Remote Sensing Change Detection with Modern Vision Architectures
Feb 21, 2026Yufan Wang, Sokratis Makrogiannis, Chandra KambhamettuRemote Sensing Change Detection Hybrid CNN-Transformer Architectures
University of South Florida · Delaware State University
ChangeT $^3$ : Evaluating Vision Test-Time Training for Remote Sensing Change Detection
Oct 11, 2026Sharukshan Niranjan, Oshadha Samarakoon, Nichula Wasalathilaka +3Remote Sensing Change Detection Test-Time Training
University of Peradeniya, Peradeniya, Sri Lanka

Mar 20, 2026J/K move · Enter open · S save

Beyond Quadratic: Linear-Time Change Detection with RWKV

Zhenyu Yang, Gensheng Pei, Tao Chen, Xia Yuan+3

School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery, China · Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea

Remote Sensing Change Detection

Open paper AskPDF arXiv

Existing paradigms for remote sensing change detection are caught in a trade-off: CNNs excel at efficiency but lack global context, while Transformers capture long-range dependencies at a prohibitive computational cost. This paper introduces ChangeRWKV, a new architecture that reconciles this conflict. By building upon the Receptance Weighted Key Value (RWKV) framework, our ChangeRWKV uniquely combines the parallelizable training of Transformers with the linear-time inference of RNNs. Our approach core features two key innovations: a hierarchical RWKV encoder that builds multi-resolution feature representation, and a novel Spatial-Temporal Fusion Module (STFM) engineered to resolve spatial misalignments across scales while distilling fine-grained temporal discrepancies. ChangeRWKV not only achieves state-of-the-art performance on the LEVIR-CD benchmark, with an 85.46% IoU and 92.16% F1 score, but does so while drastically reducing parameters and FLOPs compared to previous leading methods. This work demonstrates a new, efficient, and powerful paradigm for operational-scale change detection. Our code and model are publicly available.

Abstract

Explore similar work

Beyond Quadratic: Linear-Time Change Detection with RWKV

NeXt2Former-CD: Efficient Remote Sensing Change Detection with Modern Vision Architectures

ChangeT3^33: Evaluating Vision Test-Time Training for Remote Sensing Change Detection

ChangeT $^3$ : Evaluating Vision Test-Time Training for Remote Sensing Change Detection