GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection

Durgesh Ameta; Ujjwal Mishra; Praful Hambarde; Amit Shukla

cs.CVMar 1, 2026

GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection

Authors: Durgesh Ameta, Ujjwal Mishra, Praful Hambarde, Amit Shukla

Organizations: School of Computing, IIIT Una HP-177209, India · Center of Artificial Intelligence and Robotics (CAIR), IIT Mandi, Mandi, HP 175005, India

Abstract

Change detection (CD) in remote sensing aims to identify semantic differences between satellite images captured at different times. While deep learning has significantly advanced this field, existing approaches based on convolutional neural networks (CNNs), transformers and Selective State Space Models (SSMs) still struggle to precisely delineate change regions. In particular, traditional transformer-based methods suffer from quadratic computational complexity when applied to very high-resolution (VHR) satellite images and often perform poorly with limited training data, leading to under-utilization of the rich spatial information available in VHR imagery. We present GRAD-Former, a novel framework that enhances contextual understanding while maintaining efficiency through reduced model size. The proposed framework consists of a novel encoder with Adaptive Feature Relevance and Refinement (AFRAR) module, fusion and decoder blocks. AFRAR integrates global-local contextual awareness through two proposed components: the Selective Embedding Amplification (SEA) module and the Global-Local Feature Refinement (GLFR) module. SEA and GLFR leverage gating mechanisms and differential attention, respectively, which generates multiple softmax heaps to capture important features while minimizing the captured irreverent features. Multiple experiments across three challenging CD datasets (LEVIR-CD, CDD, DSIFN-CD) demonstrate GRAD-Former's superior performance compared to existing approaches. Notably, GRAD-Former outperforms the current state-of-the-art models across all the metrics and all the datasets while using fewer parameters. Our framework establishes a new benchmark for remote sensing change detection performance. Our code will be released at: https://github.com/Ujjwal238/GRAD-Former

Figures & tables

Fig. 1: Example images showing different challenges in the CD task, along with their ground-truth change masks. It is especially difficult to ignore irrelevant changes (marked in red boxes), such as (a) shadows and lighting differences, (b) seasonal variations, and (c) moving cars and roof changes, while accurately detecting (d) both large and small meaningful changes (marked in the green box).

Fig. 2: The overall design of the proposed GRAD-Former is depicted as follows: (a) The full network structure is outlined, showcasing the pre- and post-change input images processed through a shared encoder. This encoder extracts features across four stages, which are then combined using DA module. At each stage, the fused feature maps are merged within the decoder, which employs multiple convolutional and transposed convolution layers to generate upsampled features. These upsampled feature maps are ultimately utilized in the prediction layer to produce the final change map. (b) Detailed architecture of the decoder, it processes the fused feature maps $\hat{F}_{fuse}^{i}$ extracted from the four stages ( $i=1,2,3,4$ ) by concatenating them along the channel dimension. It then applies a $1\times 1$ convolution to project the features. Subsequently, upsampling is performed twice to restore the spatial dimensions to match the model input, utilizing a cascaded transpose convolution followed by a residual block consisting of two convolutional layers. Finally, a convolutional layer is employed to generate the prediction scores. (c) illustrates the encoder block structure, incorporating the novel Adaptive Feature Relevance and Refinement (AFRAR) module alongside a convolutional MLP layer. Instance normalization is applied across channels before input is fed into the AFRAR and MLP layers. (d) The detailed architecture of the AFRAR module is shown, which first splits the input ${\mathcal{F}}_{norm}^{i}$ along the channel and combines SEA and GLFR modules to extract relevant features, whose outputs are concatenated to form $\bar{\mathcal{F}}_{norm}^{i}$ .

Fig. 3: Detailed illustration of the proposed Difference Amalgamation (DA) module that captures semantic and differential features by concatenating

\hat{\mathcal{F}}_{pre}^{i}

and

\hat{\mathcal{F}}_{post}^{i}

along the channel dimension, along with their difference. Next, a convolutional layer is applied to reduce the channel count, followed by an activation layer within the DA module, producing

\hat{\mathcal{F}}_{fuse}^{i}

, where

i\in[1,2,3,4]

.)).

Fig. 4: (a) Overview of the proposed Global-Local Feature Refinement (GLFR) Module, An attention mechanism designed to overcome the diffused focus problem in transformers by implementing differential multi-head attention. The module generates

Q

K

V

matrices from input

\mathcal{F}^{i}_{\text{GLFR}}

and splits

Q

K

into pairs to calculate dual softmax attention maps (

A_{1}

A_{2}

). By computing the difference

A=A_{1}-\lambda\cdot A_{2}

with learnable scaling factor

\lambda

, the module creates sparse attention patterns that focus exclusively on relevant features while filtering out noise. The resulting attention output is combined with local features, providing an efficient balance between global context and local detail without the computational burden typical of transformer architectures. (b) depicts the proposed Selective Embedding Amplification (SEA) module that selectively amplifies important features through gating mechanism. The input features

\mathcal{F}^{i}_{\text{SEA}}

undergo

L2

normalization and multiplication by learnable parameter

\alpha

. Using another learnable parameter

\gamma

, a normalization factor is computed. The gating function (

G(x)=1+\tanh(x+\beta)

) then adaptively weights each channel based on its importance where

\beta

is another learnable scalar. Then the final output

\bar{\mathcal{F}}^{i}_{\text{SEA}}=\mathcal{F}^{i}_{\text{SEA}}\cdot G

enhances relevant features while suppressing noise, making the model robust against the sparsity of information in high-resolution satellite imagery.

Type	Method	Publication	CDD			DSIFN-CD			LEVIR-CD
Type	Method	Publication	$F_{1}$	IoU	OA(%)	$F_{1}$	IoU	OA(%)	$F_{1}$	IoU	OA(%)
CNN-based	FC-EF [ 19 ]	ICIP-2018	65.38	48.57	93.07	64.38	47.47	87.55	81.16	68.29	98.12
	FC-Sima-diff [ 19 ]	ICIP-2018	70.37	54.29	94.31	62.03	44.96	88.17	83.77	72.08	98.44
	FC-Sima-conc [ 19 ]	ICIP-2018	72.65	57.05	94.47	65.42	48.61	87.60	86.26	75.85	98.60
	DTCDSCN [ 64 ]	LGRS-2021	94.14	88.93	98.63	87.21	76.58	95.59	89.75	81.41	98.98
	SNUNet [ 30 ]	LGRS-2022	95.34	91.11	98.90	79.64	66.17	93.48	88.59	79.51	98.85

TABLE I: Performance comparison of different methods on CDD, DSIFN-CD, and LEVIR-CD datasets. Best results are in green and second-best results are in red .

Fig. 5: Qualitative comparisons on the DSIFN-CD dataset. We show the comparison with the eight best existing change detection approaches in the literature, whose codebases are publicly available.(a) Pre-Change image (A), (b) Post-Change image (B), (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DTCDSCN [ 64 ] , (g) ChangeMamba [ 52 ] , (h) BIT [ 36 ] , (i) ChangeFormer [ 36 ] , (j) ELGC-Net [ 42 ] , (k) GRAD-Former (OURS) and (l) Ground truth. Color scheme: white represents TP (i.e., “changed”), black corresponds to TN (i.e., “unchanged”), red indicates FP, and green denotes FN.

Fig. 6: Qualitative comparisons on the LEVIR-CD dataset. We show the comparison with the eight best existing change detection approaches in the literature, whose codebases are publicly available.(a) Pre-Change image (A), (b) Post-Change image (B), (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DTCDSCN [ 64 ] , (g) ChangeMamba [ 52 ] , (h) BIT [ 36 ] , (i) ChangeFormer [ 36 ] , (j) ELGC-Net [ 42 ] , (k) GRAD-Former (OURS) and (l) Ground truth. Color scheme: white represents TP (i.e., “changed”), black corresponds to TN (i.e., “unchanged”), red indicates FP, and green denotes FN.

Fig. 7: Qualitative comparisons on the CDD dataset. We show the comparison with the eight best existing change detection approaches in the literature, whose codebases are publicly available.(a) Pre-Change image (A), (b) Post-Change image (B), (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DTCDSCN [ 64 ] , (g) ChangeMamba [ 52 ] , (h) BIT [ 36 ] , (i) ChangeFormer [ 36 ] , (j) ELGC-Net [ 42 ] , (k) GRAD-Former (OURS) and (l) Ground truth. Color scheme: white represents TP (i.e., “changed”), black corresponds to TN (i.e., “unchanged”), red indicates FP, and green denotes FN.

Method	Params(M)	GFLOPs	$F_{1}$
FC-EF [ 19 ]	1.35	14.31	64.38
FC-EF-conc [ 19 ]	1.54	21.32	62.03
FC-EF-diff [ 19 ]	1.35	18.91	65.42
DTCDSCN [ 64 ]	31.25	52.89	87.21
SNUNet [ 30 ]	12.03	21.33	79.64
ISNet [ 65 ]	14.36	23.58	78.42

TABLE II: Comparison of different methods by model size, computational requirements, and performance metrics. Our proposed method achieves the highest

F_{1}

score (93.14%) with a moderate parameter count.

Module			Params (M)	IoU	$F_{1}$	OA(%)
SEA	GLFR	DA	Params (M)	IoU	$F_{1}$	OA(%)
$\checkmark$	$\times$	$\times$	9.99	79.69	87.93	96.98
$\times$	$\times$	$\checkmark$	10.72	80.30	88.9	97.09
$\checkmark$	$\times$	$\checkmark$	10.71	81.22	88.99	97.80
$\times$	$\checkmark$	$\times$	10.19	82.92	90.63	97.60
$\times$	$\checkmark$	$\checkmark$	10.20	83.10	90.79	99.08

TABLE III: Ablation study of combinations of Selective Embedding Amplification (SEA) module, Global-Local Feature Refinement (GLFR) module and the Difference Amalgamation (DA) module and their performance metrics.

Attention Type	Params (M)	$F_{1}$	IoU	OA(%)
Simple Attention	10.98	84.32	72.89	98.18
Pooled-Transpose Attention	10.83	91.08	83.62	99.1
Differential Attention	10.90	91.52	84.36	99.14

TABLE IV: Comparison of different attention mechanisms in the GLFR module on LEVIR-CD dataset.

Loss Function	$F_{1}$	IoU	OA(%)
Focal Loss	84.32	72.89	98.18
mIoU Loss	48.93	47.90	95.80
Cross-Entropy Loss	91.52	84.36	99.14

TABLE V: Comparative analysis with different loss functions applied during the model training on LEVIR-CD dataset, with optimal performance results emphasized in bold.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Type	Method	Params(M)	Publication	WHU-CD
Type	Method	Params(M)	Publication	$F_{1}$	IoU	OA(%)
CNN-based	FC-EF [ 19 ]	1.35	ICIP-18	80.64	67.56	98.12
	FC-Sima-diff [ 19 ]	1.54	ICIP-18	67.64	51.1	95.82
	FC-Sima-conc [ 19 ]	1.35	ICIP-18	64.66	47.78	95.32
	DTCDSCN [ 64 ]	31.25	LGRS-21	87.79	78.23	98.89
	SNUNet [ 30 ]	12.03	LGRS-22	90.29	82.3	99.05

Appendix

Table 1: Comparison of different methods for remote sensing change detection on the WHU dataset over

F_{1}

score, IoU and Overall Accuracy(OA (%)) where

F_{1}

and IoU are calculated over the change class. The highest and second highest values in each metric column are highlighted.

Figure 1: Performance comparison of change detection models as per Table 1 . The plot illustrates the trade-off between model size (parameters in millions) and performance ( $F_{1}$ score over WHU-CD dataset). Bubble color indicates computational complexity (GFLOPs). The dashed blue line represents the Pareto frontier of optimal models. Lying on the Pareto’s front our model achieves state-of-the-art $F_{1}$ score (91.29%),while maintaining reasonable computational efficiency (129.5 GFLOPs).

Figure 2: A visual comparison of various state-of-the-art change detection methods applied to the WHU-CD dataset: (a) pre-change image, (b) post-change image, (c) FC-EF [ 19 ] , (d) FC-Siam-diff [ 19 ] , (e) FC-Siam-conc [ 19 ] , (f) DT-SCN [ 64 ] , (g) BIT [ 36 ] , (h) ChangeFormer [ 68 ] , (i) GRAD-Former, and (j) the ground truth. In the results, white pixels represent true positives (areas of change), black pixels represent true negatives (unchanged areas), and red pixels denote both false positives and false negatives.

Explore similar work

Mar 20, 2026

Beyond Quadratic: Linear-Time Change Detection with RWKV

Existing paradigms for remote sensing change detection are caught in a trade-off: CNNs excel at efficiency but lack global context, while Transformers capture long-range dependencies at a prohibitive computational cost. This paper introduces ChangeRWKV, a new architecture that reconciles this conflict. By building upon the Receptance Weighted Key Value (RWKV) framework, our ChangeRWKV uniquely combines the parallelizable training of Transformers with the linear-time inference of RNNs. Our approach core features two key innovations: a hierarchical RWKV encoder that builds multi-resolution feature representation, and a novel Spatial-Temporal Fusion Module (STFM) engineered to resolve spatial misalignments across scales while distilling fine-grained temporal discrepancies. ChangeRWKV not only achieves state-of-the-art performance on the LEVIR-CD benchmark, with an 85.46% IoU and 92.16% F1 score, but does so while drastically reducing parameters and FLOPs compared to previous leading methods. This work demonstrates a new, efficient, and powerful paradigm for operational-scale change detection. Our code and model are publicly available.

Zhenyu Yang, Gensheng Pei, Tao Chen +4

School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · State Key Laboratory of Intelligent Manufacturing of Advanced Construction Machinery, China · Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon, Korea

Remote Sensing Change Detection

Ask PDF

Feb 21, 2026

NeXt2Former-CD: Efficient Remote Sensing Change Detection with Modern Vision Architectures

State Space Models (SSMs) have recently gained traction in remote sensing change detection (CD) for their favorable scaling properties. In this paper, we explore the potential of modern convolutional and attention-based architectures as a competitive alternative. We propose NeXt2Former-CD, an end-to-end framework that integrates a Siamese ConvNeXt encoder initialized with DINOv3 weights, a deformable attention-based temporal fusion module, and a Mask2Former decoder. This design is intended to better tolerate residual co-registration noise and small object-level spatial shifts, as well as semantic ambiguity in bi-temporal imagery. Experiments on LEVIR-CD, WHU-CD, and CDD datasets show that our method achieves the best results among the evaluated methods, improving over recent Mamba-based baselines in both F1 score and IoU. Furthermore, despite a larger parameter count, our model maintains inference latency comparable to SSM-based approaches, suggesting it is practical for high-resolution change detection tasks.

Yufan Wang, Sokratis Makrogiannis, Chandra Kambhamettu

University of South Florida · Delaware State University

Remote Sensing Change Detection Hybrid CNN-Transformer Architectures Convolutional Neural Networks

Ask PDF

Oct 11, 2026

ChangeT $^3$ : Evaluating Vision Test-Time Training for Remote Sensing Change Detection

Bitemporal remote sensing change detection requires representations that preserve spatial detail while capturing contextual information relevant to changes between acquisitions. Visual test-time training offers an alternative to explicit token-to-token attention by adapting a compact inner model to each image. This letter describes ChangeT

^3

, a vanilla Siamese configuration combining the existing hierarchical ViT

^3

-Tiny encoder with the multiscale ChangeFormer decoder. The two image streams share learned encoder parameters, while their fast weights are computed independently from their respective spatial features. Each encoder block retains the inherited global simplified SwiGLU and local adaptive depthwise-convolution branches. Temporal comparison is performed by the decoder through learned fusion of paired features at four resolutions, followed by coarse-to-fine aggregation and binary prediction. This separation provides a direct setting for examining image-conditioned inner updates without introducing an additional temporal interaction module. Trained from scratch on LEVIR-CD+ with the same decoder, recipe, and budget as a ChangeFormer control, ChangeT

^3

reaches an F1 of 80.61

\pm

0.90 against 78.84

\pm

0.21 for ChangeFormer (three seeds each); as in the standard protocol for this dataset, checkpoints are selected on the test split. Trained with both inner updates disabled, the same encoder reaches 78.63

\pm

0.20. Neither model transfers zero-shot to SYSU-CD, DSIFN-CD, or WHU-CD, and ChangeT

^3

is the slower of the two at small inputs despite similar parameter and operation counts.

Sharukshan Niranjan, Oshadha Samarakoon, Nichula Wasalathilaka +3

University of Peradeniya, Peradeniya, Sri Lanka

Remote Sensing Change Detection Test-Time Training Vision Transformer

Ask PDF

Abstract

Explore similar work

Beyond Quadratic: Linear-Time Change Detection with RWKV

NeXt2Former-CD: Efficient Remote Sensing Change Detection with Modern Vision Architectures

ChangeT3^33: Evaluating Vision Test-Time Training for Remote Sensing Change Detection

ChangeT $^3$ : Evaluating Vision Test-Time Training for Remote Sensing Change Detection