CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning

Eunki Cho; Hyeon Bae Kim; Seong Tae Kim

cs.CVOct 26, 2026

CoDeRNet: Selective Cross-Task Routing under Heterogeneous Supervision for Change Detection and Captioning

Authors: Eunki Cho, Hyeon Bae Kim, Seong Tae Kim

Organizations: Kyung Hee University, Republic of Korea

Abstract

Change Detection & Captioning (CDC) aims to jointly localize changed regions and describe their semantic content from bi-temporal remote sensing images. While recent approaches have demonstrated the effectiveness of unified modeling, they primarily rely on shared representations or jointly optimized architectures, leaving task-specific communication under heterogeneous supervision less explored. However, detection and captioning differ fundamentally in their supervision granularity, leading to heterogeneous representations with uneven reliability across tasks and spatial locations. In this paper, we reformulate CDC as a selective information routing problem between heterogeneous task branches, where the key challenge is to determine when and where information should be transferred. To this end, we propose CoDeRNet, a unified framework that preserves task-specific representations while enabling bidirectional exchange of complementary information. CoDeRNet introduces a Confidence-Decomposed Routing (CoDeR) mechanism, which decomposes the routing condition into receiver-side need for support and sender-side reliability cues to guide selective transfer. Experiments on LEVIR-MCI and WHU-CDC show that CoDeRNet achieves strong overall CDC performance, particularly improving caption generation while preserving competitive change localization. Extensive analyses further show that the benefits arise not merely from joint learning or generic cross-task communication, but from explicitly modeling selective cross-task information routing. These results highlight the importance of conditional information transfer for unified CDC under heterogeneous supervision.

Figures & tables

Figure 1: Overview of CoDeRNet. Shared change features are transformed into task-specific detection and captioning features, followed by Confidence-Decomposed Routing (CoDeR) for selective bidirectional transfer before task-specific decoding.

Task	Method	Metrics
Task	Method	mIoU	BLEU-1	BLEU-2	BLEU-3	BLEU-4	METEOR	ROUGE-L	CIDEr-D
Change Detection	SNUNet [ 12 ]	82.53	-	-	-	-	-	-	-
	ChangeFormer [ 1 ]	84.35	-	-	-	-	-	-	-
	TFI-GR [ 19 ]	79.34	-	-	-	-	-	-	-
	A2Net [ 18 ]	80.82	-	-	-	-	-	-	-
	HANet [ 13 ]	71.23	-	-	-	-	-	-	-

Table 1: Comparison on LEVIR-MCI [ 22 ] . “Change Detection” and “Change Captioning” denote single-task models trained for only one task, whereas “Change Detection & Captioning” denotes unified models that perform both tasks jointly within a single framework. Bold and underlined values indicate the best and second-best results, respectively.

Task	Method	Metrics
Task	Method	IoU	BLEU-1	BLEU-2	BLEU-3	BLEU-4	METEOR	ROUGE-L	CIDEr-D
Change Detection	SNUNet [ 12 ]	74.81	-	-	-	-	-	-	-
	ChangeFormer [ 1 ]	77.88	-	-	-	-	-	-	-
	TFI-GR [ 19 ]	77.60	-	-	-	-	-	-	-
	A2Net [ 18 ]	80.43	-	-	-	-	-	-	-
	HANet [ 13 ]	79.95	-	-	-	-	-	-	-

Table 2: Comparison on WHU-CDC [ 35 ] . “Change Detection” and “Change Captioning” denote single-task models trained for only one task, whereas “Change Detection & Captioning” denotes unified models that perform both tasks jointly within a single framework. Bold and underlined values indicate the best and second-best results, respectively.

Figure 2: Qualitative comparison on LEVIR-MCI. In the captions, building-related words (e.g., house , building ) and road-related words (e.g., road ) are highlighted using colors consistent with the corresponding mask classes.

Seg	Cap	CoDeR	mIoU	BLEU-4	METEOR	ROUGE-L	CIDEr-D
✓	-	-	87.03	-	-	-	-
-	✓	-	-	63.18	40.21	75.27	137.41
✓	✓	-	87.67	64.41	39.93	74.98	136.18
✓	✓	✓	87.79	66.27	41.08	76.36	141.79

Table 3: Ablation studies on LEVIR-MCI [ 22 ] . (a) Effect of joint training and explicit interaction, comparing single-task training, joint training without CoDeR, and joint training with CoDeR. (b) Effect of CoDeR directionality, comparing two single-direction settings with the bidirectional design. Bold indicates the best result in each column.

Mask	mIoU	BLEU-4	METEOR	ROUGE-L	CIDEr-D
Sender reliability $(C_{s})$	87.60	65.12	40.49	75.73	139.57
Receiver need $(1-C_{t})$	87.65	64.65	40.81	75.94	141.02
Single gate	87.59	65.46	40.27	75.54	138.19
Decomposed routing $((1-C_{t})\cdot C_{s})$	87.79	66.27	41.08	76.36	141.79

Table 4: Ablation studies on LEVIR-MCI [ 22 ] for analyzing the effectiveness of CoDeR. (a) Routing mask decomposition ablation compares CoDeR with reduced variants and a single-gate design. (b) Interaction operator ablation compares CoDeR with representative cross-task communication operators under the same framework. Bold and underline indicate the best and second-best results in each column, respectively.

Model	Factual grounding	Cross-output consistency
MCINet [ 22 ]	$3.520\pm 1.38$	$3.632\pm 1.25$
Ours (w/o CoDeR)	$3.645\pm 1.22$	$3.782\pm 1.09$
Ours (w/ CoDeR)	$\mathbf{3.658}\pm 1.27$	$\mathbf{3.813}\pm 1.13$

Table 5: Blind human evaluation on the change-containing subset of LEVIR-MCI. Values are mean

\pm

standard deviation on a five-point scale, with 840 ratings per system for each criterion across 660 unique image pairs. Higher scores indicate better factual grounding or cross-output consistency. Bold indicates the highest mean in each column.

Table 6: Ablation of the CoDeR routing scale on LEVIR-MCI [ 22 ] . Each subtable reports performance when same-scale routing is applied at different receiving scales in one direction, while the opposite direction is fixed to its default setting. Bold indicates the best result in each column.

Backbone	CoDeR	mIoU	BLEU-4	MET.	R-L	CID.
ConvNeXt-B [ 27 ]	$\times$	87.67	64.41	39.93	74.98	136.18
ConvNeXt-B [ 27 ]	$\checkmark$	87.79	66.27	41.08	76.36	141.79
SwinV2-B [ 26 ]	$\times$	87.49	63.55	40.24	75.26	136.93
SwinV2-B [ 26 ]	$\checkmark$	87.34	64.45	40.30	75.31	138.20
MiT-B5 [ 42 ]	$\times$	87.74	63.92	40.15	75.49	137.54
MiT-B5 [ 42 ]	$\checkmark$	87.78	65.38	40.54	75.54	140.28

Table 7: Robustness of CoDeRNet across backbone choices on LEVIR-MCI [ 22 ] . (a) Backbone-family ablation: performance changes with and without CoDeR across different backbones. (b) Backbone-size ablation: performance changes with and without CoDeR across ConvNeXt variants from Tiny to Large. Bold indicates the best result in each column. In the CoDeRNet rows, colored values indicate improvements ( gain ) or degradations ( drop ) relative to the corresponding baseline. B4, MET., R-L, and CID. denote BLEU-4, METEOR, ROUGE-L, and CIDEr-D, respectively.

Type	Setting	mIoU	BLEU-4	METEOR	ROUGE-L	CIDEr-D
Multi	both $(\text{all}\leftrightarrow\text{all})$	87.66	65.81	40.18	75.93	138.83
	seg only $(\text{all}\leftrightarrow 1/16)$	87.75	65.44	41.03	76.22	142.18
	cap only $(1/8\leftrightarrow\text{all})$	87.86	64.56	40.83	75.93	140.32
Single (Ours)	$(1/8\leftrightarrow 1/16)$	87.79	66.27	41.08	76.36	141.79

Table 8: Ablation studies on LEVIR-MCI [ 22 ] . (a) Single-scale versus multi-scale CoDeR application. The default design applies CoDeR at one receiving scale per direction (seg

\leftarrow

cap at

1/8

, cap

\leftarrow

seg at

1/16

), while the multi-scale variants apply independent CoDeR blocks at all four scales for the indicated direction(s). (b) Region-token pooling size ablation for caption decoding. Bold indicates the best result in each column.

Aug.	mIoU	BLEU-4	METEOR	ROUGE-L	CIDEr-D
-	86.61	64.51	39.44	74.72	136.06
G	86.67	64.45	40.64	75.97	139.21
G+R	87.79	66.27	41.08	76.36	141.79

Table 9: Additional ablation studies on LEVIR-MCI [ 22 ] . (a) Effect of augmentation type, where

G

denotes geometric augmentation and

R

denotes directional caption remapping. (b) Effect of change fusion composition, where

F

is formed from temporal features at

t_{0}

and

t_{1}

together with signed and magnitude change cues

\Delta

and

|\Delta|

. Bold indicates the best result in each column.

Transform	Category	Rule / Example
Remapping rules
H-Flip	Single-level	$\textit{left}\leftrightarrow\textit{right}$
	Phrase-level	$\textit{upper left}\leftrightarrow\textit{upper right}$
	Phrase-level	$\textit{lower left}\leftrightarrow\textit{lower right}$
	Phrase-level	$\textit{top left}\leftrightarrow\textit{top right}$
	Phrase-level	$\textit{bottom left}\leftrightarrow\textit{bottom right}$

Table 10: Directional caption remapping under geometric augmentation on LEVIR-MCI. To preserve caption–image consistency, direction-sensitive expressions are remapped according to the applied geometric transform. Two-word directional phrases are matched prior to single-word directional levels.

Seg	Cap	CoDeR	mIoU	cIoU
$\checkmark$	$\times$	$\times$	87.03	82.05
$\checkmark$	$\checkmark$	$\times$	87.67	82.97
$\checkmark$	$\checkmark$	$\checkmark$	87.79	83.07

Table 11: Additional detection evaluation. (a) Multi-class mIoU and change-class IoU (cIoU) on LEVIR-MCI [ 22 ] . Seg and Cap indicate change detection and change captioning training, respectively. cIoU is computed by merging building and road into a single change class. Bold indicates the best result in each metric column. (b) Image-level paired bootstrap analysis.

\Delta

denotes the full-test-set performance difference between CoDeRNet and MCINet. Differences and 95% confidence intervals are expressed in percentage points. Positive values favor CoDeRNet.

Figure 3: Qualitative comparison on LEVIR-MCI. In the captions, building-related words (e.g., house , building ) and road-related words (e.g., road ) are highlighted using colors consistent with the corresponding mask classes.

Figure 4: Qualitative comparison on WHU-CDC. In the captions, building-related words (e.g., house , building ) are highlighted using colors consistent with the corresponding mask classes.

Remote Sensing Image Change Captioning

Remote sensing image change captioning describes meaningful changes between satellite or aerial images in natural language.

See all 13 papers

CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation
Jun 27, 2026Jinhong Hu, Xiaoping Wang, Shuyin Huang +3Image Captioning Image Segmentation
College of Computer Science and Electronic Engineering, Hunan University, China · Guangxi Minzu University, China · National University of Defense Technology, China
From Change Captions to Change Detection: Semantic-Appearance Agreement Framework for Remote Sensing Change Detection
Sep 23, 2026Yuan Qian, Jie MaRemote Sensing Change Detection Remote Sensing
School of Information Science and Technology, Beijing Foreign Studies University, Beijing 100089, China
CoSA: Correlation-Guided Change Attention with Learnable Residual Gating for Remote Sensing Change Detection
Jun 20, 2026Abdirashid Omar, Jonghyuk ParkRemote Sensing Change Detection Attention Mechanisms
Department of Data Science, Graduate School of Kookmin University Seoul 02707, Republic of Korea

Jun 27, 2026J/K move · Enter open · S save

CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation

Jinhong Hu, Xiaoping Wang, Shuyin Huang, Guojin Zhong+2

College of Computer Science and Electronic Engineering, Hunan University, China · Guangxi Minzu University, China · National University of Defense Technology, China

Image Captioning Image Segmentation

Open paper AskPDF arXiv

Understanding and localizing subtle changes between paired images is critical for tasks such as surveillance and image editing. However, traditional Image Change Captioning (ICC) methods lack spatial grounding, limiting their precision. We introduce Image Change Captioning and Segmentation (ICCS), a new multimodal task that jointly requires structured change description and pixel-level localization. To address ICCS, we propose the Change-aware Captioning and Reasoning Chain (CCRC), a dual-chain framework that decouples semantic reasoning from spatial segmentation. The first chain, Chain-of-Change-Captioning (CCC), enhances fine-grained change perception via a visual fusion module based on Multi-Head Change-aware Attention inserted between the visual and language components of a Multimodal Large Language Model (MLLM). CCC also determines whether a change is segmentable. If not, it alone generates the caption. Otherwise, the second chain, Chain-of-Change-Segmenting (CCS), is activated, leveraging spatial priors from CCC and refining masks with a Change-aware Token Refiner for accurate boundary localization. We evaluate CCRC on both synthetic and real-world change detection benchmarks with pixel-level supervision. Experiments show CCRC achieves state-of-the-art performance.

Abstract

Remote Sensing Image Change Captioning

Explore similar work

CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation

From Change Captions to Change Detection: Semantic-Appearance Agreement Framework for Remote Sensing Change Detection

CoSA: Correlation-Guided Change Attention with Learnable Residual Gating for Remote Sensing Change Detection