GPF-Net: Gated Progressive Fusion Learning for Polyp Re-Identification

Suncheng Xiang; Xiaoyang Wang; Junjie Jiang; Hejia Wang; Dahong Qian

cs.CVDec 25, 2025

GPF-Net: Gated Progressive Fusion Learning for Polyp Re-Identification

Authors: Suncheng Xiang, Xiaoyang Wang, Junjie Jiang, Hejia Wang, Dahong Qian

Organizations: Shanghai Jiao Tong University · Peking University · Shanghai Fifth People’s Hospital

Abstract

Colonoscopic Polyp Re-Identification (ReID) aims to match the same polyp across a large gallery of images captured from different viewpoints and with different cameras, playing a critical role in computer-aided diagnosis for the prevention and treatment of colorectal cancer. However, the coarse granularity of high-level features often limits performance on small polyps, where fine-grained details are essential for accurate matching. To address this challenge, we propose a novel multimodal feature fusion architecture, termed the Gated Progressive Fusion Network, which selectively integrates features from multiple levels through fully connected gating mechanisms. Building on this framework, we introduce a gated progressive fusion strategy that enables layer-wise refinement of semantic information, facilitating multi-level feature interactions to enhance both generalization ability and robustness. Extensive experiments on standard benchmarks demonstrate the advantages of the multimodal setting over state-of-the-art unimodal ReID models, particularly when combined with the proposed fusion strategy tailored for general-purpose scenarios.

Figures & tables

Fig. 1: We introduce a multimodal polyp dataset for polyp ReID, where each polyp ReID will encounter many challenges such as variation in terms of backgrounds, viewpoint, and illumination.

Fig. 2: The overall architecture of our GPF-Net is designed around two core components: a visual-textual feature extraction model and a two-stage feature fusion module, which work in tandem to process multi-modal data.

Method	Venue	Video Retrieval $\uparrow$
Method	Venue	mAP	Rank-1	Rank-5	Rank-10
ViSiL [ 11 ]	ICCV	24.9	14.5	30.6	51.6
CoCLR [ 9 ]	NIPS	16.3	6.5	22.6	33.9
TCA [ 18 ]	WCAV	27.8	16.1	35.5	53.2
ViT [ 3 ]	CVPR	20.4	9.7	30.6	43.5
CVRL [ 16 ]	CVPR	23.6	11.3	32.3	53.2

TABLE I: Performance comparison with state-of-the-art methods on Colo-Pair dataset. Bold indicates the best and underline the second best.

Method	Market-1501		DukeMTMC-reID		CUHK03
Method	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1
PCB [ 22 ]	81.6	93.8	69.2	83.3	57.5	63.7
MHN [ 4 ]	85.0	95.1	77.2	77.3	76.5	71.7
ISP [ 34 ]	88.6	95.3	80.0	89.6	71.4	75.2
CBDB [ 19 ]	85.0	94.4	74.3	87.7	72.8	75.4
C2F [ 29 ]	87.7	94.8	74.9	87.4	84.1	81.3

TABLE II: Performance comparison with other methods on Person ReID datasets. Bold indicates the best and underline the second best.

Fig. 3: Examples of Top-3 ranking results for some query images of GPF-Net. In each row, images are arranged in descending order according to their similarities with the query on the left.

Fig. 4: Visualization of attention maps of our GPF-Net under different input setting: (a) Original images; (b) CNN-based training method without text information; (c) Our gated progressive fusion strategy. Our model effectively learns global context and meaningful visual features, improving multimodal understanding and enabling automatic discovery of deeper, more representative image features.

Pre-training	Text data	Image data	Colo-Pair $\uparrow$
Pre-training	Text data	Image data	mAP	Rank-1	Rank-5
Baseline	$\times$	$\times$	27.94	19.53	39.13
GPF-Net w/ text	$\checkmark$	$\times$	29.54	20.63	42.81
GPF-Net w/ image	$\times$	$\checkmark$	59.91	67.85	82.64
GPF-Net (Ours)	$\checkmark$	$\checkmark$	68.86	80.17	89.62

TABLE III: Ablation study of different pre-training settings from Colo-Pair dataset. Note that the pre-trained model is then fine-tuned on dataset for downstream polyp ReID task.

Method	ViT [ 3 ]	CVRL [ 16 ]	VT-ReID [ 26 ]	DMCL [ 25 ]	GPF-Net (Ours)
Params	86.6	59.7	86.1	153.3	51.3
FLOPs	17.6	115	50.9	68.7	71.5

TABLE IV: Compared to other object retrieval methods in terms of Params(M) and FLOPs(G).

Multi-Scale Feature Fusion

Multi-scale feature fusion combines representations at different resolutions or receptive fields to improve perception, prediction, and reconstruction.

See all 187 papers

Multi-Modal Object Re-Identification with Dual Semantic Guidance and Global-Local Mutual Modulation
Jul 31, 2026Weixiang Zhou, Xingguo Xu, Yuhao Wang +4Cross-Modal Representation Learning Multimodal Robustness
School of Mathematical Sciences and the Key Laboratory for Computational Mathematics and Data Intelligence of Liaoning Province, Dalian University of Technology, Dalian, 116024, China · School of Future Technology, School of Artificial Intelligence, Dalian University of Technology, Dalian, 116024, China · Department of Radiology and Biomedical Imaging, University of California, San Francisco, 94107, USA +1
ODSS-Net: Orthogonally Decoupled Shared-Specific Representations for Multi-Modal Object Re-Identification
Oct 16, 2026Zhifeng Ma, Yuhong Feng, Chongyi Huang +2Multimodal Disentangled Representation Learning Multimodal Fusion
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China · Tencent, Shenzhen, China
Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance
Jul 26, 2026Weixiang Zhou, Jiabei Zuo, Yuhao Wang +3Cross-Modal Alignment Multimodal Fusion
School of Mathematical Sciences and the Key Laboratory for Computational Mathematics and Data Intelligence of Liaoning Province, Dalian University of Technology, Dalian, 116024, China · School of Computer Science and Technology, Dalian University of Technology, Dalian, 116024, China · School of Future Technology, School of Artificial Intelligence, Dalian University of Technology, Dalian, 116024, China +1

Jul 31, 2026J/K move · Enter open · S save

Multi-Modal Object Re-Identification with Dual Semantic Guidance and Global-Local Mutual Modulation

Weixiang Zhou, Xingguo Xu, Yuhao Wang, Cong Wang+3

School of Mathematical Sciences and the Key Laboratory for Computational Mathematics and Data Intelligence of Liaoning Province, Dalian University of Technology, Dalian, 116024, China · School of Future Technology, School of Artificial Intelligence, Dalian University of Technology, Dalian, 116024, China · Department of Radiology and Biomedical Imaging, University of California, San Francisco, 94107, USA+1

Cross-Modal Representation Learning Multimodal Robustness Cross-Modal Alignment Multimodal Fusion

Open paper AskPDF arXiv

Multi-modal object Re-Identification (ReID) aims to retrieve target instances by leveraging complementary information across modalities. However, existing methods suffer from two challenges. First, they often fail to exploit well-aligned and reliable semantic priors, making them vulnerable to background clutter and cross-modal misalignment. On the other hand, they typically rely on holistic feature modeling, overlooking the synergy between global and local representations. To overcome these limitations, we propose a robust multi-modal ReID framework with dual semantic guidance and global-local mutual modulation, which mainly consists of three key components, namely the Text-Semantic Injector (TSI), the Masked Global-Local Modulator (MGLM), and the Hierarchical MoE Fusion (HMF). The TSI enhances semantic awareness by integrating clean and coherent textual features into visual tokens. The MGLM enables part-aware cross-modal interaction through joint guidance from soft masks and global context, improving fine-grained feature alignment. Finally, the HMF adaptively aggregates multi-spectral features under local semantic supervision, yielding discriminative and robust representations. Extensive experiments on three multi-modal ReID benchmarks demonstrate the effectiveness of the proposed method. The code will be made publicly available at https://github.com/zw-absin/DSGM upon acceptance.

Abstract

Multi-Scale Feature Fusion

Explore similar work

Multi-Modal Object Re-Identification with Dual Semantic Guidance and Global-Local Mutual Modulation

ODSS-Net: Orthogonally Decoupled Shared-Specific Representations for Multi-Modal Object Re-Identification

Multi-Modal Object Re-Identification with Prompt-S6 and Semantic-Aware Knowledge Guidance