Cross-modal Fuzzy Alignment Network for Text-Aerial Person Retrieval and A Large-scale Benchmark

Yifei Deng; Chenglong Li; Yuyang Zhang; Guyue Hu; Jin Tang

cs.CVMar 21, 2026

Cross-modal Fuzzy Alignment Network for Text-Aerial Person Retrieval and A Large-scale Benchmark

Authors: Yifei Deng, Chenglong Li, Yuyang Zhang, Guyue Hu, Jin Tang

Organizations: State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology · School of Computer Science and Technology, Anhui University · The University of Hong Kong · School of Artificial Intelligence, Anhui University

Abstract

Text-aerial person retrieval aims to identify targets in UAV-captured images from eyewitness descriptions, supporting intelligent transportation and public security applications. Compared to ground-view text--image person retrieval, UAV-captured images often suffer from degraded visual information due to drastic variations in viewing angles and flight altitudes, making semantic alignment with textual descriptions very challenging. To address this issue, we propose a novel Cross-modal Fuzzy Alignment Network, which quantifies the token-level reliability by fuzzy logic to achieve accurate fine-grained alignment and incorporates ground-view images as a bridge agent to further mitigate the gap between aerial images and text descriptions, for text--aerial person retrieval. In particular, we design the Fuzzy Token Alignment module that employs the fuzzy membership function to dynamically model token-level association strength and suppress the influence of unobservable or noisy tokens. It can alleviate the semantic inconsistencies caused by missing visual cues and significantly enhance the robustness of token-level semantic alignment. Moreover, to further mitigate the gap between aerial images and text descriptions, we design a Context-Aware Dynamic Alignment module to incorporate the ground-view agent as a bridge in text--aerial alignment and adaptively combine direct alignment and agent-assisted alignment to improve the robustness. In addition, we construct a large-scale benchmark dataset called AERI-PEDES by using a chain-of-thought to decompose text generation into attribute parsing, initial captioning, and refinement, thus boosting textual accuracy and semantic consistency. Experiments on AERI-PEDES and TBAPR demonstrate the superiority of our method.

Figures & tables

Figure 1: (a) Illustration of the text-aerial person retrieval task. (b) Semantically identical targets with substantial viewpoint discrepancies between ground and aerial perspectives. (c) Incomplete visual cues in aerial images result in only partial alignment with the corresponding captions.

Figure 2: Overview of the proposed Cross-Modal Fuzzy Alignment Network.

Datasets		AERI-PEDES	TBAPR [ 38 ]	CUHK-PEDES [ 23 ]	ICFG-PEDES [ 7 ]	RSTPReid [ 48 ]
Image	Number	144,548	65,880	40,206	54,522	20,505
	Ground & Aerial	$\surd$	$\surd$	$\times$	$\times$	$\times$
	Multi-Sources	$\surd$	$\surd$	$\surd$	$\times$	$\times$
Caption	Average Length	38.6	56.1	23.5	37.2	25.8
	Maximum Length	105	87	96	83	70
	Generated (Train) & Human (Test)	$\surd$	$\times$	$\times$	$\times$	$\times$

Table 1: Comparison AERI-PEDES with other datasets for text-image person retrieval.

Figure 3: (a) Illustrates sample data from the AERI-PEDES benchmark. (b) Shows the text-image similarity distribution in comparison with CUHK-PEDES. (c) Shows the text-image similarity distribution in comparison with RSTPReid.

Figure 4: Chain-of-Thought Based Caption Generation Framework.

Method	Ref	AERI-PEDES					TBAPR
Method	Ref	Rank-1	Rank-5	Rank-10	mAP	RSum	Rank-1	Rank-5	Rank-10	mAP	RSum
IRRA [ 13 ]	CVPR23	35.14	53.23	63.19	33.42	151.57	39.63	58.72	67.69	35.31	166.04
APTM [ 44 ]	MM23	34.62	53.95	64.5	31.09	153.07	43.59	62.03	69.75	38.71	175.37
RDE [ 32 ]	CVPR24	38.56	58.26	67.89	37.16	164.71	37.31	54.06	60.75	32.17	152.12
CFAM [ 49 ]	CVPR24	30.77	51.37	61.61	30.4	143.75	48.34	66.31	73.21	42.67	184.78
NAM [ 36 ]	CVPR24	42.47	61.72	69.99	40.22	174.17	46.56	63.13	70.13	40.92	179.82

Table 2: Comparison results on the AERI-PEDES, and TBAPR datasets.

Methods	Rank-1	Rank-5	Rank-10	mAP	RSum
Baseline	43.88	61.11	69.85	41.58	174.84
+ CDA	46.18	64.40	72.46	43.98	183.04
+ FTA	44.55	61.77	70.32	41.89	176.64
+ CDA + FTA	47.16	65.66	73.83	44.79	186.65

Table 3: Ablation experiments of different components on AERI-PEDES dataset.

Bridge	Rank-1	Rank-5	Rank-10	mAP	RSum
None	45.06	64.53	73.21	43.27	182.80
Aerial	46.08	64.60	73.33	44.20	184.01
Ground	47.16	65.66	73.83	44.79	186.65

Table 4: Impact of Different Bridge Modalities in CDA.

Figure 5: Impact of the Number of Learnable Query Tokens.

$k$	1	2	4	8	12	16
Rank-1	47.16	46.91	47.04	46.77	46.23	46.04
mAP	44.79	44.58	44.50	43.83	43.70	43.55
RSum	186.65	185.88	185.91	184.11	183.99	183.31

Table 5: Sensitivity analysis of the hyperparameter

k

in the context-aware dynamic alignment module.

Image-Text Retrieval

Image-text retrieval matches images with natural-language queries, enabling search across visual collections using either modality.

See all 38 papers

Oct 11, 2025

TCMA: Text-Conditioned Multi-granularity Alignment for Drone Cross-Modal Text-Video Retrieval

Unmanned aerial vehicles (UAVs) have become powerful platforms for real-time, high-resolution data collection, producing massive volumes of aerial videos. Efficient retrieval of relevant content from these videos is crucial for applications in urban management, emergency response, security, and disaster relief. While text-video retrieval has advanced in natural video domains, the UAV domain remains underexplored due to limitations in existing datasets, such as coarse and redundant captions. Thus, in this work, we construct the Drone Video-Text Match Dataset (DVTMD), which contains 2,864 videos and 14,320 fine-grained, semantically diverse captions. The annotations capture multiple complementary aspects, including human actions, objects, background settings, environmental conditions, and visual style, thereby enhancing text-video correspondence and reducing redundancy. Building on this dataset, we propose the Text-Conditioned Multi-granularity Alignment (TCMA) framework, which integrates global video-sentence alignment, sentence-guided frame aggregation, and word-guided patch alignment. To further refine local alignment, we design a Word and Patch Selection module that filters irrelevant content, as well as a Text-Adaptive Dynamic Temperature Mechanism that adapts attention sharpness to text type. Extensive experiments on DVTMD and CapERA establish the first complete benchmark for drone text-video retrieval. Our TCMA achieves state-of-the-art performance, including 45.5% R@1 in text-to-video and 42.8% R@1 in video-to-text retrieval, demonstrating the effectiveness of our dataset and method. The code and dataset will be released.

Zixu Zhao, Yang Zhan, Yunhao Li +1

Carnegie Mellon University, Pittsburgh, PA 15213 USA · The Hong Kong Polytechnic University, Hong Kong, China · Wuhan University, Wuhan 430079, China

Cross-Modal Retrieval Vision-Language Alignment

Ask PDF

Apr 20, 2026

Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation

In text-to-image person retrieval tasks, the diversity of natural language expressions and the implicitness of visual semantics often lead to the problem of Expression Drift, where semantically equivalent texts exhibit significant feature discrepancies in the embedding space due to phrasing variations, thereby degrading the robustness of image-text alignment. This paper proposes a semantic compensation framework (MVR) driven by Large Language Models (LLMs), which enhances cross-modal representation consistency through multi-view semantic reformulation and feature compensation. The core methodology comprises three components: Multi-View Reformulation (MVR): A dual-branch prompting strategy combines key feature guidance (extracting visually critical components via feature similarity) and diversity-aware rewriting to generate semantically equivalent yet distributionally diverse textual variants; Textual Feature Robustness Enhancement: A training-free latent space compensation mechanism suppresses noise interference through multi-view feature mean-pooling and residual connections, effectively capturing "Semantic Echoes"; Visual Semantic Compensation: VLM generates multi-perspective image descriptions, which are further enhanced through shared text reformulation to address visual semantic gaps. Experiments demonstrate that our method can improve the accuracy of the original model well without training and performs SOTA on three text-to-image person retrieval datasets.

Chao Yuan, Yujian Zhao, Haoxuan Xu +1

Beihang University

VLM Robustness Vision-Language Retrieval Vision-Language Alignment

Ask PDF

Jun 29, 2026

Cross-Resolution Semantic Transfer for Robust Text-to-Image Retrieval in Low-Resolution Surveillance

Text-to-image person re-identification (TIPR) retrieves target persons using natural language descriptions. However, existing methods largely overlook resolution variance in real-world surveillance. They characterize cross-resolution TIPR through two coupled failure modes: Evidence Reliability Collapse (ERC), where degraded visual tokens become unreliable for grounding fine-grained text, and Ranking Distribution Drift (RDD), where mixed-resolution galleries distort similarity neighborhoods and destabilize retrieval rankings. To address this challenge, we propose Cross-Resolution Semantic Transfer (CRST), a CLIP-style framework with three modules: resolution-conditioned reasoning, text-guided refinement and CR-RDA. Resolution-conditioned reasoning estimates token reliability to suppress corrupted evidence. Text-guided refinement injects semantic priors to recover discriminative cues. CR-RDA transfers HR neighborhood geometry to stabilize LR ranking under mixed resolutions. Experiments on CUHK-PEDES, ICFG-PEDES, and RSTPReid show that CRST improves ultra-low-resolution Rank-1 and mAP on average by 5.7% and 5.3%, while stabilizing mixed-resolution retrieval without sacrificing high-resolution accuracy.The code will be made publicly available.

Wenjie Qian, Bin Yang, Xiao Wang +4

𝑎School of Computer Science and Technology, Wuhan University of Science and Technology · 𝑏School of Computer Science, National Engineering Research Center for Multimedia Software, Wuhan University · 𝑐Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology

Visual Representation Learning Cross-Modal Retrieval Person Re-Identification

Ask PDF

Abstract

Image-Text Retrieval

Explore similar work

TCMA: Text-Conditioned Multi-granularity Alignment for Drone Cross-Modal Text-Video Retrieval

Towards Robust Text-to-Image Person Retrieval: Multi-View Reformulation for Semantic Compensation

Cross-Resolution Semantic Transfer for Robust Text-to-Image Retrieval in Low-Resolution Surveillance