Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification

Jiachen Li; Xiaojin Gong; Dongping Zhang

cs.CVMar 14, 2026

Multi-Grained Vision-Language Alignment for Domain Generalized Person Re-Identification

Authors: Jiachen Li, Xiaojin Gong, Dongping Zhang

Organizations: College of Information Science and Electronic Engineering, Zhejiang University · College of Information Engineering, China Jiliang University

Abstract

Domain Generalized person Re-identification (DG Re-ID) is a challenging task, where models are trained on source domains but tested on unseen target domains. Although previous pure vision-based models have achieved significant progress, the performance remains further improved. Recently, Vision-Language Models (VLMs) present outstanding generalization capabilities in various visual applications. However, directly adapting a VLM to Re-ID shows limited generalization improvement. This is because the VLM only produces with global features that are insensitive to ID nuances. To tacle this problem, we propose a CLIP-based multi-grained vision-language alignment framework in this work. Specifically, several multi-grained prompts are introduced in language modality to describe different body parts and align with their counterparts in vision modality. To obtain fine-grained visual information, an adaptively masked multi-head self-attention module is employed to precisely extract specific part features. To train the proposed module, an MLLM-based visual grounding expert is employed to automatically generate pseudo labels of body parts for supervision. Extensive experiments conducted on both single- and multi-source generalization protocols demonstrate the superior performance of our approach. The implementation code will be released at https://github.com/RikoLi/MUVA.

Figures & tables

Figure 1: Comparison of the single- and multi-grained alignments. (a) indicates the single-grained alignment. Similar IDs from diverse visual domains can be summarized by a sentence in a global view. Domain-specific attributes such as illumination, view angle, image resolution and so on are eliminated due to the nature of language. However, the single-grained description ignores ID nuances. (b) indicates our multi-grained alignment, where extra fine-grained descriptions of parts are employed to represent more subtle ID differences. Note that exact words in descriptions are used as examples to explain our idea. In implementation, several learnable prompts are adopted for more flexible descriptions instead of manually designed words.

Figure 2: Pipeline of our framework. Before training, the VGE generates part locations for upcoming usage. In stage-1, the multi-grained learnable prompts of each ID are aligned with correponding images. The part locations are directly used to extract local visual features. In stage-2, the proposed AM-MSA module is adopted in each transformer layer to extract local features and optimized with the entire image encoder. In this procedure, the part locations act as pseudo labels to supervise the AM-MSA module. Prototypical memories are utilized to train two stages. In inference, only the image encoder is utilized.

Figure 3: Illustration of the AM-MSA module. (a) indicates its inner architecture. The red path highlights a bypass branch which adaptively generates an attention mask

\mathbf{A}_{l}

for local information aggregation in a transformer. For simplicity, the residual connections in the transformer are omitted. (b) indicates the architecture of the RMP module, where a cross-attention layer, an MLP layer, several layer normalization layers and a sigmoid activation function are used to predict the foreground mask of each part.

Figure 4: Visualization of part locating with VGE on Market1501 [ 89 ] dataset. Different parts are detected in green boxes.

Figure 5: Illustration of the imperfect results. In (a), the distribution of the height of bounding boxes for leg regions from Market1501 [ 89 ] dataset is presented. Blue denotes the raw results and red denotes the calibrated results. Without calibration, we find irregular peaks at values approximating the entire image height, which correspond to incorrect locations containing the whole person body. Extremely small values are also incorrect, which often represent over-splitting. (b) is a sample of oversized partition. (c) and (d) are examples of over-split leg regions. Green boxes are predictions of the VGE.

Figure 6: Illustration of the generation of pseudo label masks. The masks are generated by a binary voting strategy. After that, the masks are flattened in row-wise and stacked together.

Model	Type	MA $\to$ D		MA $\to$ MS		D $\to$ MA		D $\to$ MS		MS $\to$ MA		MS $\to$ D
Model	Type	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1
QAConv ∗ [ 37 ] ${}_{\textit{ECCV'20}}$	Image Matching	28.7	48.8	7.0	22.6	27.2	58.6	8.9	29.0	43.1	72.6	52.6	69.4
TransMatcher ∗ [ 38 ] ${}_{\textit{NeurIPS'21}}$		42.6	61.8	18.4	47.3	46.9	77.5	20.7	52.8	52.0	80.1	50.8	67.9
QAConv-GS ∗ [ 39 ] ${}_{\textit{CVPR'22}}$		39.3	60.5	17.2	45.9	41.1	72.6	17.0	46.9	49.5	79.1	47.5	67.3
QAConv-MS ∗ [ 8 ] ${}_{\textit{TCSVT'24}}$		-	-	19.9	49.7	-	-	-	-	54.5	83.0	-	-
PAT [ 46 ] ${}_{\textit{ICCV'23}}$	Part-based	48.9	67.9	18.2	42.8	45.2	71.9	19.2	43.9	47.3	72.2	54.9	71.4

Table 1: Comparison with state-of-the-art performances under single-source protocol.

*

indicates that the input image size is larger than 256

\times

128.

\dagger

indicates that the method uses target domain data for test-time updating.

\ddagger

indicates that the method uses extra datasets for pre-training. The results marked in gray are produced by running the source code provided by authors, and all others are directly adopted from their original papers. Best and suboptimal results are emphasized in bold and underline, respectively.

Model	Type	MA $\to$ C3		D $\to$ C3		MS $\to$ C3		C3 $\to$ MA		C3 $\to$ D		C3 $\to$ MS
Model	Type	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1
QAConv [ 37 ] ${}_{\textit{ECCV'20}}$	Image Matching	8.6	9.9	6.8	7.9	22.6	25.3	20.1	45.9	15.0	29.4	3.9	14.9
TransMatcher ∗ [ 38 ] ${}_{\textit{NeurIPS'21}}$		21.4	22.2	18.2	18.9	22.5	23.7	38.8	69.7	34.6	56.8	15.2	45.2
QAConv-GS ∗ [ 39 ] ${}_{\textit{CVPR'22}}$		18.1	19.1	14.7	15.1	20.6	20.9	37.9	69.0	34.9	57.5	16.2	47.6
QAConv-MS ∗ [ 8 ] ${}_{\textit{TCSVT'24}}$		24.9	26.6	-	-	28.5	31.0	-	-	-	-	-	-
PAT [ 46 ] ${}_{\textit{ICCV'23}}$	Part-based	26.0	25.4	18.9	18.8	-	-	-	-	-	-	-	-

Table 2: Additional performances under single-source protocol on CUHK03-NP [ 92 ] .

*

indicates that the input image size is larger than 256

\times

128.

\ddagger

Model	Type	D+MS+C3 $\to$ MA		MA+MS+C3 $\to$ D		MA+D+C3 $\to$ MS		MA+D+MS $\to$ C3
Model	Type	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1
QAConv-MS ∗ [ 8 ] ${}_{\textit{TCSVT'24}}$	Image Matching	65.2	87.1	-	-	25.9	57.7	40.3	44.0
PAT [ 46 ] ${}_{\textit{ICCV'23}}$	Part-based	51.7	75.2	56.5	71.8	21.6	45.6	31.5	31.1
ISR ${}^{*}\ddagger$ [ 18 ] ${}_{\textit{ICCV'23}}$	Pre-training	70.5	87.0	-	-	30.3	56.4	37.8	36.6
M 3 L [ 87 ] ${}_{\textit{CVPR'21}}$	Meta-Learning	48.1	74.5	50.5	69.4	12.9	33.0	29.9	30.7
SuA-SpML [ 82 ] ${}_{\textit{TIP'23}}$	Meta-Learning	59.1	83.2	51.9	71.6	20.4	47.1	33.3	34.4

Table 3: Comparison with state-of-the-art performances under multi-source protocol.

*

indicates that the input image size is larger than 256

\times

128.

\dagger

indicates that the method uses target domain data for test-time updating.

\ddagger

Model	Market1501		DukeMTMC-reID		MSMT17
Model	mAP	Rank-1	mAP	Rank-1	mAP	Rank-1
TransReID 256×128 [ 24 ] ${}_{\textit{ICCV'21}}$	88.9	95.2	82.0	90.7	67.4	85.3
DCAL [ 96 ] ${}_{\textit{CVPR'22}}$	87.5	94.7	80.1	89.0	64.0	83.1
AAformer ∗ [ 98 ] ${}_{\textit{TNNLS'23}}$	88.0	95.4	80.9	90.1	65.6	84.4
CLIP-ReID [ 35 ] ${}_{\textit{AAAI'23}}$	89.6	95.5	82.5	90.0	73.4	88.7
PromptSG [ 73 ] ${}_{\textit{CVPR'24}}$	94.6	97.0	81.6	91.0	87.2	92.6

Table 4: Comparison with state-of-the-art Re-ID methods on source domains.

*

indicates that the input image size is larger than

256\times 128

. Note that results on CUHK03-NP [ 92 ] are not presented due to lack of experiments from recent works. Best and suboptimal results are emphasized in bold and underline, respectively.

Model	Granularity				MA $\to$ MS		MS $\to$ MA
Model	G	H	U	L	mAP	Rank-1	mAP	Rank-1
MUVA (w/ G)	✓				26.0	54.7	51.9	77.4
MUVA (w/ G+H)	✓	✓			25.0	53.7	53.4	78.6
MUVA (w/ G+U)	✓		✓		26.4	55.8	52.9	78.8
MUVA (w/ G+L)	✓			✓	27.3	56.2	56.7	81.6
MUVA (w/ G+H+U)	✓	✓	✓		26.9	56.1	55.5	79.9

Table 5: Ablations on the effectiveness of different granularity combinations. “G”, “H”, “U” and “L” indicate global, head, upper body and leg regions, respectively. Best results are emphasized in bold.

Model	Granularity	MA $\to$ MS		MS $\to$ MA
Model	Granularity	mAP	Rank-1	mAP	Rank-1
MUVA (vision-only)	Single-grained	25.2	53.8	52.6	77.1
MUVA (vision-language)	Single-grained	26.0	54.7	51.9	77.4
MUVA (vision-only)	Multi-grained	26.7	55.2	56.8	80.8
MUVA (vision-language)	Multi-grained	27.6	56.8	59.5	82.7

Table 6: Ablations on the effectiveness of the language modality. “vision-only” indicates that the model is only trained with the image encoder. “vision-language” indicates that the model is trained by vision-language alignment. Best results are emphasized in bold.

Model	MA $\to$ MS		MS $\to$ MA
Model	mAP	Rank-1	mAP	Rank-1
MUVA (w/ PAP)	27.7	56.7	58.5	82.2
MUVA (w/ DSP)	27.4	56.4	58.5	82.0
MUVA (original)	27.6	56.8	59.5	82.7

Table 7: Ablations on different prompt designs. “PAP” and “DSP” indicate the uses of part-agnostic prompt and domain-specific prompt, respectively. “original” indicates the use of original prompt in our model. Best results are emphasized in bold.

Model	MA $\to$ MS		MS $\to$ MA
Model	mAP	Rank-1	mAP	Rank-1
MUVA (stripe-like)	25.8	54.3	55.5	80.1
MUVA (AM-MSA)	27.6	56.8	59.5	82.7

Table 8: Ablations on the effectiveness of AM-MSA module. “AM-MSA” indicates our approach using the AM-MSA module to explore parts. “stripe-like” indicates the use of traditional stripe-like partition. Best results are emphasized in bold.

Model	MA $\to$ MS		MS $\to$ MA
Model	mAP	Rank-1	mAP	Rank-1
MUVA (w/o calibration)	27.3	56.1	57.7	81.7
MUVA (w/ calibration)	27.6	56.8	59.5	82.7

Table 9: Ablations on pseudo label mask calibration. Best results are emphasized in bold.

Model	RMP Module		MA $\to$ MS		MS $\to$ MA
Model	LN	RC	mAP	Rank-1	mAP	Rank-1
MUVA (RMP w/o LN+RC)			26.9	55.6	58.4	82.2
MUVA (RMP w/ LN)	✓		27.4	56.1	58.7	82.8
MUVA (RMP w/ RC)		✓	27.4	56.6	59.2	82.6
MUVA (RMP w/ LN+RC)	✓	✓	27.6	56.8	59.5	82.7

Table 10: Ablations on the structure of the RMP module. “LN” indicates the layer normalization. “RC” indicates the use of residual connection from inputs to outputs of the cross-attention. Best results are emphasized in bold.

Model	MA $\to$ MS		MS $\to$ MA
Model	mAP	Rank-1	mAP	Rank-1
MUVA (w/ RCRE)	25.9	55.1	57.4	81.7
MUVA (w/o RCRE)	27.6	56.8	59.5	82.7

Table 11: Ablations on the effectiveness of adopting random cropping and random erasing [ 91 ] as augmentations. “RCRE” indicates two augmentations. Best results are emphasized in bold.

Number of Heads	MA $\to$ MS		MS $\to$ MA
Number of Heads	mAP	Rank-1	mAP	Rank-1
1	27.4	56.1	58.2	81.7
2	27.6	56.8	58.5	82.2
4	27.4	55.9	58.7	82.4
8	27.2	56.0	59.5	82.7
16	26.9	55.7	59.2	82.7

Table 12: Ablations on the number of the cross-attention heads in the RMP module. Best results are emphasized in bold.

Model	Parameters (M)		Latency (ms)			Memory (GB)			MA $\to$ MS		MS $\to$ MA
Model	Train.	Infer.	S1	S2	Infer.	S1	S2	Infer.	mAP	Rank-1	mAP	Rank-1
PAT [ 46 ]	87.1	86.5	-	364.3	23.2	-	5.7	1.1	18.2	42.8	45.8	71.6
CLIP-FGDI [ 84 ]	129.5	86.2	100.9	273.1	22.1	1.6	6.5	0.8	20.2	47.0	44.7	73.0
MUVA (w/o AM-MSA)	126.2	86.1	58.9	152.1	22.1	2.1	4.6	0.7	26.0	54.7	51.9	77.4
MUVA (w/o lang.)	117.4	115.9	-	243.4	71.1	-	6.8	0.9	26.7	55.2	56.8	80.8
MUVA	161.7	115.9	161.2	247.2	69.4	7.2	6.9	0.9	27.6	56.8	59.5	82.7

Table 13: Comparisons of different models on parameter scale, time latency per batch, GPU memory consumption and Re-ID performance. “Train.” and “Infer.” indicate training and inference, respectively. “S1” and “S2” indicate stage-1 and stage-2 training, respectively. The batch size is set to 1 for the VGE and 64 for others.

Figure 7: Visualization results of the part masks generated in the AM-MSA module. The model is trained on MSMT17 [ 67 ] dataset and visualized on the query and gallery sets of Market1501 [ 89 ] dataset. For each sample, the original image and the masks for head, upper body and legs are presented from left to right. Brighter regions indicate higher model focuses.

Figure 8: Some cases of visual grounding on occluded samples from DukeMTMC-reID [ 90 ] dataset. Left group shows slight occlusion. Middle group shows severe occlusion. Right group shows occlusion caused by other people. In each sample, head, upper body and legs are detected from left to right, respectively.

Person Re-Identification

Person re-identification matches the same individual across camera views, times, clothing, and sensing modalities despite changes in appearance.

See all 84 papers

Multimodal LLM-Empowered Re-Ranking for Generalizable Person Re-Identification
Jun 15, 2026Jiachen Li, Xiaojin GongMultimodal Reranking Multimodal Large Language Models
College of Information Science and Electronic Engineering Zhejiang University
ReText: Text Boosts Generalization in Image-Based Person Re-identification
Feb 5, 2026Timur Mamedov, Karina Kvanchiani, Anton Konushin +1Person Re-Identification
Tevian, Moscow, Russia · Lomonosov Moscow State University
ALADIN:Attribute-Language Distillation Network for Person Re-Identification
Mar 23, 2026Wang Zhou, Boran Duan, Haojun Ai +2Person Re-Identification Cross-Modal Knowledge Distillation
Wuhan University No.299 Bayi Road, 430072, PR China

Jun 15, 2026J/K move · Enter open · S save

Multimodal LLM-Empowered Re-Ranking for Generalizable Person Re-Identification

Jiachen Li, Xiaojin Gong

College of Information Science and Electronic Engineering Zhejiang University

Multimodal Reranking Multimodal Large Language Models LLM Reranking Cross-Domain Generalization Person Re-Identification

Open paper AskPDF arXiv

Domain Generalizable (DG) person re-identification (Re-ID) has attracted growing research interest due to its potential for deployment in unseen real-world scenarios. Most existing approaches address DG Re-ID by focusing on training domain-generalizable encoders but ignore the possible refinements in inference stage. In contrast, this work explores an alternative direction which improves inference re-ranking to enhance DG Re-ID. Conventional re-ranking methods typically rely on neighborhood-based distances to refine the initial ranking list, inherently depending on features produced by the Re-ID encoder. However, they deteriorate on target domains since the encoder lacks sufficient generalizability to produce reliable feature distances on unseen scenarios. Inspired by the remarkable generalization capabilities of recent Multimodal Large Language Models (MLLMs), we propose an MLLM-empowered distance metric to improve re-ranking in DG Re-ID. Specifically, we first adapt an MLLM to Re-ID data through supervised fine-tuning, which incorporates a domain-agnostic prompt and a query-candidate hard mining scheme. Then, the adapted MLLM is employed to compute a

μ

-distance during inference, which is robust to domain gap and significantly enhances subsequent re-ranking performance. Our approach is model-agnostic and can be seamlessly integrated into previous re-ranking frameworks. Extensive experiments demonstrate that our approach consistently yields substantial performance improvements across multiple DG Re-ID benchmarks. The code of this work will be released at https://github.com/RikoLi/MUSE soon.

Abstract

Person Re-Identification

Explore similar work

Multimodal LLM-Empowered Re-Ranking for Generalizable Person Re-Identification

ReText: Text Boosts Generalization in Image-Based Person Re-identification

ALADIN:Attribute-Language Distillation Network for Person Re-Identification