LENS: Likelihood Evaluation and Neighbor Smoothing for Attribute-Based Person Retrieval on Unseen Cameras

Nguyen Dang Duy Tan; Vo Thanh Nguyen; Linh T. T. Tran; Huynh Nguyen Phat

cs.CVOct 24, 2026

LENS: Likelihood Evaluation and Neighbor Smoothing for Attribute-Based Person Retrieval on Unseen Cameras

Authors: Nguyen Dang Duy Tan, Vo Thanh Nguyen, Linh T. T. Tran, Huynh Nguyen Phat

Organizations: AI VIET NAM · Ho Chi Minh City University of Technology and Engineering · University of Information Technology, VNU-HCM · University of Science, VNU-HCM

Abstract

Attribute-based person retrieval ranks surveillance images from a binary query of semantic attributes, but mean Average Attribute Degree of Match (mADM) drops on unseen cameras as predicted probabilities shift. Common pipelines rank gallery images by their L1 distance to the query, which penalizes a confident single-attribute mismatch no more than small uncertainties across many attributes, while post-hoc recalibration fitted on source validation data fails to transfer because the per-attribute maps fitted on source data do not match those of new cameras. We propose LENS (Likelihood Evaluation and Neighbor Smoothing), combining fine-tuned DINOv3 and CLIP models with a distilled student. LENS regularizes mutually exclusive attribute groups with group cross-entropy in training, scores gallery images by the log-likelihood of the exact query, and smooths gallery logits over nearest neighbors using DINOv3 appearance features from the same forward pass. On held-out datasets, all three changes raise mADM: the likelihood score improves all six retrieval metrics for the main ensemble and mADM for all 48 development models, and training-free kNN smoothing improves all 46 tested development models. On the hidden test set of the UPAR 2027 benchmark, LENS reaches 68.17 mADM.

Figures & tables

Figure 1: Overview of the LENS retrieval pipeline. (a) Three attribute models process one gallery image $g$ from the held-out PETA fold. Each encoder freezes the patch embedding and first eight blocks (snowflake icon) and fine-tunes the remaining blocks with a query head ( Fig. 2 ). The DINOv3 and CLIP models use BCE and group CE losses, and the student is distilled from two teachers. At inference, their logits are averaged into $z_{g}$ , while the L2-normalized mean of the 98 patch tokens of the DINOv3 model forms $e_{g}$ . (b) Five of the $M=50$ nearest neighbors of $g$ by cosine similarity of $e_{g}$ (shown here with pretrained DINOv3 features, retrieving the same person in other frames) are averaged in kNN smoothing ( Eq. 6 ), flipping the Age-Old logit sign to match the label and improving the rank of $g$ under Eq. 5 from 22 to 10.

Figure 2: Pre-LN cross-attention query head with hidden width $d$ ( $d=1280$ for DINOv3; $d=1024$ for CLIP and the student; see Sec. 4 ). Inset: Age-Old cross-attention weights over the $14\times 7$ grid of $g$ in Fig. 1 .

Fold	Score	mADM	mAP	mINP	R-1	R-5	R-10
PETA	L1 distance [ 36 ]	62.88	24.03	18.67	21.75	39.59	47.22
	likelihood ( 5 )	69.91	35.51	29.08	31.57	54.68	63.07
	likelihood †	69.84	35.38	28.99	31.59	54.17	63.30
	group-softmax †	68.62	35.47	28.77	32.10	54.34	63.41
Market	L1 distance [ 36 ]	69.29	25.08	8.97	32.53	51.46	58.00
	likelihood ( 5 )	74.90	34.39	16.84	37.97	61.30	70.87

Table 1: Scoring rules on the DINOv3 + CLIP ensemble (CLIP with text score and WiSE-FT), in points. Bold marks the better of L1 distance and likelihood. † Same single-seed logits as Tab. 4 .

Figure 3: Top five gallery images of three held-out queries under the L1 distance and under LENS (likelihood score with kNN smoothing). Both rankings start from the same probabilities of the DINOv3 + CLIP models of the fold (trained with group CE on the two other datasets, CLIP without WiSE-FT), and kNN smoothing uses the fine-tuned DINOv3 features of the fold ( $M=50$ , $\alpha=0.2$ ). Green borders mark exact matches, and the labels under the other images give the single mismatched attribute or the number of mismatched attributes. The queries were selected among those with 2 to 8 exact matches for which the L1 distance has no exact match in its top five and LENS has at least three.

	PETA	Market
NLL on the held-out dataset (nat/image)	6.20	5.26
NLL on source datasets (nat/image)	4.48	4.28
Calibration term, held-out dataset	1.11	1.01
Calibration term, source datasets	0.80	1.06
mADM, no recalibration	69.53	74.66
mADM, oracle recalibration	71.95	76.26

Table 2: Recalibration after training for the DINOv3 + CLIP ensemble (seed 1, CLIP with text score, without WiSE-FT) using

\sigma(a_{k}z+b_{k})

per attribute. Oracle recalibration fits on held-out dataset labels, and source recalibration fits on source validation images. The calibration term [ 14 ] is the NLL removed by the per-attribute Platt map fitted on the same images (a lower bound).

Figure 4: Per-attribute Platt parameters

\sigma(a_{k}z+b_{k})

fitted on source vs. held-out labels for the ensemble in Tab. 2 (

y=x

dashed, Pearson

r

, omitting attributes without both classes on the held-out split).

Model	PETA	Market	Mean
DINOv3	66.36	73.87	70.12
DINOv3, two fine-tuned copies	66.69	74.00	70.34
CLIP	65.81	71.72	68.76
CLIP + text score	67.17	72.32	69.74
CLIP + text score + WiSE-FT	67.38	71.98	69.68
DINOv3 + CLIP	68.92	74.46	71.69

Table 3: Held-out mADM (points) of individual models and ensembles under the likelihood score. The second row averages two DINOv3 models fine-tuned independently with the same recipe. The last row is the final setting without group CE, kNN smoothing, and the student.

Figure 5: Improving predicted probabilities on held-out PETA. (a) Training the DINOv3 ViT-L/16 student ( $176\times 96$ ) on hard labels alone vs. distilling from DINOv3-L + CLIP-L teachers across epochs. (b) Held-out NLL per image by attribute group (asterisks mark groups in Eq. 4 ), with a PETA crop whose 12-color probability sum drops from $1.42$ to $1.30$ .

DINOv3 + CLIP ensemble	PETA	Market	Mean
Likelihood score	69.84	74.59	72.22
+ kNN smoothing	70.23	75.31	72.77
+ group CE	70.89	75.28	73.08
+ group CE + kNN smoothing (final)	71.03	75.66	73.35
kNN with pretrained DINOv3	71.00	75.52	73.26
row 4 + query expansion	70.99	75.94	73.47

Table 4: Held-out mADM (points) of the DINOv3 + CLIP ensemble (seed 1) under the likelihood score. Group CE is applied to both models, and kNN smoothing uses

M=50

\alpha=0.2

, and fine-tuned DINOv3 features except in the fifth row. The fourth row is the final setting without the student; the first row differs slightly from the last row of Tab. 3 because this table uses CLIP seed 1 only and a retrained DINOv3 model.

Neighbor feature	$M$ , $\alpha$	PETA	Market	PA100k
no smoothing (mADM)		68.45	78.14	73.86
DINOv3, mean patch token	3, 0.25	$-$ 0.03	+0.67	+0.52
DINOv3, mean patch token	10, 0.25	+0.56	+0.86	+0.67
DINOv3, mean patch token	20, 0.25	+0.64	+0.86	+0.65
DINOv3, mean patch token	50, 0.25	+0.60	+0.85	+0.70
DINOv3, mean patch token	50, 0.2	+0.70	+0.69	+0.66

Table 5: Change in mADM (points) from Eq. 6 relative to the first row on an earlier DINOv3 + CLIP ensemble of the PETA fold (without text score) across the unseen target gallery (held-out PETA) and the two source-domain validation galleries (Market-1501 and PA100k, excluded from training).

Figure 6: Generality and feature analysis of kNN smoothing. (a) $\Delta$ mADM across all 46 development models with saved gallery features ( $M=50$ , $\alpha=0.2$ ) using pretrained DINOv3 features: all 41 PETA models ( $[+0.11,+1.65]$ ) and all 5 Market-1501 models ( $[+1.00,+1.21]$ ) improve. (b) Mean $\Delta$ mADM across the three galleries in Tab. 5 vs. $M$ ( $\alpha=0.25$ ), with a PETA crop $g^{\prime}$ and top-3 neighbors.

Final model	mADM
DINOv3 ViT-B/16 student	63.65
DINOv3 + CLIP + DINOv3 ViT-L/16 student	67.22
+ text score, WiSE-FT, kNN smoothing	67.83
+ group CE for DINOv3 and CLIP	68.17

Table 6: mADM (points) on the hidden test set of the UPAR 2027 benchmark under the likelihood score. All models are trained on the training and validation splits of the three datasets.

Person Re-Identification

Person re-identification matches the same individual across camera views, times, clothing, and sensing modalities despite changes in appearance.

See all 92 papers

Aug 2, 2025

Open-Attribute Person Retrieval: Finding People Through Distinctive and Novel Attributes

Person retrieval in surveillance videos often depends on attributes described by witnesses or operators. However, the most useful cues in practice are not always common appearance descriptions (e.g., gender, clothing color), but rare and distinctive attributes that can sharply reduce the search space (e.g., holding a weapon, lying on the ground). Existing text-based person retrieval benchmarks and methods largely focus on identity-centric retrieval with common pedestrian descriptions, leaving such retrieval-critical attributes underexplored. In this paper, we introduce Open-Attribute Person Retrieval (OAPR), a practical retrieval setting that aims to retrieve all pedestrian instances matching a given attribute query, including rare or previously unseen visual concepts, regardless of identity. To support this task, we construct EPAD, an Expanded Pedestrian Attribute Dataset with 267,885 pedestrian images and a unified vocabulary of 65 attributes, including safety-critical actions, assistive devices, and object interactions that are rarely covered in prior benchmarks. We further propose GAP-CLIP, a lightweight CLIP-based framework that learns gated attribute-aware body-part representations for OAPR. Extensive experiments on EPAD demonstrate that GAP-CLIP achieves the strongest top-K retrieval performance on the full attribute space and on out-of-distribution attributes. The code and dataset are available at https://github.com/mlnjeongpark/Open-Attribute-Person-Retrieval.

Minjeong Park, Hongbeen Park, Sangwon Lee +1

The Pennsylvania State University · Korea Telecom Research · Korea University +1

Information Retrieval Image Retrieval

Ask PDF

Oct 24, 2026

UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition

Pedestrian Attribute Recognition (PAR) predicts semantic attributes of individuals, such as gender, age, clothing and accessories, and its descriptions support attribute-based person retrieval and person re-identification. However, accurately recognizing fine-grained attributes across domains remains challenging due to occlusion, appearance variations, and shifts in image distributions. Ensembles of independently trained foundation encoders combine their logits with fixed fusion weights, so every attribute uses the same mix and the encoders are never trained together. We propose UniQuery, which instead jointly fine-tunes DINOv3 and CLIP for attribute-specific fusion. It combines two query heads, one for each encoder, with a joint query head whose attribute queries attend to tokens from both encoders, and the prediction is the average of the logits of the three query heads. We evaluate UniQuery on the benchmark of the UPAR Challenge 2027 at the WACV 2027 Real-World Surveillance workshop, where models are trained on PA100k, PETA and Market-1501 with a shared set of attributes. In leave-one-dataset-out experiments, UniQuery outperforms the independent ensemble in Challenge_Avg on both held-out datasets. Ablations show that learning fusion over independently trained encoders does not improve upon logit averaging in the mean over the two held-out datasets, while freezing the encoders lowers the Challenge_Avg of UniQuery. On the hidden test set of the benchmark, UniQuery achieves a Challenge_Avg of 0.8001.

Nguyen Dang Duy Tan, Linh T. T. Tran, Huynh Nguyen Phat +1

AI VIET NAM · University of Information Technology, VNU-HCM · University of Science, VNU-HCM +1

Feature Fusion Vision Foundation Models

Ask PDF

Oct 20, 2026

In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification

Person re-identification (ReID) ranks an image gallery using either an image query or a textual query. In image-to-image (I2I) ReID, viewpoint, illumination, occlusion, and background changes can make distractors appear more similar than the true match. In text-to-image (T2I) ReID, incomplete or ambiguous descriptions may omit key visual evidence. We study this problem for vision-language model (VLM) based ReID, where attributes such as clothing color and carried objects provide useful fine-grained matching cues. However, directly using attribute-prompt similarities as retrieval scores is unreliable because templated prompts can be biased, correlated, and poorly calibrated. We therefore propose PRompt-guided Attribute-based Inference-time Similarity Enhancement (PRAISE), a query-side test-time adaptation framework for VLM-based ReID. Instead of treating prompts as scores, PRAISE uses a transferable attribute-prompt bank to construct an Attribute-Conditioned Subspace (ACS): a query-dependent low-rank subspace induced by the most compatible attribute prompts. ACS provides a controlled attribute-conditioned geometry for reshaping query--gallery affinities and guiding lightweight inference-time updates without identity or attribute labels. Experiments across multiple I2I- and T2I-ReID benchmarks show that PRAISE consistently improves VLM retrieval backbones and outperforms both generic and ReID-specific TTA baselines.

Rajarshi Bhattacharya, Aryan Shukla, Shakeeb Murtaza +4

LIVIA, École de technologie supérieure · Genetec Inc.

Person Re-Identification Vision-Language Retrieval Test-Time Adaptation of VLMs Representation Geometry in VLMs

Ask PDF

Abstract

Person Re-Identification

Explore similar work

Open-Attribute Person Retrieval: Finding People Through Distinctive and Novel Attributes

UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition

In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification