LENS: Likelihood Evaluation and Neighbor Smoothing for Attribute-Based Person Retrieval on Unseen Cameras

Nguyen Dang Duy Tan; Vo Thanh Nguyen; Linh T. T. Tran; Huynh Nguyen Phat

cs.CVOct 24, 2026

LENS: Likelihood Evaluation and Neighbor Smoothing for Attribute-Based Person Retrieval on Unseen Cameras

Authors: Nguyen Dang Duy Tan, Vo Thanh Nguyen, Linh T. T. Tran, Huynh Nguyen Phat

Organizations: AI VIET NAM · Ho Chi Minh City University of Technology and Engineering · University of Information Technology, VNU-HCM · University of Science, VNU-HCM

Abstract

Attribute-based person retrieval ranks surveillance images from a binary query of semantic attributes, but mean Average Attribute Degree of Match (mADM) drops on unseen cameras as predicted probabilities shift. Common pipelines rank gallery images by their L1 distance to the query, which penalizes a confident single-attribute mismatch no more than small uncertainties across many attributes, while post-hoc recalibration fitted on source validation data fails to transfer because the per-attribute maps fitted on source data do not match those of new cameras. We propose LENS (Likelihood Evaluation and Neighbor Smoothing), combining fine-tuned DINOv3 and CLIP models with a distilled student. LENS regularizes mutually exclusive attribute groups with group cross-entropy in training, scores gallery images by the log-likelihood of the exact query, and smooths gallery logits over nearest neighbors using DINOv3 appearance features from the same forward pass. On held-out datasets, all three changes raise mADM: the likelihood score improves all six retrieval metrics for the main ensemble and mADM for all 48 development models, and training-free kNN smoothing improves all 46 tested development models. On the hidden test set of the UPAR 2027 benchmark, LENS reaches 68.17 mADM.

Figures & tables

Figure 1: Overview of the LENS retrieval pipeline. (a) Three attribute models process one gallery image $g$ from the held-out PETA fold. Each encoder freezes the patch embedding and first eight blocks (snowflake icon) and fine-tunes the remaining blocks with a query head ( Fig. 2 ). The DINOv3 and CLIP models use BCE and group CE losses, and the student is distilled from two teachers. At inference, their logits are averaged into $z_{g}$ , while the L2-normalized mean of the 98 patch tokens of the DINOv3 model forms $e_{g}$ . (b) Five of the $M=50$ nearest neighbors of $g$ by cosine similarity of $e_{g}$ (shown here with pretrained DINOv3 features, retrieving the same person in other frames) are averaged in kNN smoothing ( Eq. 6 ), flipping the Age-Old logit sign to match the label and improving the rank of $g$ under Eq. 5 from 22 to 10.

Figure 2: Pre-LN cross-attention query head with hidden width $d$ ( $d=1280$ for DINOv3; $d=1024$ for CLIP and the student; see Sec. 4 ). Inset: Age-Old cross-attention weights over the $14\times 7$ grid of $g$ in Fig. 1 .

Fold	Score	mADM	mAP	mINP	R-1	R-5	R-10
PETA	L1 distance [ 36 ]	62.88	24.03	18.67	21.75	39.59	47.22
	likelihood ( 5 )	69.91	35.51	29.08	31.57	54.68	63.07
	likelihood †	69.84	35.38	28.99	31.59	54.17	63.30
	group-softmax †	68.62	35.47	28.77	32.10	54.34	63.41
Market	L1 distance [ 36 ]	69.29	25.08	8.97	32.53	51.46	58.00
	likelihood ( 5 )	74.90	34.39	16.84	37.97	61.30	70.87

Table 1: Scoring rules on the DINOv3 + CLIP ensemble (CLIP with text score and WiSE-FT), in points. Bold marks the better of L1 distance and likelihood. † Same single-seed logits as Tab. 4 .

Figure 3: Top five gallery images of three held-out queries under the L1 distance and under LENS (likelihood score with kNN smoothing). Both rankings start from the same probabilities of the DINOv3 + CLIP models of the fold (trained with group CE on the two other datasets, CLIP without WiSE-FT), and kNN smoothing uses the fine-tuned DINOv3 features of the fold ( $M=50$ , $\alpha=0.2$ ). Green borders mark exact matches, and the labels under the other images give the single mismatched attribute or the number of mismatched attributes. The queries were selected among those with 2 to 8 exact matches for which the L1 distance has no exact match in its top five and LENS has at least three.

	PETA	Market
NLL on the held-out dataset (nat/image)	6.20	5.26
NLL on source datasets (nat/image)	4.48	4.28
Calibration term, held-out dataset	1.11	1.01
Calibration term, source datasets	0.80	1.06
mADM, no recalibration	69.53	74.66
mADM, oracle recalibration	71.95	76.26

Table 2: Recalibration after training for the DINOv3 + CLIP ensemble (seed 1, CLIP with text score, without WiSE-FT) using

\sigma(a_{k}z+b_{k})

per attribute. Oracle recalibration fits on held-out dataset labels, and source recalibration fits on source validation images. The calibration term [ 14 ] is the NLL removed by the per-attribute Platt map fitted on the same images (a lower bound).

Figure 4: Per-attribute Platt parameters

\sigma(a_{k}z+b_{k})

fitted on source vs. held-out labels for the ensemble in Tab. 2 (

y=x

dashed, Pearson

r

, omitting attributes without both classes on the held-out split).

Model	PETA	Market	Mean
DINOv3	66.36	73.87	70.12
DINOv3, two fine-tuned copies	66.69	74.00	70.34
CLIP	65.81	71.72	68.76
CLIP + text score	67.17	72.32	69.74
CLIP + text score + WiSE-FT	67.38	71.98	69.68
DINOv3 + CLIP	68.92	74.46	71.69

Table 3: Held-out mADM (points) of individual models and ensembles under the likelihood score. The second row averages two DINOv3 models fine-tuned independently with the same recipe. The last row is the final setting without group CE, kNN smoothing, and the student.

Figure 5: Improving predicted probabilities on held-out PETA. (a) Training the DINOv3 ViT-L/16 student ( $176\times 96$ ) on hard labels alone vs. distilling from DINOv3-L + CLIP-L teachers across epochs. (b) Held-out NLL per image by attribute group (asterisks mark groups in Eq. 4 ), with a PETA crop whose 12-color probability sum drops from $1.42$ to $1.30$ .

DINOv3 + CLIP ensemble	PETA	Market	Mean
Likelihood score	69.84	74.59	72.22
+ kNN smoothing	70.23	75.31	72.77
+ group CE	70.89	75.28	73.08
+ group CE + kNN smoothing (final)	71.03	75.66	73.35
kNN with pretrained DINOv3	71.00	75.52	73.26
row 4 + query expansion	70.99	75.94	73.47

Table 4: Held-out mADM (points) of the DINOv3 + CLIP ensemble (seed 1) under the likelihood score. Group CE is applied to both models, and kNN smoothing uses

M=50

\alpha=0.2

, and fine-tuned DINOv3 features except in the fifth row. The fourth row is the final setting without the student; the first row differs slightly from the last row of Tab. 3 because this table uses CLIP seed 1 only and a retrained DINOv3 model.

Neighbor feature	$M$ , $\alpha$	PETA	Market	PA100k
no smoothing (mADM)		68.45	78.14	73.86
DINOv3, mean patch token	3, 0.25	$-$ 0.03	+0.67	+0.52
DINOv3, mean patch token	10, 0.25	+0.56	+0.86	+0.67
DINOv3, mean patch token	20, 0.25	+0.64	+0.86	+0.65
DINOv3, mean patch token	50, 0.25	+0.60	+0.85	+0.70
DINOv3, mean patch token	50, 0.2	+0.70	+0.69	+0.66

Table 5: Change in mADM (points) from Eq. 6 relative to the first row on an earlier DINOv3 + CLIP ensemble of the PETA fold (without text score) across the unseen target gallery (held-out PETA) and the two source-domain validation galleries (Market-1501 and PA100k, excluded from training).

Figure 6: Generality and feature analysis of kNN smoothing. (a) $\Delta$ mADM across all 46 development models with saved gallery features ( $M=50$ , $\alpha=0.2$ ) using pretrained DINOv3 features: all 41 PETA models ( $[+0.11,+1.65]$ ) and all 5 Market-1501 models ( $[+1.00,+1.21]$ ) improve. (b) Mean $\Delta$ mADM across the three galleries in Tab. 5 vs. $M$ ( $\alpha=0.25$ ), with a PETA crop $g^{\prime}$ and top-3 neighbors.

Final model	mADM
DINOv3 ViT-B/16 student	63.65
DINOv3 + CLIP + DINOv3 ViT-L/16 student	67.22
+ text score, WiSE-FT, kNN smoothing	67.83
+ group CE for DINOv3 and CLIP	68.17

Table 6: mADM (points) on the hidden test set of the UPAR 2027 benchmark under the likelihood score. All models are trained on the training and validation splits of the three datasets.

Person Re-Identification

Person re-identification matches the same individual across camera views, times, clothing, and sensing modalities despite changes in appearance.

See all 92 papers

Open-Attribute Person Retrieval: Finding People Through Distinctive and Novel Attributes
Aug 2, 2025Minjeong Park, Hongbeen Park, Sangwon Lee +1Information Retrieval Image Retrieval
The Pennsylvania State University · Korea Telecom Research · Korea University +1
UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition
Oct 24, 2026Nguyen Dang Duy Tan, Linh T. T. Tran, Huynh Nguyen Phat +1Feature Fusion Vision Foundation Models
AI VIET NAM · University of Information Technology, VNU-HCM · University of Science, VNU-HCM +1
In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification
Oct 20, 2026Rajarshi Bhattacharya, Aryan Shukla, Shakeeb Murtaza +4Person Re-Identification Vision-Language Retrieval
LIVIA, École de technologie supérieure · Genetec Inc.

Aug 2, 2025J/K move · Enter open · S save

Open-Attribute Person Retrieval: Finding People Through Distinctive and Novel Attributes

Minjeong Park, Hongbeen Park, Sangwon Lee, Jinkyu Kim

The Pennsylvania State University · Korea Telecom Research · Korea University+1

Information Retrieval Image Retrieval

Open paper AskPDF arXiv

Person retrieval in surveillance videos often depends on attributes described by witnesses or operators. However, the most useful cues in practice are not always common appearance descriptions (e.g., gender, clothing color), but rare and distinctive attributes that can sharply reduce the search space (e.g., holding a weapon, lying on the ground). Existing text-based person retrieval benchmarks and methods largely focus on identity-centric retrieval with common pedestrian descriptions, leaving such retrieval-critical attributes underexplored. In this paper, we introduce Open-Attribute Person Retrieval (OAPR), a practical retrieval setting that aims to retrieve all pedestrian instances matching a given attribute query, including rare or previously unseen visual concepts, regardless of identity. To support this task, we construct EPAD, an Expanded Pedestrian Attribute Dataset with 267,885 pedestrian images and a unified vocabulary of 65 attributes, including safety-critical actions, assistive devices, and object interactions that are rarely covered in prior benchmarks. We further propose GAP-CLIP, a lightweight CLIP-based framework that learns gated attribute-aware body-part representations for OAPR. Extensive experiments on EPAD demonstrate that GAP-CLIP achieves the strongest top-K retrieval performance on the full attribute space and on out-of-distribution attributes. The code and dataset are available at https://github.com/mlnjeongpark/Open-Attribute-Person-Retrieval.

Abstract

Person Re-Identification

Explore similar work

Open-Attribute Person Retrieval: Finding People Through Distinctive and Novel Attributes

UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition

In PRAISE of Attributes: Grounding Retrieval Geometry with Query-Conditioned Subspaces for Person Re-Identification