UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition

Nguyen Dang Duy Tan; Linh T. T. Tran; Huynh Nguyen Phat; Vo Thanh Nguyen

cs.CVOct 24, 2026

UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition

Authors: Nguyen Dang Duy Tan, Linh T. T. Tran, Huynh Nguyen Phat, Vo Thanh Nguyen

Organizations: AI VIET NAM · University of Information Technology, VNU-HCM · University of Science, VNU-HCM · Ho Chi Minh City University of Technology and Engineering

Abstract

Pedestrian Attribute Recognition (PAR) predicts semantic attributes of individuals, such as gender, age, clothing and accessories, and its descriptions support attribute-based person retrieval and person re-identification. However, accurately recognizing fine-grained attributes across domains remains challenging due to occlusion, appearance variations, and shifts in image distributions. Ensembles of independently trained foundation encoders combine their logits with fixed fusion weights, so every attribute uses the same mix and the encoders are never trained together. We propose UniQuery, which instead jointly fine-tunes DINOv3 and CLIP for attribute-specific fusion. It combines two query heads, one for each encoder, with a joint query head whose attribute queries attend to tokens from both encoders, and the prediction is the average of the logits of the three query heads. We evaluate UniQuery on the benchmark of the UPAR Challenge 2027 at the WACV 2027 Real-World Surveillance workshop, where models are trained on PA100k, PETA and Market-1501 with a shared set of attributes. In leave-one-dataset-out experiments, UniQuery outperforms the independent ensemble in Challenge_Avg on both held-out datasets. Ablations show that learning fusion over independently trained encoders does not improve upon logit averaging in the mean over the two held-out datasets, while freezing the encoders lowers the Challenge_Avg of UniQuery. On the hidden test set of the benchmark, UniQuery achieves a Challenge_Avg of 0.8001.

Figures & tables

Figure 1: UniQuery on one PETA image. Striped parts with a lock are frozen, light tints mark trained parts, gray boxes are operations without parameters, and token cells are colored by the PCA of the patch tokens. In the text-score branch, $P$ is the pretrained CLIP projection, $e$ the projected and normalized CLS token, $w_{k}$ the vector of the sentence for attribute $k$ minus that of a person , and $\delta_{k}$ and $b_{k}$ are learned. In training, each query head has its own loss with the shift $s$ ( Eq. 1 ). At inference, the logits of the three query heads are averaged and compared with a threshold of 0.5. The final model, trained on the training and validation splits, saw this image in training.

Figure 2: One query head. The three query heads of UniQuery share this design and differ only in the tokens they read. The inset is the attention of the hat query in the DINOv3 query head on the image of Fig. 1 , averaged over the eight attention heads.

Figure 3: The idea of the text score for the hat attribute, on PETA validation images. The text score of an image is the vertical distance from its point to the diagonal. The image without a hat is more similar to the hat sentence than the image with a hat, but it lies below the diagonal and so gets a lower text score.

Model	PETA	Market	Avg.
Independently trained DINOv3	81.46	86.31	83.89
Independently trained CLIP	81.44	85.50	83.47
Ensemble of the two encoders, seed 1	82.45	86.30	84.38
Ensemble of the two encoders, seed 2	82.29	86.33	84.31
UniQuery, DINOv3 query head only	81.26	86.07	83.67
UniQuery, CLIP query head only	81.45	85.56	83.51

Table 1: Held-out Challenge_Avg with the same loss as in Eq. 1 . Avg. is the mean of the two folds, and the final setting is UniQuery with its three query heads averaged. The row “DINOv3 + CLIP query heads” averages these two query heads of the same model, without the joint query head. The variant adds a text score for clothing colors on the upper or lower body region, with a different set of sentences, and in training hides each encoder from the joint query head with probability 0.25.

Figure 4: The joint query head next to the two single-encoder query heads, for the attributes on which these two differ by at least 2 points and that have at least 50 positive images. Values are means over two seeds of the variant in Tab. 1 .

Figure 5: How the joint query head uses the two encoders. (a) Attention of the attention heads that read DINOv3 and of those that read CLIP, on eight validation images that the final model saw in training. (b, c) Statistics over the 6,963 held-out PETA images, for the PETA-fold model.

Logit-adjusted loss, $\tau=0.42$
Encoders (logits averaged)	PETA	Market	Avg.
CLIP	81.79	85.72	83.75
CLIP + CLIP (two seeds)	81.80	85.78	83.79
DINOv3	81.14	86.78	83.96
DINOv3 + CLIP	82.29	86.49	84.39
Weighted BCE

Table 2: Held-out Challenge_Avg of separately fine-tuned encoders with averaged logits. These runs come from an earlier stage, trained with

\tau=0.42

instead of 0.32 in Eq. 1 , or with weighted BCE ( Eq. 1 without

s_{a}

), because only this stage has two fine-tuned copies of the same encoder. The two DINOv3 copies are two runs of the same recipe. In the weighted-BCE block, the CLIP and DINOv3 rows are the mean of the two copies.

Logit average plus a correction learned from
Mix of the two models	PETA	Market	Avg.	Source
Logit average, nothing learned	82.45	86.30	84.38	89.46
Learned ratio per attribute	82.43	86.31	84.37	89.49
DINOv3 attribute vectors	81.15	85.99	83.57	90.93
CLIP attribute vectors	81.55	86.06	83.81	90.52
both, concatenated into an MLP	81.27	85.92	83.60	91.02

Table 3: Learning the mix on top of the two separately fine-tuned models of the ensemble, which stay fixed (held-out Challenge_Avg). Corrections are computed by a linear layer and start at zero. They are trained with the loss of Eq. 1 on 85% of the validation images of the two source datasets (about 22k images in the PETA fold and 14k in the Market fold). The Source column is the two-fold mean Challenge_Avg on the remaining 15% of source images, which the fusion heads do not see in training.

Encoder pair	Training	PETA	Market	Avg.
DINOv3 + CLIP (UniQuery)	fine-tuned	82.54	86.39	84.46
DINOv3 + CLIP	frozen	81.07	85.80	83.43
C-RADIOv4-H + CLIP	fine-tuned	81.87	86.43	84.15
DINOv3 + C-RADIOv4-H	fine-tuned	82.11	86.50	84.31
DINOv3 + SigLIP2-L	fine-tuned	81.81	86.07	83.94

Table 4: Freezing or replacing the encoders of UniQuery (held-out Challenge_Avg). With frozen encoders, only 32.6M parameters are trained.

Figure 6: Zero-shot AUC of the text score on validation images, one point per attribute. Points more than 0.05 below the diagonal are named.

Model	Loss	Challenge_Avg
Independent ensemble	weighted BCE †	0.7893
Independent ensemble	logit-adjusted	0.7989
UniQuery, variant	logit-adjusted	0.7992
UniQuery (final)	logit-adjusted	0.8001

Table 5: Challenge_Avg on the hidden test set of the UPAR 2027 benchmark (scale 0 to 1). Final models are trained on the training and validation splits of all three datasets. All models use a threshold of 0.5 without post-processing, and the variant is the one in Tab. 1 . † With WiSE-FT [ 36 ] for CLIP.

Feature Fusion

Feature fusion combines complementary representations across modalities, scales, layers, or feature types to improve prediction and perception.

See all 119 papers

UniPAR: A Unified Framework for Pedestrian Attribute Recognition
Mar 5, 2026Minghe Xu, Rouying Wu, Jiarui Xu +5Unified Multimodal Models
Faculty of Data Science, City University of Macau, Macau SAR, China · School of Big Data, Zhuhai College of Science and Technology, Zhuhai 519041, China · Faculty of Innovation Engineering, Macau University of Science and Technology, Macau SAR, China +1
PFM-VEPAR: Prompting Foundation Models for RGB-Event Camera based Pedestrian Attribute Recognition
Mar 20, 2026Minghe Xu, Rouying Wu, ChiaWei Chu +2Event-Based Vision Multimodal Sensor Fusion
City University of Macau, Macau SAR, China · Zhuhai College of Science and Technology, Zhuhai, China · Macau University of Science and Technology, Macau SAR, China +1
Uncertainty-Aware Pedestrian Attribute Recognition via Evidential Deep Learning
Apr 29, 2026Zhuofan Lou, Shihang Zhang, Fangle Zhu +2Evidential Deep Learning Epistemic Uncertainty
College of Electronics and Information Engineering, Sichuan University, China

Mar 5, 2026J/K move · Enter open · S save

UniPAR: A Unified Framework for Pedestrian Attribute Recognition

Minghe Xu, Rouying Wu, Jiarui Xu, Minhao Sun+4

Faculty of Data Science, City University of Macau, Macau SAR, China · School of Big Data, Zhuhai College of Science and Technology, Zhuhai 519041, China · Faculty of Innovation Engineering, Macau University of Science and Technology, Macau SAR, China+1

Unified Multimodal Models

Open paper AskPDF arXiv

Pedestrian Attribute Recognition is a foundational computer vision task that provides essential support for downstream applications, including person retrieval in video surveillance and intelligent retail analytics. However, existing research is frequently constrained by the ``one-model-per-dataset" paradigm and struggles to handle significant discrepancies across domains in terms of modalities, attribute definitions, and environmental scenarios. To address these challenges, we propose UniPAR, a unified Transformer-based framework for PAR. By incorporating a unified data scheduling strategy and a dynamic classification head, UniPAR enables a single model to simultaneously process diverse datasets from heterogeneous modalities, including RGB images, video sequences, and event streams. We also introduce an innovative phased fusion encoder that explicitly aligns visual features with textual attribute queries through a late deep fusion strategy. Experimental results on the widely used benchmark datasets, including MSP60K, DukeMTMC, and EventPAR, demonstrate that UniPAR achieves performance comparable to specialized SOTA methods. Furthermore, multi-dataset joint training significantly enhances the model's cross-domain generalization and recognition robustness in extreme environments characterized by low light and motion blur. The source code of this paper will be released on https://github.com/Event-AHU/OpenPAR

Abstract

Feature Fusion

Explore similar work

UniPAR: A Unified Framework for Pedestrian Attribute Recognition

PFM-VEPAR: Prompting Foundation Models for RGB-Event Camera based Pedestrian Attribute Recognition

Uncertainty-Aware Pedestrian Attribute Recognition via Evidential Deep Learning