PFM-VEPAR: Prompting Foundation Models for RGB-Event Camera based Pedestrian Attribute Recognition

Minghe Xu; Rouying Wu; ChiaWei Chu; Xiao Wang; Yu Li

cs.CVMar 20, 2026

PFM-VEPAR: Prompting Foundation Models for RGB-Event Camera based Pedestrian Attribute Recognition

Authors: Minghe Xu, Rouying Wu, ChiaWei Chu, Xiao Wang, Yu Li

Organizations: City University of Macau, Macau SAR, China · Zhuhai College of Science and Technology, Zhuhai, China · Macau University of Science and Technology, Macau SAR, China · School of Computer Science and Technology, Anhui University, Hefei 230601, China

Abstract

Event-based pedestrian attribute recognition (PAR) leverages motion cues to enhance RGB cameras in low-light and motion-blur scenarios, enabling more accurate inference of attributes like age and emotion. However, existing two-stream multimodal fusion methods introduce significant computational overhead and neglect the valuable guidance from contextual samples. To address these limitations, this paper proposes an Event Prompter. Discarding the computationally expensive auxiliary backbone, this module directly applies extremely lightweight and efficient Discrete Cosine Transform (DCT) and Inverse DCT (IDCT) operations to the event data. This design extracts frequency-domain event features at a minimal computational cost, thereby effectively augmenting the RGB branch. Furthermore, an external memory bank designed to provide rich prior knowledge, combined with modern Hopfield networks, enables associative memory-augmented representation learning. This mechanism effectively mines and leverages global relational knowledge across different samples. Finally, a cross-attention mechanism fuses the RGB and event modalities, followed by feed-forward networks for attribute prediction. Extensive experiments on multiple benchmark datasets fully validate the effectiveness of the proposed RGB-Event PAR framework. The source code of this paper will be released on https://github.com/Event-AHU/OpenPAR

Figures & tables

Figure 1: Comparison between existing RGB-Event PAR models and the newly proposed one.

Figure 2: An overview of our proposed RGB-Event pedestrian attribute recognition framework, i.e., PFM-VEPAR. In this paper, we propose a lightweight, frequency-aware Event Prompter that enhances RGB feature learning with minimal computational cost. Given an RGB-Event pair, non-overlapping patches are fed into a ViT backbone and the Event Prompter. An associative memory–enhanced context mining strategy strengthens multi-modal representation, followed by cross-attention fusion and a feed-forward network for attribute prediction.

Figure 3: An illustration of the (left) Modern Hopfield Network and (right) Hopfield layers.

Methods	Publish	mA	Acc	Prec	Recall	F1	Test Time	Params (MB)	Flops (GB)	Size (MB)	#MS (MB)
#01 DeepMAR [ 17 ]	ACPR15	66.57	69.53	74.90	88.54	81.57	64s	23	25	181	592
#02 ALM [ 30 ]	ICCV19	57.18	64.17	75.59	73.20	74.38	813s	17	8	66	785
#03 Strong Baseline [ 13 ]	arXiv20	73.75	61.86	67.23	80.78	75.43	47s	24	25	91	576
#04 RethinkingPAR [ 13 ]	arXiv20	81.37	80.84	86.31	87.57	86.93	44s	24	24	91	580
#05 SSCNet [ 12 ]	ICCV21	63.10	66.07	72.72	83.22	77.62	49s	24	9	90	588
#06 VTB [ 5 ]	TCSVT22	88.41	83.83	87.89	89.31	88.53	243s	93	66	333	525

Table 1: Comparison with public methods on EventPAR datasets. Test time refers to inference time. Size denotes the storage size of the saved model checkpoint file on disk, and #MS represents the peak GPU memory footprint required during the training optimization process.

Methods	Backbone	Acc	Prec	Recall	F1
Zhou et al. [ 51 ]	ConvNext	65.51	78.42	77.40	77.91
VTB-PLIP [ 54 ]	ResNet50	43.95	60.80	58.52	59.10
Rethink-PLIP [ 54 ]	ResNet50	40.25	55.41	56.13	55.77
SSPNet [ 50 ]	ResNet50	67.53	78.54	79.78	79.15
MambaPAR [ 37 ]	Vim	61.55	75.54	74.84	74.24
PFM-VEPAR(Ours)	VTB-B/16	67.58	80.21	78.13	79.16

Table 2: Results on DukeMTMC-VID-Attribute RGB-Event based PAR dataset.

Figure 4: Component Analysis of PFM-VEPAR.

RGB	Event	mA	Acc	Prec	Recall	F1
1	5	90.05	84.67	88.26	90.18	89.21
2	5	89.34	84.32	87.99	90.03	89.00
3	5	89.63	84.83	88.32	90.33	89.31
4	5	89.95	84.58	88.23	90.09	89.15
5	5	89.70	84.58	88.12	90.21	89.15

Table 3: Comparison of different input settings.

Transform	Layers	mA	Acc	Prec	Recall	F1
DCT/IDCT	8	87.24	84.32	88.42	89.53	88.97
DCT/IDCT	10	87.41	84.39	88.46	89.60	89.03
DCT/IDCT	6 8	88.43	85.14	88.92	90.04	89.48
DCT/IDCT	8 10	88.37	85.01	88.86	89.95	89.40
DCT/IDCT	6 8 10	88.17	85.10	88.88	90.04	89.46
DFT/IDFT	6 8	87.66	83.10	87.22	89.32	88.25

Table 4: Comparison of Different Localizations and Layers of Event prompter.

Images	RGB	Event	mA	Acc	Prec	Recall	F1
100	✓	✓	90.05	84.67	88.26	90.18	89.21
100	✓		89.42	84.53	88.20	90.10	89.14
100		✓	88.23	84.47	88.43	89.75	89.08
200	✓	✓	89.42	84.51	88.09	90.18	89.12
200	✓		88.89	84.40	88.27	89.93	89.05
200		✓	88.17	84.36	88.35	89.70	89.02

Table 5: Comparison of different samples of RGB/Event Memory Augmentation.

Figure 5: Visualization of some predicted pedestrian attributes using our proposed Model PFM-VEPAR.

Event-Based Vision

Event-based vision uses asynchronous camera measurements to perceive motion and scene changes with high temporal resolution and dynamic range.

See all 232 papers

UniPAR: A Unified Framework for Pedestrian Attribute Recognition
Mar 5, 2026Minghe Xu, Rouying Wu, Jiarui Xu +5Unified Multimodal Models
Faculty of Data Science, City University of Macau, Macau SAR, China · School of Big Data, Zhuhai College of Science and Technology, Zhuhai 519041, China · Faculty of Innovation Engineering, Macau University of Science and Technology, Macau SAR, China +1
UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition
Oct 24, 2026Nguyen Dang Duy Tan, Linh T. T. Tran, Huynh Nguyen Phat +1Feature Fusion Vision Foundation Models
AI VIET NAM · University of Information Technology, VNU-HCM · University of Science, VNU-HCM +1
Real-time pedestrian attribute recognition with YOLOv8 and ResNet18
Jun 19, 2026Houssam El Mir
College of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, 310023, Zhejiang, China

Mar 5, 2026J/K move · Enter open · S save

UniPAR: A Unified Framework for Pedestrian Attribute Recognition

Minghe Xu, Rouying Wu, Jiarui Xu, Minhao Sun+4

Faculty of Data Science, City University of Macau, Macau SAR, China · School of Big Data, Zhuhai College of Science and Technology, Zhuhai 519041, China · Faculty of Innovation Engineering, Macau University of Science and Technology, Macau SAR, China+1

Unified Multimodal Models

Open paper AskPDF arXiv

Pedestrian Attribute Recognition is a foundational computer vision task that provides essential support for downstream applications, including person retrieval in video surveillance and intelligent retail analytics. However, existing research is frequently constrained by the ``one-model-per-dataset" paradigm and struggles to handle significant discrepancies across domains in terms of modalities, attribute definitions, and environmental scenarios. To address these challenges, we propose UniPAR, a unified Transformer-based framework for PAR. By incorporating a unified data scheduling strategy and a dynamic classification head, UniPAR enables a single model to simultaneously process diverse datasets from heterogeneous modalities, including RGB images, video sequences, and event streams. We also introduce an innovative phased fusion encoder that explicitly aligns visual features with textual attribute queries through a late deep fusion strategy. Experimental results on the widely used benchmark datasets, including MSP60K, DukeMTMC, and EventPAR, demonstrate that UniPAR achieves performance comparable to specialized SOTA methods. Furthermore, multi-dataset joint training significantly enhances the model's cross-domain generalization and recognition robustness in extreme environments characterized by low light and motion blur. The source code of this paper will be released on https://github.com/Event-AHU/OpenPAR

Abstract

Event-Based Vision

Explore similar work

UniPAR: A Unified Framework for Pedestrian Attribute Recognition

UniQuery: Unifying Vision Models with a Joint Query Head for Cross-Domain Pedestrian Attribute Recognition

Real-time pedestrian attribute recognition with YOLOv8 and ResNet18