CAAP: Capture-Aware Adversarial Patch Attacks on Palmprint Recognition Models

Renyang Liu; Jiale Li; Jie Zhang; Cong Wu; Xiaojun Jia; Shuxin Li; Wei Zhou; Kwok-Yan Lam

cs.CVApr 8, 2026

CAAP: Capture-Aware Adversarial Patch Attacks on Palmprint Recognition Models

Authors: Renyang Liu, Jiale Li, Jie Zhang, Cong Wu, Xiaojun Jia, Shuxin Li, Wei Zhou, Kwok-Yan Lam, +1 more

Organizations: Institute of Data Science, National University of Singapore, Singapore 117602, Singapore · Centre for Frontier AI Research (CFAR), A*STAR, Singapore, 138634 · School of Cyber Science and Engineering, Wuhan University, China, 430072 · College of Computing and Data Science, Nanyang Technological University, Singapore, 639798 · School of Engineering, Yunnan University, Kunming 650500, China

Abstract

Palmprint recognition is deployed in security-critical applications, including access control and palm-based payment, due to its contactless acquisition and highly discriminative ridge-and-crease textures. However, the robustness of deep palmprint recognition systems against physically realizable attacks remains insufficiently understood. Existing studies are largely confined to the digital setting and do not adequately account for the texture-dominant nature of palmprint recognition or the distortions introduced during physical acquisition. To address this gap, we propose CAAP, a capture-aware adversarial patch framework for palmprint recognition. CAAP learns a universal patch that can be reused across inputs while remaining effective under realistic acquisition variation. To match the structural characteristics of palmprints, the framework adopts a cross-shaped patch topology, which enlarges spatial coverage under a fixed pixel budget and more effectively disrupts long-range texture continuity. CAAP further integrates three modules: ASIT for input-conditioned patch rendering, RaS for stochastic capture-aware simulation, and MS-DIFE for feature-level identity-disruptive guidance. We evaluate CAAP on the Tongji, IITD, and AISEC datasets against generic CNN backbones and palmprint-specific recognition models. Experiments show that CAAP achieves strong untargeted and targeted attack performance with favorable cross-model and cross-dataset transferability. The results further show that, although adversarial training can partially reduce the attack success rate, substantial residual vulnerability remains. These findings indicate that deep palmprint recognition systems remain vulnerable to physically realizable, capture-aware adversarial patch attacks, underscoring the need for more effective defenses in practice. Code available at https://github.com/ryliu68/CAAP.

Figures & tables

Fig. 1: Training framework of CAAP . A universal cross-shaped patch, specified by a fixed mask $M$ and a learnable texture $P$ , is rendered onto the input ROI through ASIT, which predicts input-conditioned rendering parameters. The composited sample is then processed by RaS to model capture-aware variation during training. In parallel, MS-DIFE extracts multi-scale features from the clean and rendered samples to provide the identity-related objective, while the victim recognizer provides the attack objective. The overall optimization is jointly driven by $\mathcal{L}_{\mathrm{adv}}$ , $\mathcal{L}_{\mathrm{id}}$ , $\mathcal{L}_{\mathrm{tv}}$ , and $\mathcal{L}_{\mathrm{vis}}$ .

\mathcal{L}_{\mathrm{adv}}^{(x)}\leftarrow\frac{1}{K}\mathcal{L}_{\mathrm{adv}}^{(x)},\ \mathcal{L}_{\mathrm{id}}^{(x)}\leftarrow\frac{1}{K}\mathcal{L}_{\mathrm{id}}^{(x)}

Algorithm 1 CAAP Training

Fig. 2: Attacking phase of CAAP . After training, the patch texture and the ASIT parameters are fixed. Given a test ROI image $x$ , ASIT predicts the rendering parameters for the current input, and the adversarial sample is generated by a single forward rendering pass without test-time optimization. The resulting sample is then evaluated by the victim recognizer.

x_{\mathrm{adv}}=\mathcal{S}_{\theta_{\mathrm{geo}},\theta_{\mathrm{pho}}}(x,P,M)

Algorithm 2 CAAP Attacking

Attack	VGG-16	ResNet-18	MobileNetV2	ShuffleNetV2	CompNet	CCNet	CO3Net
AdvPatch	35.40	13.81	78.61	27.65	28.54	7.74	28.83
Patch MI	35.22	12.62	74.93	27.37	30.61	8.79	29.17
Patch PGD	35.58	12.75	70.96	30.87	30.03	8.79	29.28
APPA	35.95	33.20	56.66	91.76	11.74	5.86	24.44
CSPA	38.50	43.56	85.41	98.04	25.89	9.61	25.56
AdvLogo	97.30	95.28	73.66	97.94	66.39	2.93	3.60

TABLE I: The untargeted attack success rate (%) on IITD dataset.

Attack	VGG-16	ResNet-18	MobileNetV2	ShuffleNetV2	CompNet	CCNet	CO3Net
AdvPatch	80.39	100.00	82.50	90.91	49.36	85.77	91.32
Patch MI	80.39	100.00	82.50	45.45	52.09	81.63	72.34
Patch PGD	90.20	28.57	82.50	45.45	51.36	80.84	87.58
APPA	82.35	68.57	75.00	81.82	47.51	60.72	73.53
CSPA	98.75	98.92	96.03	95.93	59.00	94.23	89.45
AdvLogo	54.69	51.04	87.70	53.42	22.92	18.41	35.83

TABLE II: The untargeted attack success rate (%) on Tongji dataset.

Attack	VGG-16	ResNet-18	MobileNetV2	ShuffleNetV2	CompNet	CCNet	CO3Net
AdvPatch	31.42	87.84	48.32	97.90	67.16	24.42	92.37
Patch MI	29.51	89.52	40.13	97.90	66.32	20.42	92.80
Patch PGD	21.02	20.63	69.54	25.46	65.47	19.79	92.37
APPA	95.33	30.61	10.08	97.06	53.89	14.00	87.97
CSPA	94.06	97.90	22.48	97.74	87.79	23.37	95.97
AdvLogo	97.88	97.48	73.66	97.90	12.42	10.95	72.88

TABLE III: The untargeted attack success rate (%) on AISEC dataset.

Attack	VGG-16	ResNet-18	MobileNetV2	ShuffleNetV2	CompNet	CCNet	CO3Net
AdvPatch	0.91	0.53	12.48	2.10	4.73	3.17	20.20
Patch MI	0.91	0.40	6.24	0.84	0.81	4.23	20.88
Patch PGD	0.91	0.53	11.49	1.54	1.04	5.16	20.65
APPA	0.91	0.13	5.82	7.42	0.69	0.70	0.23
AdvLogo	0.91	3.73	12.77	21.57	67.94	13.50	31.60
CSPA	1.46	4.66	23.69	27.03	80.62	23.59	76.64

TABLE IV: The targeted attack success rate (%) on IITD dataset.

Attack	VGG-16	ResNet-18	MobileNetV2	ShuffleNetV2	CompNet	CCNet	CO3Net
AdvPatch	0.29	11.53	8.59	17.67	78.31	49.29	99.51
Patch MI	0.05	0.94	2.91	11.39	74.60	67.61	99.33
Patch PGD	0.27	11.50	8.71	17.06	77.89	71.16	99.75
APPA	0.00	0.00	1.54	4.81	32.62	52.91	98.83
AdvLogo	7.97	10.35	6.77	20.52	95.58	97.96	100.00
CSPA	9.70	18.78	13.78	36.05	96.37	99.68	100.00

TABLE V: The targeted attack success rate (%) on Tongji dataset.

Attack	VGG-16	ResNet-18	MobileNetV2	ShuffleNetV2	CompNet	CCNet	CO3Net
AdvPatch	0.00	0.21	0.63	0.63	34.95	0.00	79.87
Patch MI	1.27	0.42	1.89	2.52	38.53	0.00	91.95
Patch PGD	0.00	0.21	0.84	0.42	37.68	0.00	90.89
APPA	1.27	0.00	2.10	0.84	7.16	0.21	66.31
AdvLogo	0.42	1.47	3.36	7.77	78.74	1.89	94.49
CSPA	1.06	2.31	3.36	14.50	90.11	38.53	98.52

TABLE VI: The targeted attack success rate (%) on AISEC dataset.

Fig. 3: Pairwise cross-model transferability. Each heatmap reports ASR (%) when a patch crafted on a source model (rows) is directly transferred to a target model (columns).

Method	MobileNetV2	ShuffleNetV2	ResNet-18	VGG-16	CompNet	CCNet	CO3Net
Patch PGD	22.17	30.17	73.52	87.05	12.22	0.17	13.61
CSPA	43.75	83.67	84.75	96.28	96.98	41.45	96.60
CAAP s	91.03	96.15	83.47	99.18	60.98	7.95	72.49
CAAP c	92.97	96.53	85.30	89.50	98.71	79.25	96.70

TABLE VII: Hold-out transfer ASR (%). Adversarial patches are crafted using an ensemble of six source models and directly evaluated on a single unseen target model.

Method	MobileNetV2	ShuffleNetV2	ResNet-18	VGG-16	CompNet	CCNet	CO3Net
Patch PGD	62.41	61.76	31.56	50.27	17.19	7.17	32.96
CSPA	73.48	54.62	24.10	62.34	20.88	7.76	22.69
CAAP s	97.73	96.79	86.59	88.50	60.07	28.02	74.21
CAAP c	93.77	86.31	92.83	99.27	69.51	27.78	87.16

TABLE VIII: Cross-dataset transfer ASR (%) from Tongji to IITD.

Fig. 4: ASR of CAAP on three palmprint-specific recognizers before and after adversarial training.

Fig. 5: Identity-level ASR of physical untargeted and targeted attacks by CAAP under real print-and-capture acquisition.

Fig. 6: Effect of cross-shaped patch size. ASR (%) is reported against CCNet, CO3Net, and CompNet as the long-arm length varies from 25 to 45, with the short arm fixed to 25% of the long arm.

Fig. 7: Ablation on patch shape. ASR (%) is reported for four patch geometries against three palmprint-specific recognizers.

Fig. 8: Effect of patch placement position. ASR (%) is reported for attention-guided, center, random, and fixed top-left placement.

Setting	Components			Victim model
Setting	ASIT	MS-DIFE	RaS	CCNet	CO3Net	CompNet
Base				63.79	57.59	35.79
ASIT	✓			97.43	97.52	89.52
MS-DIFE		✓		62.24	57.54	36.21
RaS			✓	65.46	57.29	35.94
ASIT+MS-DIFE	✓	✓		97.82	97.47	84.09

TABLE IX: Ablation on component combinations. Base removes ASIT, MS-DIFE, and RaS, whereas All enables all three components.

Fig. 9: Hyper-parameter sensitivity on Tongji. Untargeted ASR (%) is reported for CCNet, CompNet, and CO3Net, together with their mean, under one-at-a-time sweeps of

\lambda_{\mathrm{id}}

\lambda_{\mathrm{vis}}

, and

\lambda_{\mathrm{tv}}

. Note: The patch size is set to 30 in this ablation study to make the differences across settings more discernible.

Explore similar work

Aug 14, 2026

GBU-Palm: A Multimodal Video Dataset and Benchmark for Palm Presentation Attack Detection

Existing palm presentation attack detection (PAD) datasets are often limited by static imagery, restricted acquisition conditions, or insufficient multimodal video data, hindering systematic evaluation across environments, modalities, and attack types. We present GBU-Palm, a large-scale multimodal video dataset and benchmark containing 21,326 videos from 105 subjects and 210 palms across six acquisition environments, including bona fide, Print, and Replay presentations, with 6,310 synchronized RGB-NIR samples. We construct leakage-controlled protocols that separate palm identity and attack lineage and benchmark four representative video architectures under environment-matched and held-out-environment settings. Results reveal substantial architecture-dependent degradation under environmental shift and show that RGB-NIR fusion does not consistently outperform RGB-only input. We further analyze model behavior through true accept (TA), true reject (TR), false accept (FA), and false reject (FR) decomposition, spectral masking, temporal-order intervention, and frozen-backbone NIR probing, revealing distinct failure patterns and evidence utilization across architectures. GBU-Palm provides a unified and challenging benchmark for developing and evaluating robust multimodal palm PAD methods under cross-environment conditions.

Yingjie Ma, Zitong Yu, Wei Jia +2

Shenzhen University · Great Bay University · Dongguan Key Laboratory for Intelligence and Information Technology +3

Multimodal Robustness Multimodal Model Evaluation Multimodal Fusion

Ask PDF

Apr 11, 2026

FlowPalm: Optical Flow Driven Non-Rigid Deformation for Geometrically Diverse Palmprint Generation

Recently, synthetic palmprints have been increasingly used as substitutes for real data to train recognition models. To be effective, such synthetic data must reflect the diversity of real palmprints, including both style variation and geometric variation. However, existing palmprint generation methods mainly focus on style translation, while geometric variation is either ignored or approximated by simple handcrafted augmentations. In this work, we propose FlowPalm, an optical-flow-driven palmprint generation framework capable of simulating the complex non-rigid deformations observed in real palms. Specifically, FlowPalm estimates optical flows between real palmprint pairs to capture the statistical patterns of geometric deformations. Building on these priors, we design a progressive sampling process that gradually introduces the geometric deformations during diffusion while maintaining identity consistency. Extensive experiments on six benchmark datasets demonstrate that FlowPalm significantly outperforms state-of-the-art palmprint generation approaches in downstream recognition tasks. Project page: https://yuchenzou.github.io/FlowPalm/

Yuchen Zou, Huikai Shao, Lihuang Fang +2

Xi’an Jiaotong University · Sichuan Digital Economy Industry Development Research Institute · Southern University of Science and Technology

Synthetic Data Generation Biometric Identification Diffusion-Based Image Generation

Ask PDF

Mar 30, 2026

From Pixels to Reality: Physical-Digital Patch Attacks on Real-World Camera

This demonstration presents Digital-Physical Adversarial Attacks (DiPA), a new class of practical adversarial attacks against pervasive camera-based authentication systems, where an attacker displays an adversarial patch directly on a smartphone screen instead of relying on printed artifacts. This digital-only physical presentation enables rapid deployment, removes the need for total-variation regularization, and improves patch transferability in black-box conditions. DiPA leverages an ensemble of state-of-the-art face-recognition models (ArcFace, MagFace, CosFace) to enhance transfer across unseen commercial systems. Our interactive demo shows a real-time dodging attack against a deployed face-recognition camera, preventing authorized users from being recognized while participants dynamically adjust patch patterns and observe immediate effects on the sensing pipeline. We further demonstrate DiPA's superiority over existing physical attacks in terms of success rate, feature-space distortion, and reductions in detection confidence, highlighting critical vulnerabilities at the intersection of mobile devices, pervasive vision, and sensor-driven authentication infrastructures.

Victoria Leonenkova, Ekaterina Shumitskaya, Dmitriy Vatolin +1

Lomonosov MSU, Moscow, Russia · AI Center, Lomonosov MSU, IAI MSU, Moscow, Russia

Adversarial Attacks Adversarial Patch Attacks Physical Adversarial Attacks Black-Box Adversarial Attacks Transfer-Based Adversarial Attacks Face Recognition

Ask PDF