Generalist Multimodal LLMs Gain Biometric Expertise via Human Salience

Jacob Piland; Byron Dowling; Christopher Sweet; Adam Czajka

cs.CVMar 17, 2026

Generalist Multimodal LLMs Gain Biometric Expertise via Human Salience

Authors: Jacob Piland, Byron Dowling, Christopher Sweet, Adam Czajka

Organizations: Department of Computer Science and Engineering, University of Notre Dame, Notre Dame, IN 46556, USA · Center for Research Computing, University of Notre Dame, Notre Dame, IN 46556, USA

Abstract

Iris presentation attack detection (PAD) is critical for secure biometric deployments, yet developing specialized models faces significant practical barriers: collecting data representing future unknown attacks is impossible, and collecting diverse-enough data, yet still limited in terms of its predictive power, is expensive. Additionally, sharing biometric data raises privacy concerns. Due to rapid emergence of new attack vectors demanding adaptable solutions, we thus investigate in this paper whether general-purpose multimodal large language models (MLLMs) can perform iris PAD when augmented with human expert knowledge, operating under strict privacy constraints that prohibit sending biometric data to public cloud MLLM services. Through analysis of vision encoder embeddings applied to our dataset, we demonstrate that pre-trained vision transformers in MLLMs inherently cluster many iris attack types despite never being explicitly trained for this task. However, where clustering shows overlap between attack classes, we find that structured prompts incorporating human salience (verbal descriptions from subjects identifying attack indicators) enable these models to resolve ambiguities. Testing on an IRB-restricted dataset of 224 iris images spanning seven attack types, using only university-approved services (Gemini 2.5 Pro) or locally-hosted models (e.g., Llama 3.2-Vision), we show that Gemini with expert-informed prompts outperforms both a specialized convolutional neural networks (CNN)-based baseline and human examiners, while the locally-deployable Llama achieves near-human performance. Our results establish that MLLMs deployable within institutional privacy constraints offer a viable path for iris PAD.

Figures & tables

Figure 1: Experimental pipeline and paper contributions. We start with the generation of synthetic MESH salience from human annotations and corresponding image dataset. This salience, along with the control and human salience, is then combined with our novel prompts for testing. Results are obtained for both models to which we have ethical access and compared with a human and salience-guided CNN baseline.

Panel of Figure 2 — Figure 2: Uniform Manifold Approximation and Projection (UMAP) visualization of iris samples encoded by (left) SigLIP vision-only embeddings and (right) SigLIP + Gemma multimodal embeddings using a simple binary prompt asking whether the iris is “real and healthy” or “synthetic/unhealthy.” Despite never being trained specifically for iris PAD, SigLIP alone achieves partial separation of attack types. However, adding even minimal semantic guidance through Gemma shows much improved visual separation between live vs. spoof discrimination, with clearer cluster boundaries and reduced overlap between classes. This visual separation motivates our investigation into whether general-purpose MLLMs can address specialized biometric security tasks through appropriate prompting.

Panel of Figure 3 — Figure 3: From left to right (with third-party dataset sources, where appropriate): live iris (with no abnormalities) [ 31 ] , StyleGAN2-generated sample, StyleGAN3-generated sample, iris wearing textured contact lens, then printed and re-captured in near infrared light [ 31 ] , synthetic sample generated by a non deep learning-based algorithm [ 51 ] , diseased eye [ 58 ] , glass prosthesis, post-mortem sample [ 60 ] , iris printout [ 13 ] , iris wearing textured contact lens [ 15 ] , and artificial eye [ 30 ] .

Panel of Figure 4 — Figure 4: Authentic (left) and synthetically generated by a StyleGAN2 model (right) iris images along with the expert and non-expert descriptions (“Human Text Annotations” in Fig. 1 ).

Prompt	In-Context	Salience
Type	Learning	Type
Short		None
Short + Human	✓	Human description
Short + Llama	✓	Llama MESH
Short + Gemini	✓	Gemini MESH
Long		None

Table 1: Prompt configurations used in experiments with each of the models considered in this work (Gemini and Llama).

Figure 5: Learning curve analysis of the eight Gemini experiments showing the MSE scores converge before even 25 samples from each attack type.

Attack type	Number of samples
Live iris	30
Artificial	14
Contacts+print	30
Diseased	30
Post-mortem	30
Printout	30

Table 2: Attack types and number of samples.

Model	Prompt variant	MSE
Human Subjects		0.062
Baseline CNN		$0.345\pm 0.041$
	Short	0.416
Gemini	Short + Human	0.183
	Short + Llama	0.273
	Short + Gemini	0.118

Table 3: MSE scores for Gemini and Llama based on the prompt type used. The best prompt variant of the short and long prompts is bolded for each model.

Panel of Figure 6 — Figure 6: Histograms of the test set iris PAD classification confidence values generated by the two MLLMs. We see that Llama is more capable of expressing uncertainty.

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,091 papers

Emergent Morphing Attack Detection in Open Multi-modal Large Language Models
Feb 17, 2026Marija Ivanovska, Vitomir ŠtrucImage Forgery Detection Face Presentation Attack Detection
Faculty of Electrical Engineering, University of Ljubljana, Tržaška cesta 25, Ljubljana, 1000, Slovenia
Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection
Aug 30, 2026Hatef Otroshi Shahreza, Asif Hussain Khan, Peter Lorenz +2Image Forgery Detection Vision Foundation Model Adaptation
Idiap Research Institute, Switzerland · Université de Lausanne, Switzerland
Diff-IrisPAD: Modeling Bona Fide Iris Distributions for Unsupervised Presentation Attack Detection
Oct 21, 2026Geetanjali Sharma, Harsh Yadav, Tanmay Deshpande +3Biometric Identification Reconstruction-Based Anomaly Detection
Indian Institute of Technology Mandi, Himachal Pradesh, India · SAFE Center, Norwegian University of Science and Technology (NTNU), Gjøvik, Norway

Feb 17, 2026J/K move · Enter open · S save

Emergent Morphing Attack Detection in Open Multi-modal Large Language Models

Marija Ivanovska, Vitomir Štruc

Faculty of Electrical Engineering, University of Ljubljana, Tržaška cesta 25, Ljubljana, 1000, Slovenia

Image Forgery Detection Face Presentation Attack Detection Multimodal Large Language Models Zero-Shot Learning Multimodal Model Evaluation

Open paper AskPDF arXiv

Face morphing attacks threaten biometric verification, yet most morphing attack detection (MAD) systems require task-specific training and generalize poorly to unseen attack types. Meanwhile, open-source multimodal large language models (MLLMs) have demonstrated strong visual-linguistic reasoning, but their potential in biometric forensics remains underexplored. In this paper, we present the first systematic zero-shot evaluation of open-source MLLMs for single-image MAD, using publicly available weights and a standardized, reproducible protocol. Across diverse morphing techniques, many MLLMs show non-trivial discriminative ability without any fine-tuning or domain adaptation, and LLaVA1.6-Mistral-7B achieves state-of-the-art performance, surpassing highly competitive task-specific MAD baselines by at least 23% in terms of equal error rate (EER). The results indicate that multimodal pretraining can implicitly encode fine-grained facial inconsistencies indicative of morphing artifacts, enabling zero-shot forensic sensitivity. Our findings position open-source MLLMs as reproducible, interpretable, and competitive foundations for biometric security and forensic image analysis. This emergent capability also highlights new opportunities to develop state-of-the-art MAD systems through targeted fine-tuning or lightweight adaptation, further improving accuracy and efficiency while preserving interpretability. To support future research, all code and evaluation protocols will be released upon publication.

Abstract

Multimodal Large Language Models

Explore similar work

Emergent Morphing Attack Detection in Open Multi-modal Large Language Models

Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection

Diff-IrisPAD: Modeling Bona Fide Iris Distributions for Unsupervised Presentation Attack Detection