SetPrint: Black-Box LLM Fingerprinting via Setwise Correspondence of Response Changes
Organizations: Department of Electrical and Computer Engineering and INMC, Seoul National University, Seoul, Republic of Korea
Abstract
Black-box LLM fingerprinting aims to determine whether a target model was derived from a source model when only the output responses to input queries are observable. One way to reveal this relationship is to perturb each query and observe the resulting responses of both models, yielding two corresponding sets of response variations. These sets provide two complementary sources of information: the relations among response changes within each set, and the correspondence of response changes between the two sets. Existing approaches, however, have not fully exploited both. We introduce SetPrint, a training-free method that jointly captures both within-set relations and between-set correspondence. SetPrint expresses each response change relative to the mean change of its own set, and then compares changes from the two models that were induced by the same perturbation. On LeaFBench, SetPrint consistently outperforms existing training-free black-box fingerprinting methods. Further analyses attribute these gains to both within-set relations and between-set correspondence.
Figures & tables
| Method | Access | Supervised | AUROC | TPR@1%FPR | pAUC |
| REEF † | White box | No | |||
| LLMmap † | Black box | Yes | |||
| LLMDNA + SVM ‡ | Black box | Yes | |||
| MET † | Black box | No | |||
| SEF † | Black box | No | |||
| ZeroPrint † | Black box | No |
| Measure | AUROC | TPR@1%FPR |
|---|---|---|
| MMD (Linear) | ||
| MMD (RBF) | ||
| Wasserstein | ||
| CKA (Linear) | ||
| CKA (RBF) | ||
| RSA (Pearson) |
| Permutation | AUROC | TPR@1%FPR |
|---|---|---|
| None (matched) | ||
| Within-base | ||
| Full-set |
Language Model Fingerprinting
Language model fingerprinting identifies and verifies models or their lineage from distinctive outputs, internal representations, or embedded signals.
See all 87 papers