PII Detection

PII: Personally Identifiable Information

PII detection locates details such as names, addresses, health information, and account identifiers so they can be protected before data is shared or processed. Systems range from rules and pattern matching to transformer-based entity taggers and prompted large language models, with OCR and vision methods used for documents and interfaces. They are evaluated on settings including clinical notes, multilingual text, long conversations, and synthetic benchmarks such as PIIBench, where performance depends on domain and identifier type. A central challenge is balancing missed disclosures against unnecessary redaction: contextual ambiguity, distribution shifts, and aggregate scores can hide failures that matter for privacy and data utility.

Latest papers

39 papers7 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine
Oct 25, 2026Vladimir BeskorovainyiPrivacy Leakage in Language Models PII Detection
Protegrity-PII Technical Report
Oct 21, 2026Hardeep Bassi, Hui Ma, Luís Seabra +3PII Detection
Protegrity 1 Menlo Park, CA, USA · Protegrity 2 Stamford, CT, USA · Protegrity 3 Lisbon, Portugal +1
Morpheus: a fully local, defense-in-depth de-identification engine for Greek clinical text - a benchmark evaluation
Oct 18, 2026Filippos-Paschalis Rorris, Nikoleta Stanitsa, Tatiana Rorri +2Clinical NLP PII Detection
Onassis Hospital, Athens, Greece · Great Ormond Street Hospital for Children, London, United Kingdom · Evangelismos General Hospital, Athens, Greece +2
Broad and Efficient PII Detection with Unified Labels
Oct 15, 2026Andy Hsu, Konstantin BerlinPII Detection Responsible AI
Cisco AI Defense
PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI Agents
Oct 12, 2026Zian Yuan, Wei LiGUI Agents PII Detection
TIDE 2.0: an open, model-agnostic engine for keyed de-identification of clinical notes
Oct 5, 2026Jose D. Posada, Somalee Datta, Priya DesaiPII Detection
Technology & Digital Solutions, Stanford Medicine, Stanford, CA, USA
Strong Multilingual Privacy Tagging at Encoder Speed
Sep 29, 2026Jonathan GraehlNamed Entity Recognition Multilingual Language Models
RWS Language Weaver
The Text Beside the Image: Detection, Utility and Leakage for Trustworthy Multimodal Medical Data and Beyond
Sep 27, 2026Andreas Maier, Monica Hinrichs-Mayer, Franziska Weber +4Privacy Auditing PII Detection
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany · Universitätsklinikum Erlangen, Germany +1
ASIRF: An Agentic Framework for Context-Dependent Sensitive Information Redaction
Sep 24, 2026Sudha Priyadarshini, Mohamed Chahine GhanemPrivacy-Preserving Language Models LLM Agents
University of Liverpool, UK · Keele University, UK
Meddies-PII: A Multilingual Framework for Personally Identifiable Information Extraction in Clinical De-identification
Sep 11, 2026Linh Uyen Le, Ho Dinh Tri, Christian Hoang +1Synthetic Data Generation Clinical Information Extraction
Meddies AI, Vietnam
MedDeID enables locally governed clinical-text de-identification from real or synthetic training data
Sep 9, 2026Stig Hellemans, Tom Stroobants, Elyne Scheurwegs +3Clinical NLP PII Detection
Adrem Data Lab, Department of Computer Science, University of Antwerp, Antwerp, Belgium · Antwerp University Hospital (UZA), Edegem, Belgium · Laboratory of Experimental Medicine and Pediatrics (LEMP), University of Antwerp, Antwerp, Belgium
Mind the Gap: Robustness Risks in PII Detection Systems
Sep 3, 2026Adeel Zafar, Slawomir NowaczykDistribution Shift Robustness PII Detection
Center for Applied Intelligent Systems Research, Halmstad University, Sweden
LeakageBench: Document-Level Leakage Risk for Redacting Personally Identifiable Information in Document Images
Sep 2, 2026Vishnu Prasad Vijaya Kumar, Santhosh Venkatesh, Ivan P. YamshchikovPII Detection Data Leakage
Center for Artificial Intelligence and Robotics (CAIRO), Technical University of Applied Sciences Würzburg-Schweinfurt (THWS), Würzburg, Germany · DataX, Frankfurt am Main, Germany
PII-TRACE: A Benchmark for Context-Aware PII Detection in Multi-Turn LLM Conversations
Aug 31, 2026Kaiyuan Zhang, Chuan Wang, Joey Zhong +4PII Detection
Perplexity · Rutgers University · Purdue University
Model Card for OpenAI Privacy Filter
Aug 18, 2026Charles de Bourcy, Sahra Ghalebikesabi, Avi Schwarzschild +22PII Detection Privacy-Preserving ML
OpenAI
Institution-Specific LLM Prompting Recovers PHI That De-identification Systems and Their Gold Standards Both Miss
Aug 17, 2026Daniel Palacios, Matthew Brady Neeley, Angel Adetomike Otto +6PII Detection Electronic Health Records
Quantitative and Computational Biosciences, Baylor College of Medicine, Houston, Texas, USA · Department of Pediatrics, Baylor College of Medicine, Houston, Texas, USA · Jan and Dan Duncan Neurological Research Institute, Texas Children’s Hospital, Houston, Texas 77030, USA +2
Surrogate Substitution Preserves PHI Detectability: A Multi-Detector Equivalence Study
Aug 4, 2026Qiming Bao, Sherry J. H. Feng, Kim Chester Eugenio +1Privacy Auditing PII Detection
Custodian Labs
DialogPII: A multilingual dataset of synthetic dialog transcripts to detect personal information
Jun 29, 2026Roland Roller, Vera Czehmann, Derya Erman +13PII Detection
German Research Center for Artificial Intelligence (DFKI), Berlin, Germany. · Technical University Berlin, Berlin, Germany. · Berlin Institute of Health (BIH), Berlin, Germany. +4
SurrogateShield: Beyond Redaction for High-Utility, Privacy-Preserving LLM Interactions
Jun 28, 2026Sherwin Vishesh JathannaPrivacy-Preserving Language Models LLM Security
Arizona State University Tempe, Arizona, USA
REDACT: A Systematically Controlled Multilingual Benchmark for Personal Information Detection
Jun 18, 2026Guneesh Vats, Anubha Agrawal, Shikha Singhal +5Privacy Auditing PII Detection
ServiceNow
RedactionBench
Jun 17, 2026Sean Brynjólfsson, Shashvat Jayakrishnan, Esha Sali +2LLM Evaluation Contextual Integrity
A10 Networks, Inc.
Redact or Keep? A Fully Local AI Cascade for Educational Dialogue De-Identification
Jun 16, 2026Haocheng Zhang, Zhuqian Zhou, Kirk Vanacore +2AI in Education Privacy-Preserving ML
Cornell University, Ithaca, NY, USA
Detecting Sensitive Personal Information in Japanese Pre-Training Corpora for Large Language Models
Jun 10, 2026Rei Minamoto, Yusuke Oda, Daisuke KawaharaLegal NLP Privacy-Preserving Language Models
Waseda University · Research and Development Center for LLMs, National Institute of Informatics
Fine-Tuning Over Architectural Complexity: Broad-Coverage PII Detection on PIIBench with DeBERTa
May 25, 2026Pritesh JhaFine-Tuning PII Detection
Privacy Policy Enforcement Guardrails for Data-Sensitive Retrieval-Augmented Generation
May 16, 2026Osama Zafar, Alexander Nemecek, Yiqian Zhang +5Data Leakage RAG Security
Dept. of Computer and Data Sciences Case Western Reserve University Cleveland, USA
GLiNER2-PII: A Multilingual Model for Personally Identifiable Information Extraction
May 11, 2026Urchade Zaratiana, Ash Lewis, George Hurn-MaloneyNamed Entity Recognition Document Information Extraction
Fastino Labs
The Moltbook Files: A Harmless Slopocalypse or Humanity's Last Experiment
May 8, 2026William Brach, Federico Torrielli, Stine Lyngsø Beltoft +3Privacy Leakage in Language Models Multi-Agent Systems
University of Turin Dept. of Computer Science Torino, Italy · University of Southern Denmark Odense, Denmark
Addressing Labelled Data Scarcity: Taxonomy-Agnostic Annotation of PII Values in HTTP Traffic using LLMs
May 7, 2026Thomas Cory, Axel KüpperPrivacy Auditing LLM-Assisted Annotation
Technische Universität Berlin
SHIELD: A Diverse Clinical Note Dataset and Distilled Small Language Models for Enterprise-Scale De-identification
May 5, 2026Jose D. Posada, David Love, Somalee Datta +1Healthcare Clinical Information Extraction
Technology & Digital Solutions Stanford Medicine · Stanford Medicine
PIIBench: A Unified Multi-Source Benchmark Corpus for Personally Identifiable Information Detection
Apr 17, 2026Pritesh JhaNamed Entity Recognition PII Detection
Independent Researcher

Oct 25, 2026J/K move · Enter open · S save

Reversible Pseudonymisation of Documents Sent to External LLMs: an Open Bilingual Benchmark and an Engine

Vladimir Beskorovainyi

Privacy Leakage in Language Models PII Detection Privacy-Preserving Language Models

Open paper AskPDF arXiv

Professionals send working documents to external large language models. A common safeguard is reversible pseudonymisation: personal data are replaced with placeholders before the request and restored locally in the answer. We present an open benchmark of five existing open tools and a regular-expression baseline on Russian and UK English, and an open engine built on the two strongest tools. The benchmark excludes each tool's own authors' dataset from its verdict, compares the engine with each tool on identical datasets for detection, reports masking precision, and tests on a held-out set of synthetic documents whose files, hashes, decision rules, run plan and stopping rule were published before any run. Two local LLMs test whether placeholders survive. All evaluation data are synthetic, and the benchmark was run by the author of the engine; code, data, predictions and raw answers are released. On external datasets no existing tool meets all predefined thresholds: in UK English the identifier and leak thresholds are not met, in Russian the identifier threshold is borderline. On the last held-out set of 1,000 documents per language, the best existing tools leave a whole value visible in 61% (Russian) and 58% (UK) of documents. On these documents, from the generator it was developed on, the engine does so in 0.3% [0.1; 0.9] and 0.2% [0.1; 0.7], leaves any part of a value visible in 0.8% and 0.2%, and meets all detection thresholds in both languages. In UK English it also restores at least 99% of its placeholders after 760 answers. In Russian it restores only 94.6% [93.6; 95.6] of name placeholders, against a 99% threshold, mainly because the inflection it borrows from pii-guard produces wrong surname forms. The engine was developed over six earlier held-out sets from the same generator; real documents were not tested.