Redacting Personal Data: A Speed and Accuracy Comparison of Regex, Presidio and Hybrid Designs
Organizations: ManpowerGroup Services India Pvt. Ltd.
Abstract
Many enterprise projects now pass free text to large language models (LLMs), and that text often contains personally identifiable information (PII) such as names, phone numbers, email addresses, home addresses and dates of birth. Once a prompt is sent, this data is copied into provider systems, application logs and observability traces. Redacting PII before the call is an obvious control, yet teams seldom have numbers on what it costs in latency or what it lets through. This paper compares four redaction designs that any LLM pipeline can adopt: a pure regular-expression (regex) redactor, Microsoft Presidio running on a spaCy model, a hybrid in which Presidio finds only person names while regex handles structured identifiers, and a fast hybrid that calls spaCy's named-entity component directly. All four share a known-entity rule, which removes the person the system already knows it is handling, and a consistent placeholder scheme, so the LLM can still tell one person from another after redaction. We evaluate the designs on a sample workflow, which we call ProfileAssist, that turns resumes into structured profiles, using 100 synthetic resumes (523 characters on average, with 900 PII values and 600 business terms that must survive). Regex ran at a 0.20 ms median but caught only 88.9% of PII, because it cannot find names that are not known in advance, such as the references listed in a resume. Presidio, hybrid and fast hybrid each caught 100% of PII and kept 100% of business terms, at median latencies of 37.59 ms, 30.23 ms and 13.48 ms. Measured against a 9,120 ms median LLM call, even the slowest option adds only 0.41% to the request. We therefore recommend the fast hybrid as a default for LLM pipelines that handle personal data, with regex kept as a safety net rather than as the only control.
Figures & tables
| Category | Item | Example | Scored |
|---|---|---|---|
| PII | Subject name (full name and parts) | Andre Kowalczyk, ANDRE KOWALCZYK, Andre | Yes |
| PII | Email address and phone number | [email protected], (901) 555-4821 | Yes |
| PII | Street address and ZIP code | 4521 Maple Ave, 38103 | Yes |
| PII | Date of birth | 04/17/1991 | Yes |
| PII | Reference name and phone number | Greta Fontaine, 412-555-7310 | Yes |
| PII | Profile URL and US SSN | linkedin.com/in/andrekowalczyk, 123-45-6789 | No |
| Entity | What the pattern matches | Used by |
|---|---|---|
| EMAIL_ADDRESS | local part, @, domain and a top-level domain of two or more letters | Regex, Hybrid, Fast hybrid |
| PROFILE_URL | linkedin.com/in/… or github.com/…, with optional scheme and www | Regex, Hybrid, Fast hybrid |
| US_SSN | three, two and four digits joined by hyphens | Regex, Hybrid, Fast hybrid |
| PHONE_NUMBER | ten-digit US number with optional +1, brackets, dots, dashes or spaces | Regex, Hybrid, Fast hybrid |
| STREET_ADDRESS | house number, optional direction, one to three capitalized words, street suffix, optional unit | All four |
| US_ZIP | five- or nine-digit ZIP after a two-letter state code (only the ZIP is replaced) | All four |
| Design | Load (ms) | P50 (ms) | P95 (ms) | PII caught | Kept | LLM share |
|---|---|---|---|---|---|---|
| Regex | 0.8 | 0.20 | 0.49 | 88.9% | 100.0% | 0.002% |
| Presidio | 5,021.2 | 37.59 | 49.96 | 100.0% | 100.0% | 0.41% |
| Hybrid | 1,416.7 | 30.23 | 43.42 | 100.0% | 100.0% | 0.33% |
| Fast hybrid | 1,698.6 | 13.48 | 17.10 | 100.0% | 100.0% | 0.15% |
PII Detection
Personally identifiable information (PII) detection finds sensitive personal details in text, conversations, images, and other data.
See all 41 papers