Localizing Text Anonymization for Trustworthy AI: Extending RAT-Bench to Malaysian Microdata and PII
Organizations: Payments Network Malaysia Sdn Bhd (PayNet) · Imperial College London
Abstract
As governments and organizations adopt foundation models for public services and workplace productivity, sensitive citizen, employee, and administrative text may enter LLM workflows through inference, retrieval, fine-tuning, or local model development. We examine whether text anonymizers transfer to new deployment settings by extending RAT-Bench to Malaysia using local microdata, Malaysian PII formats, and culturally grounded transcripts in Malaysian English and Bahasa Malaysia. We evaluate NER- and LLM-based anonymizers using an LLM attacker that infers attributes from anonymized text, measuring both re-identification success and text utility. Across Malaysian English and Bahasa Malaysia, the two LLM anonymizers provide the strongest explicit privacy-utility trade-offs, reducing Easy/Hard re-identification risk to 23-29% while preserving BLEU scores of 0.77-0.94. Non-LLM tools show sharper failure modes, either preserving utility while leaving high residual risk or reducing risk through severe over-redaction, especially in Bahasa Malaysia. These findings suggest that anonymizers should be evaluated against deployment-relevant languages, identifier formats, and cultural contexts before being relied on in public-sector, workplace, or local AI pipelines.
Figures & tables
| Method | Malaysian English | Bahasa Malaysia | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| (% ) | BLEU ( ) | (% ) | BLEU ( ) | |||||||
| Easy | Hard | Avg. | Implicit | Avg. | Implicit | Easy | Hard | Avg. | Avg. | |
| No Anonymization | 100 | 91 | 96 | 35 | 1.00 | 1.00 | 100 | 96 | 98 | 1.00 |
| Presidio | 40 | 50 | 45 | 2 | 0.82 | 0.85 | 52 | 28 | 40 | 0.17 |
| Scrubadub | 64 | 72 | 68 | 11 | 0.77 | 0.82 | 68 | 61 | 65 | 0.41 |
| GliNER | 66 | 57 | 62 | 2 | 0.87 | 0.66 | 66 | 63 | 65 | 0.49 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| RAT-Bench field | Meaning in RAT-Bench/PUMS | Malaysia IPUMS field(s) | Adaptation |
|---|---|---|---|
| ST | U.S. state of residence | BPLS, BPLC | Malaysia IPUMS does not provide current state of residence in our extract. We use Malaysian state of birth as the closest available geographic proxy. For foreign-born records, BPLC provides country of birth instead of a Malaysian state. |
| SEX | Sex | SEX | Used directly. |
| AGE | Age | AGE | Used directly. Following RAT-Bench, we generate date of birth by sampling a valid month and day consistent with calendar constraints, then assigning a birth year from age. Date of birth is generated from age using 1 January 2000 as the reference date. For age values coded as 99+, we randomly assign an age between 99 and 110 before generating date of birth. |
| RAC2P | Race / ethnicity | ETHNIC | Mapped to Malaysian ethnicity categories, including Malay, Chinese, Indian, other Bumiputera, Others, and non-citizens. |
| MAR | Marital status | MARST | Used directly. |
| SCHL | Educational attainment | EDLEV, DEGREE | Merged into a single education variable to better approximate the granularity of PUMS SCHL . We use DEGREE when a valid certificate value exists; otherwise we fall back to EDLEV . |
| Identifier type | Generation approach |
|---|---|
| Name | Generated using Claude Sonnet 4.5 with state, ethnicity, gender, citizenship status, and, for non-citizens, country-of-origin context. Prompts cover Malay, Chinese Malaysian, Malaysian Indian, other Bumiputera (non-Malay indigenous groups), mixed-heritage, and foreign-name patterns. |
| Generated from synthetic names using common email-provider domains such as Gmail, Outlook, Hotmail, and Yahoo. | |
| Address | Generated with an LLM from Malaysian address-style examples, constrained to the assigned state and postcode. Real address examples are used only as style seeds; final addresses are synthetic. |
| Credit card | Generated using card schemes relevant to Malaysia: Visa, Mastercard, American Express, and UnionPay. Generated numbers pass the Luhn check. |
| Phone number | Generated as mobile or landline numbers. Mobile numbers use Malaysian 01x prefixes; landlines use state-conditioned area codes. |
| NRIC/passport | Malaysian citizens born in Malaysia receive synthetic NRIC numbers consistent with date of birth, state code, and gender. Non-citizens and permanent residents receive passport-style identifiers conditioned on country of origin where possible. |
| Method | (% ) | BLEU ( ) |
|---|---|---|
| No anonymization | 7 | 1.00 |
| Presidio | 1 | 0.20 |
| Scrubadub | 0 | 0.49 |
| GliNER | 0 | 0.52 |
| UniNER | 5 | 0.89 |
| TEM | 5 | 0.32 |
| Method | Malaysian English | BM | ||||||
|---|---|---|---|---|---|---|---|---|
| Easy | Hard | Avg. | Impl. | Easy | Hard | Avg. | Impl. | |
| No anonymization | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 | 1.00 |
| Presidio | 0.81 | 0.83 | 0.82 | 0.85 | 0.16 | 0.18 | 0.17 | 0.20 |
| Scrubadub | 0.76 | 0.77 | 0.77 | 0.82 | 0.39 | 0.42 | 0.41 | 0.49 |
| GliNER | 0.86 | 0.88 | 0.87 | 0.66 | 0.48 | 0.49 | 0.49 | 0.52 |
| UniNER | 0.65 | 0.64 | 0.65 | 0.96 | 0.86 | 0.87 | 0.87 | 0.89 |
Privacy-Utility Tradeoff
Privacy-utility tradeoffs study how to limit disclosure of sensitive information while preserving useful data, models, and services.
See all 75 papers