PII Redaction Before LLM Prompts: Presidio, Privacy Filter, and a Production Pipeline

The safest prompt is not the one that tells an LLM to ignore personal data. It is the one that never contains data the model does not need. Put a deterministic privacy gateway before the model call, remove unnecessary fields, replace necessary identifiers with stable placeholders, and keep the re-identification map under your control.

This guide targets the practical search gap between a five-line Presidio demo and a broad GDPR policy. It shows the complete path for support tickets, documents, RAG, agents and internal assistants. It also compares the Microsoft-originated Presidio project with OpenAI Privacy Filter and managed cloud DLP services. Tool details were verified on 6 August 2026.

What is the recommended PII redaction pipeline for LLM prompts?

Use seven boundaries: minimize, detect, classify, transform, call, inspect and selectively restore. Run detection inside your trusted environment before the prompt, attachment, retrieved passage or tool result crosses into the model provider's boundary.

1. Minimize at the source. Select only fields required for the task. Do not serialize a complete customer record and redact it later.
2. Detect in layers. Combine schema-aware field rules, checksums and secret patterns with a context-aware PII detector. One method will not cover both an IBAN and a name hidden in prose.
3. Apply a policy per entity and purpose. Block forbidden content, drop unnecessary content and pseudonymize values the task still needs.
4. Create unique typed placeholders. Replace values with tokens such as PERSON_001, EMAIL_001 and CASE_ID_001. Keep repeated references consistent within the authorized scope.
5. Call the model with sanitized context. The model, tracing system and prompt cache receive placeholders, never the lookup table.
6. Inspect the output. Detect new sensitive values, unexpected raw identifiers and unauthorized placeholders before any response, log or tool call leaves the gateway.
7. Restore only at the final authorized boundary. Resolve an allowed placeholder for the intended user and channel. Never re-identify broadly inside logs, agent memory or a downstream tool argument.

trusted application
  -> field minimizer
  -> rules + contextual detector
  -> policy decision: block | drop | tokenize
  -> encrypted token vault
  -> sanitized prompt -> LLM
  -> output detector + authorization
  -> selective re-identification -> user

This gateway complements, rather than replaces, the object and field authorization described in our AI agent data-level access-control guide. Redaction limits what an approved request exposes. Authorization decides whether the request should access the record at all.

Redaction, masking, tokenization and anonymization are not synonyms

Choosing the wrong word creates a dangerous control gap. The difference is whether the value can be recovered and whether the remaining context still identifies a person.

Under GDPR Article 4(5), pseudonymization requires the additional information to be kept separately and protected. The European Data Protection Board's 2025 guidance is explicit that pseudonymized data remains personal data when it can be linked back to a person. Replacing a name with PERSON_001 reduces exposure. It does not move the workflow outside GDPR.

Even irreversible string removal may not anonymize a prompt. A rare job title, exact event date and small town can identify somebody together. Treat the complete context, metadata, attachments and model output as the assessment unit.

Which PII redaction tool should you use in 2026?

Start with application-owned field minimization and deterministic rules. Add Presidio when you need an inspectable, extensible framework. Add a local contextual model when names and free text dominate. Use a managed DLP service only when sending raw input to that cloud is already approved.

Is Presidio still a Microsoft tool?

Presidio was developed at Microsoft and many older results still call it Microsoft Presidio. In 2026 it is transitioning to the independent Data Privacy Stack organization. The project says it remains MIT licensed, its APIs continue, and new container images move from Microsoft's registry to ghcr.io/data-privacy-stack. Calling it Microsoft-originated is accurate. Treating current community governance as a Microsoft support contract is not.

The framework remains a strong default because its Analyzer and Anonymizer combine regex, deny lists, checksums, rules, named-entity recognition and contextual signals with replace, redact, hash and encrypt operators. You can add recognizers for Austrian social insurance numbers, German tax identifiers, Spanish DNI or NIE formats, Chinese identity numbers and your own customer IDs. Presidio's documentation also warns that automated detection cannot find every sensitive value. That warning belongs in your architecture, not only in the footnotes.

When does OpenAI Privacy Filter fit better?

OpenAI Privacy Filter, released in April 2026 under Apache 2.0, is a local bidirectional token classifier. The published model has 1.5 billion total parameters, 50 million active parameters and a 128,000-token context window. It detects eight categories covering private people, addresses, emails, phones, URLs, dates, account numbers and secrets.

That language understanding is useful when a regex cannot tell a private person from a public organization. It is not a universal policy engine. The model card says the base taxonomy is fixed, performance can fall on non-English or out-of-distribution text, and policy changes may need fine-tuning. It is a detector inside the pipeline, not the vault, authorization layer or compliance decision.

When is managed DLP the better buy?

Managed DLP is attractive when the organization already approves the cloud boundary, needs broad detector catalogs, and does not want to operate NLP models. Azure AI Language PII covers text, conversations and native documents. Google Sensitive Data Protection adds cryptographic one-way and reversible tokens with referential integrity. Amazon Comprehend PII provides entity location and asynchronous redaction.

The architectural catch is simple: a cloud detector must see the raw value to redact it. It can protect the downstream LLM provider while becoming a processor of the original data itself. Review region, retention, subprocessor, DPA and transfer terms as part of the EU AI data-residency decision.

Why typed placeholders beat a wall of REDACTED tokens

A model can still solve many tasks when it knows that two references point to the same customer, clinician or contract. Replacing every entity with [REDACTED] destroys that relationship. Typed, unique placeholders preserve it without revealing the value.

Raw:    Maria emailed Dr Chen about account AT00... twice.
Prompt: PERSON_001 emailed PERSON_002 about ACCOUNT_001 twice.

Scope token consistency narrowly. The same value may receive the same placeholder inside one request or approved conversation, but not across every customer and year. Global deterministic tokens create a new tracking identifier. Store mappings with tenant, purpose, expiry and authorized recipient. Encrypt them separately from prompt logs, and delete them when restoration is no longer needed.

Do not let the model invent placeholders that automatically resolve. The restore step should accept only tokens created by the gateway for this request, then verify the intended recipient and output channel. A placeholder copied into a tool argument, memory store or analytics event stays pseudonymized unless a separate policy explicitly permits restoration there.

Where should the gateway sit for chat, RAG and agents?

For RAG, redaction does not replace per-user permissions. A clean passage can still reveal a confidential strategy, and a pseudonymized embedding can remain personal data. Use our RAG permissions architecture for the access problem and this pipeline for data minimization.

How do you test a PII gateway before production?

Do not approve it from a vendor benchmark or ten hand-written prompts. Build an authorized evaluation set that represents your languages, formats, error modes and business identifiers. Use synthetic examples for breadth and carefully governed labeled samples for realism.

• Entity recall by risk tier: measure how often forbidden identifiers leak. A missed health identifier should not be averaged away by easy email matches.
• Precision and utility: measure useful text removed, task success before and after sanitization, and whether entity relationships survive.
• End-to-end leakage: test every input, retrieval, tool, output and logging path, not only the detector function.
• Language and locale: include German compounds, Spanish names, Chinese scripts, transliteration, mixed-language prompts and local identifier formats.
• Adversarial formatting: test spaces, OCR errors, Unicode lookalikes, identifiers split across fields and secrets embedded in code.
• Restoration safety: attempt cross-tenant tokens, expired tokens, invented placeholders and restoration into a forbidden channel.
• Operations: track p50 and p95 latency, timeouts, detector-version drift, vault availability and fail-closed behavior.

Set thresholds per workflow. A public marketing summarizer and a clinical case assistant do not share the same acceptable false-negative rate. In high-risk paths, uncertain detection should block or require review. In lower-risk paths, a false positive can be shown to the user for correction. Never silently fail open because the detector is slow or unavailable.

"A PII detector is a sensor, not a security boundary. The boundary is the gateway that minimizes data, enforces policy, controls re-identification and fails safely when the sensor is uncertain."

Build, buy or combine?

For most production systems, the hybrid is the honest answer: deterministic extraction for known fields, Presidio for configurable entity logic, a contextual model for ambiguous prose, and an application-owned token vault. A two-detector disagreement should become a policy signal, not a reason to pick whichever result is cheaper.

If you are buying an AI platform, add the gateway evidence to the EU AI vendor security questionnaire. If you are building it, Wavect's AI enablement service can map the data flow, implement one bounded workflow and leave your team with tests and operating documentation. Twinsoft AI shows the wider production discipline around an AI system, while AI enablement versus generic AI consulting helps decide whether you need implementation or advice.

Implementation checklist

1. Inventory raw prompts, uploaded files, retrieval sources, tool results, outputs, caches and logs.
2. Define which data classes are blocked, dropped, pseudonymized or allowed for each purpose.
3. Remove unnecessary structured fields before text assembly.
4. Deploy deterministic and contextual detectors inside an approved boundary.
5. Create unique typed tokens and a tenant-scoped encrypted mapping store.
6. Keep raw values and the mapping out of model context, traces and analytics.
7. Inspect model output and authorize each restoration by token, recipient, purpose and channel.
8. Test multilingual false negatives, utility, adversarial formats, latency and failure modes.
9. Record detector versions, policy decisions and aggregate entity counts without raw content.
10. Review the legal basis, DPA, retention, data-subject rights and DPIA need with privacy counsel.

Primary sources

• Presidio documentation and automated-detection warning
• Presidio project transition to Data Privacy Stack
• Presidio and LiteLLM prompt-masking example
• OpenAI Privacy Filter repository and model limitations
• Azure AI Language PII overview
• Google Sensitive Data Protection pseudonymization
• Amazon Comprehend PII detection and redaction
• EDPB Guidelines 01/2025 on pseudonymisation
• OWASP LLM02:2025 Sensitive Information Disclosure