Beyond Redaction: Replacing PII Without Destroying It

When PromptGuard detects a Social Security Number in a prompt, the default behavior is redaction:

Input:  "My SSN is 123-45-6789. Can you help with my tax return?"
Output: "My SSN is [SSN_REDACTED]. Can you help with my tax return?"

The privacy risk is gone. So is some of the model's ability to do its job.

What redaction costs

[SSN_REDACTED] is not a value. It is a hole with a label on it, and it breaks three things:

Reference tracking. If the same email appears twice, redaction produces two identical holes. The model cannot tell whether it is looking at one person mentioned twice or two people mentioned once.

The reply. This is the part people underestimate. The model answers about a hole, so the answer has a hole in it. "I've sent the confirmation to [EMAIL_REDACTED]" is what your user reads. You destroyed the data on the way in and there is nothing to put back on the way out.

Structure. A redacted document stops parsing. Anything downstream that expected a field where the hole is now fails.

Reversible placeholders

PromptGuard's tokenize mode replaces each detected value with a placeholder that can be reversed:

Input:     "Email jane@corp.com and bob@corp.com. Call jane@corp.com back. SSN 123-45-6789."
Tokenized: "Email ⟦PG:PII:EMAIL:0⟧ and ⟦PG:PII:EMAIL:1⟧. Call ⟦PG:PII:EMAIL:0⟧ back. SSN ⟦PG:PII:SSN:0⟧."

Three properties fall out of that, and they are the whole feature:

1. Reference consistency

jane@corp.com becomes ⟦PG:PII:EMAIL:0⟧ both times. bob@corp.com gets its own ordinal. The model can now reason about "the first person" and "the second person" correctly, because the text still distinguishes them. Redaction cannot do this; every hole looks like every other hole.

2. The reply is whole again

Because the mapping is reversible, PromptGuard restores the real values in the response before it reaches your user:

Model sees:   "I'll email ⟦PG:PII:EMAIL:0⟧ with the summary."
Your user gets: "I'll email jane@corp.com with the summary."

This is the difference that matters operationally. With redaction you are choosing between exposing PII to the provider and shipping a broken answer. With tokenization you do neither.

Streaming works too. A placeholder split across two SSE chunks is reassembled before either chunk is forwarded — the restorer buffers a trailing fragment that might be the start of a placeholder, caps that buffer so an unclosed bracket cannot make it grow without bound, and decodes UTF-8 incrementally so a multi-byte character split across a chunk boundary is never mangled. A placeholder is never leaked to the client.

3. The map is transient by construction

The placeholder-to-PII table is exactly the secret you are trying to minimize, so it lives for one request and is never written down. It is not persisted, not logged, and not attached to the security event — it travels on its own transient field specifically so that it cannot ride along into storage. A test pins that.

Why the delimiters look like that

⟦ and ⟧ are U+27E6 and U+27E7, mathematical white square brackets. That choice is deliberate and it is a security property, not a style preference.

Placeholders are the one string in the pipeline that means "substitute the real value here." If an attacker can write one into a prompt, they can try to make the restorer paste a value they were never given. So the delimiters have to be characters that essentially never occur in natural prompts, source code, JSON, or model output — otherwise you cannot distinguish a placeholder you created from one the user typed.

Choosing a mode

pii_detection.mode takes four values:

Set it per project, from the dashboard or the API:

guardrails:
  pii_detection:
    enabled: true
    level: strict
    mode: tokenize

One rough edge worth knowing: the CLI's policy apply validator has not caught up and still rejects tokenize, so set this mode from the dashboard or the API until it does.

Tokenization is the right default for anything that writes back to a user about their own data — support assistants, scheduling, anything that confirms an address or contact. Redaction remains the right default for classification, summarization, and analysis, where the reply never needs to name the value.

One caveat worth stating plainly: restoration happens on the proxy, which is what holds the map for the life of the request. If you call the Guard API directly to scan text, you get the tokenized text back and no map — that endpoint scans, it does not forward and restore.

What this is not

It is not format-preserving fake data. A tokenized SSN does not look like an SSN, and a model asked "is this SSN valid?" cannot answer from the placeholder. If your use case genuinely needs the model to reason about the shape of a value rather than its identity, tokenization is the wrong tool and no mode here is the right one.

It is also not a substitute for output scanning. Restoring values into the reply happens after the response has been scanned; the guardrails still run.

The point

Redaction treats PII as something to destroy. That is correct when the value is genuinely not needed, and it is the wrong default the moment the model has to say the value back.

The real tension was never privacy against functionality. It was that destroying data is a one-way operation and we were applying it to a round trip.