Advanced Guardrails
Prompt protection, data-loss protection and context integrity must run after Prompt Template, Prompt Decorator and RAG Injection. Inspection has to happen on the prompt exactly as it will reach the model; a guardrail that runs earlier cannot see content added afterwards.
Personal-data masking is deliberately exempt. Whether masking runs before or after RAG is your own data-boundary decision, and Apinizer never moves that policy. Placing masking before RAG keeps the caller's personal data from ever reaching the embedding provider.
If you save the policy list in an order that breaks this rule, Apinizer corrects the order when you save and tells you what was moved and why. This keeps the order you see on the Develop screen identical to the real execution order shown in tracing.
Guardrail Types
Detects and masks personal data — such as Turkish national ID numbers, IBANs, and phone numbers — in both regular requests and streamed responses.
Detects jailbreak and injection attempts in incoming prompts before they reach the model. Can be paired with an external LLM judge — see External Provider Guardrails.
Detects and masks secrets — such as API keys, certificates, and JWTs — that could otherwise leak through a request or response.
Flags or blocks requests that fall outside the topics you define for a gateway, such as a support bot being used for unrelated chit-chat. Can be paired with an external LLM judge — see External Provider Guardrails.
Detects the same request being sent repeatedly within a short time window, such as a client or agent stuck retrying.
Blocks — or optionally truncates the oldest conversation turns instead of blocking — a request that exceeds a configured token count or character limit. See Per-Request Size Limit.
Detects structural injection attempts — fabricated conversation turns or fake system/assistant markers embedded in user content to manipulate the model.
Checks whether an LLM's response is actually supported by the RAG context this proxy injected into the same request, using an external LLM judge — see Groundedness Protection.
MCP and A2A Gateways
Five of the eight guardrails above — Personal Data Masking, Prompt Protection, Data-Loss Protection (DLP), Off-Topic Protection, and Retry-Storm (Loop) Protection — aren't limited to AI Gateways: you can attach them to an MCP Gateway or an A2A Gateway as well, and they run through the exact same policy pipeline. What they inspect differs by surface, since neither protocol carries an OpenAI-format chat body:
- On an MCP Gateway, a guardrail scans the JSON-RPC tool-call arguments (
params.arguments) on the request side, and the tool result content on the response side. - On an A2A Gateway, a guardrail scans the JSON-RPC message parts (
params.message.parts[]— text and other part types) on the request side, and the task/message result envelope on the response side.
Oversized Protection, Context-Integrity Protection, Groundedness Protection, and every other AI policy type — Semantic Cache, Token Quotas and Rate Limiting, Prompt Decorator, Prompt Templates, RAG Injection, and the AI routing policy itself — stay AI-Gateway only: they depend on an actual LLM invocation or on an OpenAI-format chat message array, neither of which an MCP or A2A Gateway has. Attaching one of these to an MCP or A2A Gateway is rejected when you save.
Decoding Encoded Content
Sending a prompt as base64, hex, percent-encoding (%41) or \u0041 changes the text a guardrail sees without changing the text the model understands. Models learned to decode these during pre-training, so the payload still reaches its target, while a pattern-based guardrail sees nothing but a meaningless character run on the wire. OWASP does not treat this as a separate threat: it classifies encoding as a delivery method that carries prompt injection (LLM01) past input filters.
For that reason Personal Data Masking, Prompt Protection, Data-Loss Protection and Off-Topic Protection each offer a Decode encoded content option. When it is on, the guardrail decodes encoded segments of the prompt before scanning and inspects the decoded text as well.
With the option off, behaviour is byte-for-byte what it is today. Nothing about your existing proxies changes unless you turn it on.
Decoded content is inspected, never rewritten
The decoded text is scanned but never written back into the request body. That has one practical consequence: when personal data or a secret is found inside encoded content, the request is rejected rather than masked.
The reason is a technical constraint. Writing a masked value back into a base64 blob corrupts the payload the caller sent, and with nested or partial encodings there is no reliable way to map a byte back to its position. That leaves two options: forward the encoded personal data unmasked — which is exactly the leak masking exists to prevent — or reject the request. Apinizer rejects.
Consistently with that, a Data-Loss Protection rule whose action is mask is escalated to block when it matches decoded content. A flag action is left alone, because flagging never claimed to modify anything in the first place.
Prompt Protection and Off-Topic Protection have no such escalation: both only ever detect, so whatever your rule specifies is what happens.
Guarding against false matches
Hashes, UUIDs and API keys also look like base64 or hex. Decoding them produces byte soup, and reporting that as "hidden content" would flag nearly every request. Apinizer checks how much of the decoded output is readable text and silently discards anything below the threshold, so an ordinary prompt containing a SHA-256 digest or a random key is unaffected.
A JWT payload that decodes to readable JSON, by contrast, is inspected — that is not a false match but the intended behaviour: personal data inside the claims is in scope too.
Limits
- Compressed content (gzip, deflate) is not decoded. This is deliberate: every supported encoding shrinks when decoded, so there is no risk of exponential growth; compression would remove that guarantee.
- Nested encodings are decoded up to three levels, and there are ceilings on input size, candidate count and total decoded text. When a ceiling is reached the scan continues with a partial result and records the fact in the trace — it is never truncated silently.
- Streaming responses are not decoded chunk by chunk; the buffered scan path is covered.
- A failed decode never rejects the request: the scan of the original text has already run, and decoding only widens coverage. Sending deliberately malformed base64 cannot be used to force a rejection.
Turning It On From the Screen
All four guardrails (Personal Data Masking, Prompt Protection, Data-Loss Protection, Off-Topic Protection) carry a Decode Encoded Content switch on their policy screen:
| Guardrail | Where the switch lives |
|---|---|
| Personal Data Masking | Mask Settings section |
| Data-Loss Protection (DLP) | General Information section |
| Prompt Protection / Off-Topic Protection | Content Scope section |
Turn it on, save the policy, and click Deploy. The same setting is also reachable through the APIops REST API (API Reference: AI Gateway) or by setting decodeEncodedContent to true in the policy JSON.
Message Role Scope
Prompt Protection and Off-Topic Protection can each choose which message roles (system, user, assistant, tool) get scanned or compared, through three switches: include system prompts, include assistant prompts, include tool prompts.
These three switches share the same names on both guardrails, but their default direction is reversed — don't configure one from habit formed on the other:
| Prompt Protection | Off-Topic Protection | |
|---|---|---|
| Default (all three unset) | Every role is scanned — today's full-scan behavior | Only the last user message is compared — today's behavior |
| What the switch means | An exclude switch: turning it off narrows the scan | An include switch: turning it on widens the comparison |
| Turning a switch off | Removes that role from the scan | (already the default — no effect) |
| Turning a switch on | (already the default — no effect) | Adds that role to the comparison and triggers the mode switch below |
Prompt Protection: Exclude Switches
Prompt Protection scans every message in the request today, regardless of role — these three switches exist to narrow that scan, not widen it. includeSystemPrompts covers more than role: "system" messages: it also gates the Anthropic top-level system field, since both are the same system prompt arriving through two different surfaces, and one switch governs both.
- An unrecognized role is never excluded. A
developerrole, afunctionrole, a provider-specific role, or a message with norolefield at all is always scanned — these three switches only ever gatesystem/assistant/tool. A guardrail must never drop attacker-controlled text just because it doesn't recognize the role label. - An empty scan falls back to the raw body. If role filtering leaves nothing to scan (for example, a request that contains only a
systemmessage withincludeSystemPromptsoff), the guardrail does not silently skip the scan — it scans the raw request body instead. The net effect: these switches can only narrow what part of the request is scanned via the structured path, never make the guardrail scan nothing (fail-closed).
Off-Topic Protection: Include Switches + Mode Switch
Off-Topic Protection compares only the conversation's last user message against the topic embeddings today. Turning any one of the three switches on moves the guardrail from "last message" mode to "whole conversation" mode — the text fed into the comparison is no longer a single message, but the entire conversation including the roles you selected.
The mode switch changes not just what's in scope but the size of the text going into the embedding call, which shifts the similarity distribution — a whole conversation produces a more "diluted" embedding than a single message. After turning on any role switch, verify your existing Similarity Threshold still draws the right line, and account for the larger embedding call when you do.
Scope: AI/OpenAI-Anthropic Format Only
These three switches apply only to the AI-proxy arm that reads the OpenAI/Anthropic messages array (and, for Prompt Protection, the Anthropic system field). The same policy attached to an MCP Gateway or A2A Gateway scans that protocol's own content shape (JSON-RPC arguments / parts[]) instead, and these switches have no effect there.
Turning It On From the Screen
Prompt Protection and Off-Topic Protection carry three switches in the same Content Scope section as the Decode Encoded Content switch above: Include System Messages, Include Assistant Messages, Include Tool Messages. The same settings are also reachable through the APIops REST API (API Reference: AI Gateway) or by setting includeSystemPrompts / includeAssistantPrompts / includeToolPrompts in the policy JSON.
Personal-Data Types
Personal-data masking recognizes the following built-in types out of the box, and you can add your own custom patterns for anything installation-specific:
| Type | Example | Validation |
|---|---|---|
| Turkish National ID (TCKN) | 12345678950 | Checksum |
| IBAN (Turkey) | TR330006100519786457841326 | Checksum (mod 97) |
| Phone (Turkey) | 05321234567 | Structural |
user@example.com | Structural | |
| Credit Card | 4111111111111111 | Checksum (Luhn) |
| IP Address | 203.0.113.5 | Structural (IPv4/IPv6) |
| URL | https://example.com/path | Structural |
| Passport Number | P1234567 | Structural only |
| Social Security Number (US) | 123-45-6789 | Structural |
| Driver License Number (Turkey) | A12345678 | Structural only |
| IBAN (Generic, any country) | DE89370400440532013000 | Checksum (mod 97) |
| Crypto Wallet Address | 0x71C7656EC7ab88b098defB751B7401B5f6d8976 | Checksum (Bitcoin / Ethereum) |
The Turkish and checksum-validated types produce far fewer false positives than a plain regular expression, since a random string of digits has to actually satisfy the check-digit formula to match.
Unlike most of the other built-in types, passport numbers and Turkish driver's license numbers have no reliable check digit to validate against — these two match on structure only (a letter followed by digits, or a generic alphanumeric code), so they carry a materially higher false-positive rate. If you enable either broadly, pair it with a custom pattern that also requires a matching field name, rather than scanning free-form text for it.
Managing the Rule List
A new personal-data masking policy opens with an empty rule list — the same behaviour as every other Apinizer form: you add only the rules you want. The bulk actions above the list are:
| Action | What it does |
|---|---|
| Add Default Set | Adds one masking rule for every built-in type listed above, in a single click. Rules that are already defined are skipped, so pressing it twice never duplicates the list. |
| Add from Preset | Adds the patterns you pick from the PII Patterns catalog. The box in the modal header selects the whole catalog at once. |
| Add Definition | Creates a single rule from scratch. |
| Clear All | Removes every rule in one step, after a confirmation. Also the quickest way to reset the pre-filled Default PII Mask policy that ships with a newly created AI proxy. |
Rule-list changes are not persisted until you save the policy. A policy left with an empty rule list masks nothing — the screen flags this as a warning.
Data-Loss Protection Patterns
Data-loss protection ships with ready-made patterns for common secret formats, and you can add your own custom patterns for your organization:
| Pattern | Example |
|---|---|
| AWS access key | AKIAIOSFODNN7EXAMPLE |
| OpenAI API key | sk-... |
| GitHub personal access token | ghp_... |
| Slack token | xoxb-... |
| Google API key | AIza... |
| PEM private key block | -----BEGIN PRIVATE KEY----- |
| JWT (JSON Web Token) | eyJhbGci... |
| Generic password/secret/API-key assignment | password: ..., api_key=... |
For streamed responses, data-loss protection scans the response in chunk groups. Once a match triggers a block in one chunk group, every remaining chunk group in that stream is masked as well — this keeps a secret that happens to be split across chunk boundaries from leaking.
Built-in Pattern Versioning
The built-in patterns above ship as a versioned, checksummed package rather than being fixed in the product's code — in the same spirit as Prompt Protection's own signature pack described further down this page, applied here to secret-detection patterns instead. The package's integrity is verified automatically as part of an Apinizer upgrade: if a pattern in the package doesn't match its recorded checksum — a sign the package was corrupted or tampered with — none of its patterns are applied, and a warning is written to the application logs rather than the mismatch failing silently or blocking the upgrade. Patterns already in effect from a previously verified package are left exactly as they are.
A verified package only ever touches the built-in patterns Apinizer itself ships — a custom pattern you added under your own name is never overwritten or removed by it, no matter what changes in a later package.
Editing a built-in pattern protects it from the next package refresh. Built-in patterns are no longer read-only — an admin can edit or delete any of them from the DLP preset screen (or via APIops) the same as a custom pattern. The first time you edit one, it is flagged internally as overridden; from that point on, package verification treats that row exactly like a custom pattern and never overwrites your edit again, even if a later Apinizer upgrade changes the same pattern in the shipped package. Deleting a built-in pattern is permanent — it does not come back on the next upgrade unless a future release ships an explicit re-seed.
Unlike Prompt Protection's signature-pack banner (installed version, checksum, an "up to date" badge), the Data-Loss Protection pattern list does not show a package version on screen today — verification happens automatically in the background during an upgrade, and a failure is visible only in the application logs.
Mandate Coverage: HIPAA and PCI DSS
Beyond the handful of secret patterns shown above, the built-in package also ships dedicated patterns aimed at two common compliance mandates: the identifiers listed in the HIPAA Safe Harbor de-identification standard (45 CFR §164.514(b)(2)) and the cardholder and authentication data categories defined by PCI DSS. Coverage here is measured by mandate, not by a raw pattern count — a mandate can require detecting many different kinds of data, and some of them simply cannot be matched reliably by a regular expression.
HIPAA Safe Harbor — 18 identifiers
| Identifier | Coverage |
|---|---|
| Telephone numbers | Pattern-matched |
| Fax numbers | Pattern-matched |
| Email addresses | Pattern-matched |
| Social Security numbers | Pattern-matched |
| Medical record numbers | Pattern-matched |
| Health plan beneficiary numbers | Pattern-matched |
| Account numbers | Pattern-matched |
| Certificate / license numbers | Pattern-matched |
| Vehicle identifiers and serial numbers, including license plates | Pattern-matched |
| Device identifiers and serial numbers | Pattern-matched |
| Web URLs | Pattern-matched |
| IP addresses | Pattern-matched |
| Names | Needs the content-safety layer — see the note below |
| Geographic subdivisions smaller than a state | Needs the content-safety layer |
| Dates tied to an individual, other than year | Needs the content-safety layer |
| Any other unique identifying number, characteristic, or code | Needs the content-safety layer (the password/API-key/secret shape of this catch-all is already covered by the generic secret pattern) |
| Biometric identifiers | Out of scope — not representable as scannable text |
| Full-face photographic images | Out of scope — not representable as scannable text |
PCI DSS — cardholder and sensitive authentication data
| Element | Coverage |
|---|---|
| Primary Account Number (PAN) | Pattern-matched |
| Card expiration date | Pattern-matched |
| Full magnetic-stripe track data (Track 1 / Track 2) | Pattern-matched |
| PIN / PIN block | Pattern-matched |
| Card verification code (CVV / CVC / CAV2 / CID) | Pattern-matched |
| Cardholder name | Needs the content-safety layer |
| Magnetic-stripe service code | Out of scope — indistinguishable from any other 3-digit number |
A handful of the identifiers above — a person's name, a home address, a birth date, a cardholder's name — are free text with no fixed structure. A regular expression cannot reliably tell "John Smith" apart from any other pair of capitalized words, so detecting this class of value belongs to a semantic layer instead of a pattern-matching one: see Content Safety Categories (AILuminate) below. Apinizer deliberately routes these identifiers there rather than forcing a pattern match that would either miss most real values or flag nearly everything — claiming they were "covered by regex" would be misleading.
Beyond the Two Mandates: Presidio-Aligned Coverage
The built-in package also ships patterns for a handful of identifiers that fall outside both mandates above but are commonly flagged by general-purpose PII detectors — this part of the package is aligned with Microsoft Presidio's recognizer catalog (MIT-licensed; Apinizer's regular expressions are its own, adapted with attribution rather than copied verbatim from Presidio):
| Identifier | Coverage |
|---|---|
| US driver's license number | Pattern-matched, context-keyword anchored |
| US Individual Taxpayer Identification Number (ITIN) | Pattern-matched |
| IBAN, generic international format (any country) | Pattern-matched |
| Cryptocurrency wallet address (Bitcoin, Ethereum) | Pattern-matched |
Most of the identifiers Presidio's catalog names alongside HIPAA or PCI DSS — a Social Security number, an email address, a phone number, a URL, an IP address, a card PAN — already land on the very same rule described in the tables above; this table only lists the identifiers unique to Presidio's broader scope.
Rules you've already customized are never touched by this expansion. New built-in patterns arrive automatically on upgrade, side by side with whatever you've defined yourself, through the same pattern versioning mechanism described above.
Matching Stays Fast as Your Pattern List Grows
Data-loss protection is built so that scanning latency doesn't grow linearly with the size of your pattern list, whether the extra patterns are built-in or your own: a cheap internal pre-filter runs ahead of the more expensive pattern checks to narrow down which patterns are even worth evaluating against a given request or response. This is purely an internal performance mechanism — it never changes which patterns match, what action they take, or the result you see; there is nothing to configure for it today.
File Signature Detection
The patterns above match text. A binary file — a PDF, an Office document, an archive, an executable — that someone pastes into a prompt as base64 or hex is not text: on the wire it is an opaque run of characters, so neither a built-in pattern nor one you wrote yourself can match it. There is no secret shape to find, because the leak is the entire file. "A user pasted a whole contract into the chat" and "the model echoed a document back" both travel this way, invisible to every pattern described above.
File Signature Detection closes that gap by looking at bytes instead of characters. It decodes the base64 and hex candidates it finds in the body and compares the leading bytes of each against a table of known container formats — the magic bytes every binary format begins with. A file pasted raw, without any encoding, is recognized the same way.
The feature is off by default and has its own File Signature Detection section on the Data-Loss Protection policy screen. It is configured independently of the pattern list: a policy with no data-loss pattern at all still does real work while this switch is on, and a policy full of patterns is completely unaffected while it stays off.
Recognized formats
| File type | Covers | fileSignatureTypes value |
|---|---|---|
| PDF document | .pdf | PDF |
| ZIP / Office | .docx, .xlsx, .pptx, .jar and any other ZIP container | ZIP_OOXML |
| PNG image | .png | PNG |
| JPEG image | .jpg, .jpeg | JPEG |
| GIF image | .gif | GIF |
| GZIP archive | .gz | GZIP |
| RAR archive | .rar | RAR |
| 7-Zip archive | .7z | SEVEN_ZIP |
| ELF executable | Linux binaries and shared objects | ELF |
| PE executable | Windows .exe / .dll | PE_EXE |
| OLE2 document | Legacy Office .doc, .xls, .ppt | OLE2 |
| RTF document | .rtf | RTF |
| Java class file | Compiled .class bytecode | JAVA_CLASS |
File Types is an allow-list: leave it empty to detect all thirteen. Narrow it when a format is a legitimate part of your traffic — an assistant that receives pasted screenshots can leave PNG and JPEG out of scope while still catching documents, archives and executables.
Action: Block or Flag — Mask is not offered
The action applies to the detection as a whole rather than per format, and only two choices exist: Block (the default) rejects the request or response, Flag lets it through and records the hit. Mask is deliberately absent. Masking replaces a matched region with [REDACTED], which is well defined for a short secret such as an API key. A magic-byte match has no such region — it says "these bytes are the beginning of a PDF", not "characters 400 through 460 are the secret" — so there is nothing meaningful to redact, and rewriting part of an encoded blob would only corrupt the payload the caller sent. This is the same reasoning that escalates a mask rule to block inside decoded content, described under Decoding Encoded Content above. Setting fileSignatureAction to MASK through APIops is rejected when the policy is saved, and a Mask value that reaches the gateway from an older exported policy is treated as Block: an entire file leaving the organization is assumed to be at least as serious as a short secret, so the safe direction is the default.
Where the signature sits is part of the signal
A signature found at the very start of a candidate means that candidate almost certainly is that file — a whole document was sent. A signature found further inside a larger blob is a weaker signal: the file's bytes merely occur somewhere within other content, for example appended after something else. Both are reported, and the recorded hit says which of the two it was, so someone reviewing the report can tell "a document was uploaded" apart from "a document is buried inside something else".
It works in both directions
Data-loss protection inspects both lanes, and file-signature detection follows it: prompts on the way in, model output on the way back. The response direction is the one worth enabling deliberately — a model that reproduces a base64 document out of its context window or out of a tool result is exactly the embedded-document leak this detection exists for, and it is the direction a request-only filter never sees.
A magic prefix is never trusted on its own
Short magic bytes collide with random data by chance: the Windows PE header is two bytes (MZ), which turns up in roughly one of every 65,536 random byte pairs. Every format in the table is therefore paired with a format-specific check that runs on the bytes after the magic prefix — the PE check follows the DOS header's pointer and requires the bytes it points at to read PE\0\0, the PNG check requires the first chunk to declare exactly the length a real PNG declares, the ZIP check requires plausible "version needed" and "compression method" fields. A candidate too short to run its format's check never matches at all. This is what keeps a hash, a token or a random identifier inside a prompt from being reported as a smuggled executable.
Limits, and never a silent cut
Scanning is bounded — on input length, on how many base64/hex candidates are examined, on total decoded bytes, and on nesting depth (base64 wrapped in base64 is decoded, but the chain does not continue indefinitely). When a ceiling is reached the scan returns what it found so far and records that it was truncated, following the same "never truncated silently" contract as the encoded-content decoding limits. Only base64 and hex are treated as byte carriers: percent-encoding and \uXXXX escapes inflate binary data three- to sixfold and are not a realistic way to smuggle a file through a prompt. A base64 blob wrapped in those encodings is still reached, because the text produced by encoded-content decoding is scanned as well.
Detection does not need the Decode Encoded Content switch to be on — decoding base64 and hex is part of the detection itself. When that switch is on as well, the two share the same decoding work on a given body instead of decoding the same text twice.
While File Signature Detection is on, this policy's streamed responses are accumulated and scanned as a whole instead of chunk by chunk — whichever action you chose, Flag included. An encoded file signature can straddle any chunk boundary: the magic bytes would land in one chunk and the bytes that confirm them in the next, so a per-chunk scan would reliably find nothing at all. This is the same buffer-and-scan approach described under Streaming Responses: Buffer-and-Scan, and the practical cost is that time-to-first-token is no longer preserved for this policy's streamed responses. Leave the switch off on latency-critical proxies that carry no document-leak exposure.
How a match shows up
A file-signature hit is recorded distinctly from a secret-pattern hit, so the two never blur together in the Guardrail Hits report: the hit is tagged file-signature rather than secret, and its label carries the format and the position — for example File signature (PDF, at_start). When more than one signature matched, the label counts the rest. In tracing the recorded entry holds the format, the position, whether the bytes arrived raw or base64/hex-encoded, and how many matches there were — never the scanned content itself.
The three fields are also settable through APIops on the data-loss protection policy JSON: fileSignatureDetectionEnabled, fileSignatureAction (BLOCK or FLAG) and fileSignatureTypes (an array of the values in the table above, empty or omitted for all thirteen). See API Reference: AI Gateway.
Writing and Testing Patterns
Patterns use Java regular-expression syntax (java.util.regex.Pattern) — that is the engine the gateway matches with. It is not identical to JavaScript or PCRE: lookbehind must be fixed-width, named groups are written (?<name>...), and Unicode classes such as \p{L} are supported.
The Test the pattern panel on the PII Patterns screen runs a pattern against sample text before you save it. The test executes server-side, on the same engine as the gateway, so the matches and masked output you see there are exactly what will happen at runtime.
Patterns that can backtrack catastrophically are skipped at runtime. The gateway screens every pattern with a ReDoS check, and one that fails is never applied while a request is processed — the rule looks enabled but masks nothing. The test panel reports this as a "ReDoS risk" before you save.
Actions
Every guardrail applies one of three actions when it matches:
- Block — the request or response is rejected entirely
- Flag — processing continues, but the match is recorded for review
- Mask — the matched value is replaced before continuing
Mask Shapes
For personal-data rules the mask shape is chosen per rule:
| Shape | Input | Output |
|---|---|---|
| Replace entirely (default) | 05321234567 | *** |
| Keep first N characters (N=4) | 05321234567 | 0532******* |
| Keep last N characters (N=4) | 05321234567 | *******4567 |
| Mask first N characters (N=4) | 05321234567 | ****1234567 |
The default shape replaces the whole value with *** and leaks nothing. Partial shapes preserve the output length and deliberately leave part of the value visible; since the remaining fragment can identify a person on its own (the last four digits of a phone number, say), choose one on purpose. The replacement text is configurable too — in partial shapes its first character is used as the mask character.
If the number of characters to keep is greater than or equal to the value length nothing would be masked at all; in that case it falls back to the safe side and replaces the whole value.
Anonymization: Sequential Placeholders and Synthetic Values
Beyond the fixed and partial mask shapes above, two further shapes replace a matched value with something that reads naturally in the rest of the conversation, instead of a block of asterisks:
| Shape | Input (a TCKN, for example) | Output |
|---|---|---|
| Sequential placeholder | 12345678950 | <TCKN_1> |
| Synthetic value | 12345678950 | a different, checksum-valid TCKN, e.g. 98765432106 |
Sequential placeholder replaces every match with a type-tagged token — <TCKN_1>, <EMAIL_2>, and so on — numbered in the order each distinct value was first seen. The token format is configurable ({TYPE} and {N} placeholders).
Synthetic value replaces the match with a fake value of the same kind, generated so it still satisfies that type's own validation — a synthetic IBAN passes the mod-97 checksum, a synthetic credit card passes the Luhn check, and so on — using a reserved test range (card numbers use the 4111 test BIN, IP addresses use the documentation-reserved 203.0.113.0/24 block) so it's distinguishable from a real one on inspection.
Each new synthetic value is generated from independent random data — never computed from the original value it replaces. This is deliberate: HIPAA Safe Harbor requires that a de-identification code not be derived from the data it stands in for, and GDPR Article 4(5) requires the information that could re-identify someone to be kept genuinely separate from the pseudonym. A synthetic value computed from the original (a hash of it, say) would still look one-way, but it would let anyone holding a candidate value confirm whether that person's data was present, simply by reproducing the same fake value and checking for a match. Apinizer's synthetic values carry no such relationship — nothing about the fake value can be recomputed from, or tested against, the original.
Both shapes are consistent within one request/response cycle: the same original value always maps to the same placeholder or synthetic value everywhere it appears in that exchange, including inside a tool call and its result. They are not consistent, and deliberately so, across two different requests — the same real value maps to a different placeholder or synthetic value each time, so a value observed in one conversation can't be correlated back to the same value in another.
Both shapes are one-directional by default. Apinizer does not store a mapping from the placeholder or synthetic value back to the original unless you deliberately turn on reversible pseudonymization for that rule — see Reversible Pseudonymization (Unmask) below. With the default setting, choose Mask or Anonymize only for data you don't need to reconstruct later.
Each request tracks a bounded number of distinct values for this purpose; if that ceiling is reached, any further new value falls back to the default Replace entirely shape rather than failing the request — a value already assigned earlier in the same exchange keeps working normally.
Reversible Pseudonymization (Unmask)
Sequential Placeholder and Synthetic Value can each optionally be made reversible, so an authorized operator — or, in one mode, an authorized caller of the AI proxy itself — can recover the original value later without permanently exposing it in every response. This is off by default, and it applies to the whole policy, not to one masking rule at a time — set it on the policy screen's Re-identification (Unmask) section, or through the policy JSON's unmaskMode field or the APIops REST API. Three modes are available:
| Mode | Behavior |
|---|---|
| None (default) | Today's behavior, unchanged — nothing is stored, there is no way back. |
| Automatic (Response Inline) | The reversible storage described below is armed, and every pseudonym that reappears in the response is swapped back to its original value automatically, before the response reaches the caller — but only for a caller whose role is on the policy's Authorized Caller Roles list below, only on a non-streamed response, and outside a zero-retention profile (see the note further down). |
| Unmask API | Reversible storage is armed; an authorized Manager operator resolves a placeholder or synthetic value back to the original afterwards, through the Management API. |
Authorized Caller Roles. Automatic mode needs at least one entry here — the gateway-caller roles (the same scope values a credential is granted) allowed to receive the original value back automatically. This list is deny-by-default: leaving it empty means nobody is restored even with Automatic mode selected — the response still ships with the placeholder or synthetic value in it, and the policy screen visibly warns about the empty list.
Where the original value is kept. The original value is never written to a log, a trace, or any durable database. It exists only as an entry in the distributed cache, encrypted with AES-GCM, keyed to the request's trace ID, the rule, and the exact placeholder or synthetic text that was produced. That entry expires after one hour — once it does, the original value cannot be recovered by anyone, including Apinizer support.
Automatic restoration is audited before it happens, not after. Every automatic restoration writes an audit record first; only once that write succeeds is the original value substituted into the response. If the audit write itself fails, restoration is skipped and the response ships with the placeholder or synthetic value still in it — an automatic restoration never happens without a matching audit record.
When automatic restoration doesn't run. Besides an unauthorized caller role, automatic restoration is skipped — and the response ships exactly as if Automatic mode were off — on a streamed response (not supported in this version) and on an API proxy running under any non-Standard data-retention profile — No Payload or No Persist, either one. Each skip is recorded in the application log with its reason; there is no separate on-screen indicator for it today.
Resolving a value manually. An authorized operator calls POST /api/ai-pii-unmask with the project id, the request's trace ID (see Tracing and Replay), the rule's identifier, and the exact placeholder or synthetic text as recorded in the traffic. The same resolution is also available from the Manager UI's PII Unmask screen (AI Governance menu), which calls this endpoint on your behalf. Access requires the AI Development Manage permission on that project — denied by default — and every attempt is written to the audit trail, whether it succeeds, is denied, or the entry has already expired. The recovered value itself is never written to that audit trail, only who asked, when, and the outcome. Both the automatic path above and this manual path write to the same audit trail.
Turning on Automatic or Unmask API mode arms the reversible cache entry regardless of the proxy's data-retention profile — a No Payload or No Persist proxy does not by itself stop the reverse mapping from being written. What a non-Standard profile changes is Automatic mode's own automatic-substitution step: as noted above, that step is skipped outright there, so the response still ships masked even for an otherwise-authorized caller — the mapping remains recoverable afterwards through the Unmask API, within its TTL. If your installation runs under No Payload or No Persist deliberately and you don't want reversible storage of any kind, leave unmaskMode at None.
Reading a Value Back
Besides masking, a rule can delete, hash, encrypt or merely detect a match:
- Encrypt — the value is encrypted with AES/GCM/NoPadding and written as Base64. The key is embedded in the Apinizer installation and is never supplied by the user, so the value can only be decrypted inside that same installation. The practical route is to call
UtilCommon.decryptWithDefaultAlgorithm(<base64value>)from a Groovy script policy and print the output withrequest_log()/response_log(). Decrypting with an external tool (openssl and friends) is not possible — the key is not exported. - Hash — SHA-256 plus an installation-specific salt is applied, and it is not reversible. The same input always yields the same output, so it can be used for correlation, but the original value cannot be recovered.
Choose encryption for data that may need to be read later, hashing or masking for data that will not.
Anonymous Requests
The retry-storm (loop) guardrail keys its counter on the model, the API proxy and the caller's identity, so one caller's repeats never consume another caller's budget. On an API proxy without authentication a request carries no such identity, and the policy decides what to do:
| Option | Behaviour |
|---|---|
| Do not apply the policy (default) | The request passes and no repeat counter is kept |
| Consume from a shared pool | All anonymous traffic is counted in a single bucket |
| Reject the request | Anonymous requests are turned away with HTTP 401 |
The default matches the behaviour before this setting existed: on an anonymous request the guardrail is silently inactive. With the shared pool, tenant isolation is not expected — one caller's consumption spends everyone else's budget; it is meant for installations that deliberately allow anonymous access but still want a ceiling.
This is distinct from the proxy-level block anonymous requests setting, which rejects such requests before they ever reach the policy. The choice here applies on a proxy where anonymous access is allowed.
The semantic cache offers the same three options, for the same reason — see Semantic Cache.
Oversized Protection
Limits request size along two dimensions:
- Max tokens per request — the request is rejected if the prompt's estimated token count exceeds this value
- Max characters per prompt — the request is rejected if the raw prompt text exceeds the character limit
The default behavior when a limit is exceeded is to block the request; instead of blocking, you can opt to automatically truncate the oldest conversation turns — see Per-Request Size Limit.
This guardrail keeps oversized requests from growing unchecked in both cost and resource consumption.
Context-Integrity Protection
Detects attempts to inject a fabricated system/assistant role into the chat history (message list). While Prompt Protection scans content for meaning, this guardrail protects the structure of the conversation:
- Fake role-marker detection — patterns embedded in a user message's content that try to trick the model into treating it as a prior system/assistant turn (for example, chat-template control markers or fake instruction markers resembling
###Instruction) - Role-order validation (optional) — blocks a client from adding more system messages than allowed, or from using a role that isn't permitted
A match applies one of three actions: Block, Flag, or Mask (only fake role-marker matches can be masked; a role-order violation falls back to Flag instead of Mask — a structural mismatch can't be masked).
Execution Modes
The guardrail is evaluated before the request continues. This is required for a Block action to reliably prevent a match from going through.
The guardrail is evaluated in the background so it doesn't add latency to the request path; matches are still recorded and reported. This is the default mode for off-topic protection.
The guardrail runs and reports matches without ever blocking or masking — useful for tuning thresholds before turning on enforcement.
When an external provider is attached to Prompt Protection or Off-Topic Protection, Asynchronous mode's wait budget and Observe-Only mode's comparison behavior both extend to the external call as well — see External Provider Guardrails.
Off-Topic Protection: Allowed and Denied Topics
Off-topic protection compares an incoming prompt's embedding against the topics you define, using cosine similarity — if the prompt's closest match falls below the similarity threshold, it's treated as off-topic and flagged or blocked before the request is forwarded (to the model on an AI Gateway, or to the tool/agent on an MCP or A2A Gateway).
Two independent topic lists exist, and you can configure either one or both. With the built-in (embedding) engine enabled the screen asks for at least one topic definition — a guardrail with nothing to compare against silently does nothing at runtime, so Save stays disabled while both lists are empty. A denied-only setup is a legitimate mode and is not blocked. When the engine is set to external-only, the built-in comparison never runs and no topic list is required.
- Allowed Topics — what this proxy is supposed to answer about. A prompt that falls below the threshold on every allowed topic is treated as off-topic.
- Denied Topics — topics this proxy must never answer about, no matter how well the same prompt also matches an allowed topic. Useful for carving an exception out of an otherwise broad allowed scope — for example, denying "legal advice" on a banking assistant that allows broad financial questions.
When a prompt matches both an allowed topic and a denied topic, the denied match takes precedence — the request is flagged or blocked as denied even though it would otherwise have passed the allowed-topics check. If a request is being blocked unexpectedly, check the denied list first.
Denied topics have their own similarity threshold and action, each optional and falling back to the corresponding allowed-topics setting when left blank:
| Denied-topic setting | Falls back to |
|---|---|
| Denied Similarity Threshold | Similarity Threshold |
| Denied Topic Action | Action |
Topic embeddings — for both lists — are computed once and cached, not re-embedded on every request; add your topics before going live so the first request isn't slowed down by the embedding call.
Streaming Responses: Buffer-and-Scan
Off-Topic Protection used to scan only the incoming request (the user's own message). It can now also inspect streamed responses — the text the model itself generates is compared against the same allowed/denied topic definitions, so a response that drifts off-topic mid-stream (for example, a jailbroken model that starts answering something unrelated in detail) is detected too.
The mechanism is buffer-and-scan: every chunk of the stream is withheld from the client and silently appended to a buffer instead of being forwarded, and the embedding-similarity comparison runs exactly once, at the end of the stream — there is no per-chunk embedding call, since adding a network round trip to every one of the (often hundreds of) chunks in a streamed response would seriously hurt latency. Because the response is held back until that single comparison runs, this is an actual preventive control: a Block verdict withholds the buffered text entirely and the client receives nothing at all for that stream, while a Pass or Flag verdict releases the complete text to the client as one final delta once the stream ends.
The buffer only opens when at least one direction (allowed or denied) has its action set to Block. If both directions are Flag-only (or no topics are configured at all), streamed responses incur zero additional cost — nothing is buffered, nothing is scanned, and today's chunk-by-chunk streaming behavior is preserved exactly.
The llm-judge external judge described in External Provider Guardrails does not participate in this streaming path — only the built-in (embedding) comparison runs. A policy configured with External provider only therefore scans nothing at all on a streamed response (neither built-in nor external) — the same constraint described in External DLP Integration for that adapter's own streaming behavior.
If the accumulated buffer exceeds 65,536 characters before the stream ends (an unusually long streamed response), Off-Topic Protection gives up on scanning it as a whole: everything withheld so far is released to the client as a single unscanned burst, and every chunk after that point streams through live and unscanned for the rest of that response — the embedding comparison never runs for that stream, and this is recorded visibly in the trace. A response this long already exceeds what one whole-response embedding call can usefully classify; releasing the withheld content unscanned was judged better than silently dropping it. A typical response completes well within this limit and is fully buffered and scanned.
While a Block-configured Off-Topic Protection is buffering a response, the caller sees nothing until the stream ends: there is no time-to-first-token, only a single delta at the end carrying either the whole response (Pass/Flag) or nothing at all (Block). That verdict is recorded in tracing and the Security and Guardrails report either way. Non-streamed (unary) responses are unaffected by this trade-off — evaluation there already happens before the response reaches the client.
This buffer also never activates when Data-Loss Protection's own buffer-and-scan is active on the same response (a Block or Mask rule, or File Signature Detection, configured under Data-Loss Protection Patterns above): DLP already owns per-chunk withholding there and can release a scanned partial prefix mid-stream, and Off-Topic Protection's single whole-response classification has no way to retract content DLP already released. When both are configured on the same proxy, DLP's buffering takes over and Off-Topic Protection's streaming check does not run for that response at all — its request-side check, and its check on non-streamed responses, are unaffected.
Content Safety Categories (AILuminate)
Off-Topic Protection's Allowed and Denied Topics are free text — you write your own topic descriptions. As a starting point for content-safety risks specifically, Apinizer ships a catalog of 14 hazard categories taken from the MLCommons AILuminate v1.1 taxonomy, grouped the way the standard itself groups them:
| Group | Categories |
|---|---|
| Physical (5) | Violent Crimes, Sex-Related Crimes, Child Sexual Exploitation, Suicide & Self-Harm, Indiscriminate Weapons (CBRNE) |
| Non-Physical (5) | Intellectual Property Violations, Defamation, Non-Violent Crimes, Hate, Privacy |
| Contextual (4) | Specialized Advice — Elections, Specialized Advice — Financial, Specialized Advice — Health, Sexual Content — Pornographic |
Find the catalog on the Guardrails hub's Content Safety (AILuminate) tab. Each entry shows its category name, group, and a short definition, plus — where one exists — the Llama Guard 3 safety code it corresponds to. This is useful when your external judge is Llama Guard or a model trained on the same taxonomy: its verdict comes back as an S-code, and this catalog is where you trace that code back to the matching AILuminate category. S6 ("Specialized Advice") covers both the Financial and Health categories here; every other code maps one-to-one.
A hit against a Denied Topic drawn from this catalog is broken down by AILuminate category in the Guardrail Hits report — hit counts per category, a time series, and a project/proxy breakdown — and each recorded trace entry carries the matched category, its similarity score, and which engine reached the verdict (the built-in embedding comparison or the llm-judge).
Not a Bound Preset
Unlike the personal-data, prompt-guard and data-loss protection presets described earlier on this page, this catalog has no "Add from Preset" button and no catalog-bound rule state — there is nothing here for a policy to bind to or fall out of sync with. Each entry carries a Copy (EN) action and — when a Turkish text is defined — a Copy (TR) one: click it to copy that category's definition text to your clipboard, then paste it into Off-Topic Protection's Denied Topics list yourself. Because a policy only ever holds a copy, editing a category here never changes a policy that already uses it; to pick up a changed definition, copy it again and replace the old text in Denied Topics.
Adding Your Own Category
The catalog is editable. Add Category creates your own entry, and the fourteen that ship with Apinizer can be edited or deleted as well — a hazard specific to your institution does not need to wait for an Apinizer release. Editing needs the AI Development Manage permission, and an entry defined at the admin level can only be changed from there, not from inside a project.
Three things are worth knowing before you edit a shipped entry:
- Deleting a shipped category is permanent. The fourteen AILuminate entries are seeded once; Mongock deduplicates changesets by id and no seed changeset is
runAlways=true, so a seed that already ran does not run again. A deleted category stays deleted across version upgrades — it comes back only if a future release ships an explicit re-seed changeset. If you only want to stop using a category without losing it, disable it instead of deleting it. - Rewording a shipped category breaks its link to the standard. The definitions are quoted from AILuminate v1.1; once you change the wording, the entry no longer says what the cited standard says. That is a legitimate thing to do for your own traffic — just don't treat the reworded entry as the standard's category any more.
- A category you add has no Llama Guard code. The AILuminate-to-Llama-Guard mapping is part of the product, so the field stays empty for your own categories. Leave it that way: inventing an
S-code would make the external judge trace above point at the wrong category. The field only exists so you can record a real mapping when one genuinely applies.
Each entry also has optional Turkish twins for its display name, definition, and description. Fill them in when you serve Turkish traffic — but use the Turkish definition instead of the English one in Denied Topics, not alongside it. The policy scores a request against the closest entry in the list, so carrying both wordings of the same category raises the rate at which legitimate requests get blocked.
GDPR Article 9 Marker
Four of the fourteen categories — Hate, the Elections- and Health-related Specialized Advice categories, and Sexual Content — are flagged as touching a GDPR Article 9 special category of personal data (health data, political opinions, and so on). This flag exists because those categories are open-ended, contextual concepts: "does this text reveal someone's health condition" isn't something a fixed regular expression can reliably decide, so coverage for it belongs to this semantic (embedding-similarity) layer rather than the pattern-based Personal Data Masking or Data-Loss Protection guardrails above, which only ever match a fixed structural or literal shape.
Turkish category texts and language choice
Alongside its English definition, each category carries an optional Turkish counterpart. When the interface is in Turkish, the category name and description are shown in Turkish; if no Turkish text is defined, the English one remains visible. The English fields stay canonical — they are the AILuminate taxonomy's own wording; the Turkish texts are Apinizer translations and are not part of the standard.
The value of the Turkish text is not only readability. Because topic definitions are compared by semantic similarity, the language of the definition affects the outcome. On a deployment serving Turkish traffic, use the Turkish definition instead of the English one rather than adding both. Off-Topic Protection scores every definition in the list and takes the highest similarity, so adding the same category in two languages measurably increases how often legitimate requests are blocked by mistake.
Off-Topic Protection evaluates Denied Topics before Allowed Topics. If you add Turkish definitions to Denied Topics but leave Allowed Topics English-only, a configuration running in allow-list mode will block legitimate Turkish requests. Whichever language coverage you choose, apply it to both lists.
The similarity threshold for a topic match is only meaningful relative to the embedding model you selected: different models produce different similarity scales for the same semantic closeness. Re-evaluate the threshold whenever you change the embedding model — otherwise the guardrail may never trigger at all, or may catch far more requests than you expect.
If the threshold turns out to be unreachable for your model, Apinizer does not leave that silent: the first time topic definitions are embedded, if their highest similarity to each other falls below the threshold, a warning is written to the gateway log. Topic definitions are the most similar texts this policy will ever compare; if even they cannot clear the threshold, a real prompt will not either. The warning is advisory only — it does not lower the threshold by itself, because silently changing the effective cut-off would alter blocking behaviour for every existing installation.
This catalog reproduces category names and definitions from the MLCommons AILuminate v1.1 hazard taxonomy, licensed under CC BY 4.0. See mlcommons.org/ailuminate for the source standard. :::
Apinizer does not publish false-positive/false-negative rates for this catalog. Measuring that reliably needs a labeled, multi-lingual corpus specific to your own traffic — a number calibrated on someone else's data would not describe your false-positive rate.
External Provider Guardrails
Prompt Protection and Off-Topic Protection can call out to an external, LLM-based judge alongside (or instead of) their own built-in check: the regex rules for Prompt Protection, the embedding-similarity comparison for Off-Topic Protection. Personal Data Masking and Data-Loss Protection can use the same llm-judge adapter too, and additionally offer an adapter type of their own (http-dlp) that connects to your organization's own vendor-neutral HTTP service; these two carry some different rules for their external-provider behavior, covered separately in External DLP Integration below. Retry-Storm Protection, Oversized Protection, and Context-Integrity Protection have no external-provider option. Groundedness Protection sits at the opposite extreme: it has no built-in check at all — an llm-judge is as mandatory as the policy itself — see Groundedness Protection.
Today, the only supported external judge is an LLM judge — an existing LLM Provider connection you already use elsewhere in the AI Gateway, called with a chat-completion request and asked for a safety (or topic-relevance) verdict. This uses your existing connection; there's no separate credential or connection type to set up.
Finding the Setting on Screen
The setting lives on the guardrail policy's own edit screen — there is no separate catalog page.
Go to API Proxy → Policies → <policy> → Definition and scroll below the rule list to the
shield-marked External Provider section.
Two things make it easier to find:
- If the policy already has an external provider configured, a notice appears at the top of the Definition tab saying so; clicking it scrolls straight to the section.
- While the engine is Built-in (the default), everything below the engine selector is hidden, so you only see the selector and a short pointer. The provider selection and its fields appear as soon as you change the engine.
Prerequisite. The LLM Provider list shows the LLM connections defined in that project, not provider definitions. If the project has no connections the list comes up empty — create one from LLM Providers first.
Evaluation Engine
| Evaluation Engine | Behavior |
|---|---|
| Built-in only (default) | Only the built-in check runs — today's behavior, unchanged. |
| Built-in + External provider | Both run. Either one judging the content unsafe (or, for Off-Topic Protection, denied) is enough for the guardrail to act. |
| External provider only | The built-in check is skipped entirely; the external judge's verdict is the only one that counts. |
In Inline and Asynchronous mode, when both checks run, a Block verdict from the built-in check short-circuits — the external provider is never called, saving the round-trip cost and latency. In Observe-Only mode both always run, since the comparison described below needs both results.
Adding an External Judge
Choose an existing LLM Provider connection. Only OpenAI-compatible chat providers are accepted for this purpose: OpenAI, Azure OpenAI, vLLM, Ollama, Custom OpenAI-Compatible, DeepSeek, Groq, Moonshot (Kimi), Mistral, Zhipu (GLM), or Qwen (DashScope). A connection of any other provider type is rejected before any call is attempted.
Enter the judge model's name (for example llama-guard3 or gpt-4o-mini). Unlike some other AI Gateway fields, there is no fallback to a provider default here — leaving it empty makes every judge call fail, which (under the default failure setting below) blocks every request until you fill it in.
Timeout bounds the judge's own HTTP call (default 2000 ms). Max Input Characters truncates an overly long prompt before it reaches the judge (default 8000). Max Output Tokens caps the judge's reply (default 64 — see the reasoning-model warning below). JSON Mode asks the judge to reply with a structured verdict instead of a line of text, for providers that support it.
Judge Prompt: Native vs. Template Mode
The Judge Prompt Template field controls how the reviewed text is presented to the judge, and it behaves differently depending on which of the two guardrails you're configuring.
Leaving it blank is a deliberate, supported configuration for Prompt Protection: the (truncated) text is sent to the judge exactly as-is, with no wrapping instructions. This is the right choice for a model trained specifically for safety judging — Llama Guard 3 or ShieldGemma, for example — which already carries its own safety taxonomy inside its own chat template. Wrapping the text in another set of instructions on top of that confuses the model into treating your instructions as content to evaluate rather than as its task, and can silently produce the wrong verdict.
For a general-purpose instruct model with no safety-judging behavior of its own — gpt-4o-mini, for instance — fill in a template that tells the model what to look for and how to answer. Apinizer ships a ready-made safety template (13 numbered categories, from violent crime to election-related content) that the Fill default taxonomy button drops in with a single click; edit it or write your own from scratch. Every template must contain the <<<PROMPT>>> placeholder — this is where the (neutralized) reviewed text is substituted in at evaluation time.
For Off-Topic Protection, an empty Judge Prompt Template is not accepted — saving is rejected. A generic safety template answers "is this content safe," not "does this content belong to one of my allowed topics," so a blank template would silently produce meaningless off-topic verdicts. Describe your allowed and denied topics directly in the template instead.
Reasoning Models as the Judge
If the judge is a reasoning model — one that "thinks" before answering — its reasoning tokens are deducted from the same Max Output Tokens budget as the final verdict. With the default of 64, the model can spend its entire budget thinking and never emit the safe/unsafe line, which the guardrail then treats as a parse failure (and, under the default failure setting, a blocked request). Raise Max Output Tokens to 1024 or higher, or turn off the model's reasoning mode if it offers that option. The more reliable fix is to use a guard-tuned model in Native mode instead of a reasoning model in Template mode.
Self-Hosted Example
A self-hosted, guard-tuned model is a good starting point: no data leaves your infrastructure, and Native mode needs no prompt engineering.
With Ollama: ollama pull llama-guard3, then start Ollama normally. With vLLM: serve llama-guard3 (or another guard-tuned model) behind its OpenAI-compatible endpoint. Both platforms apply the model's own chat template automatically when called through /v1/chat/completions — that automatic template handling is exactly what Native mode relies on.
Provider type Ollama or vLLM, endpoint pointing at your running instance (for example http://<host>:11434/v1 for Ollama), authentication set to none for a local or otherwise trusted network.
On Prompt Protection or Off-Topic Protection, set Evaluation Engine to Built-in + External provider (or External provider only), pick the connection you just created, set Model to the model name you pulled, and leave Judge Prompt Template blank for Native mode (Prompt Protection) — or filled in with your topic description for Off-Topic Protection, where a template is required.
Comparing Verdicts in Observe-Only Mode
When an external provider is configured, Observe-Only mode does more than record its own matches: it also computes the built-in decision and compares it against the external judge's verdict, without ever applying either one. This comparison is recorded only as a metric, for calibrating a new judge or a gradual rollout before switching to a mode that actually enforces it — it is not part of the Guardrail Hits report described below in Observability.
Call Timing and Failure Handling
The judge's own HTTP call timeout (the Timeout setting above, default 2000 ms) is enforced by the judge connection itself no matter which execution mode you pick. On top of that, Asynchronous mode bounds how long the guardrail waits for a result using the guardrail's Max Wait setting — for the built-in check this defaults to 500 ms; for an external provider call it defaults independently to 2000 ms when left blank. In Inline mode without Max Wait set, the request thread waits directly on the judge's own Timeout value — a slow provider adds directly to request latency, so consider Asynchronous or Observe-Only when attaching an external provider (Apinizer surfaces this same warning next to the setting).
| External Failure Mode | Behavior |
|---|---|
| Block on failure (default) | If the external provider call itself errors out or times out, the request is blocked. |
| Allow on failure | The request continues, unguarded by the external judge for that call, and the failure is still recorded visibly rather than passing silently. |
This setting only governs an actual call failure or timeout — not what the judge decides. If the evaluation engine is set to use an external provider but no usable connection is actually configured (for example, a gap reached through APIops or export/import that bypasses the UI's own checks), Apinizer degrades to the built-in check instead of failing the request outright, and records the degradation visibly for troubleshooting. The external failure mode above never applies to this case.
Configuring via APIops
The external-provider settings on Prompt Protection and Off-Topic Protection travel through the APIops surface the same way the rest of each policy does: the LLM provider is referenced by name, never by its internal ID, resolved against your project's connections on import. When Evaluation Engine is left at its default (built-in only), omit the external-provider block entirely — supplying one, even an empty one, still requires a valid, supported adapter type and is rejected on import.
Security Considerations
The judge receives the (truncated) reviewed text as-is — the same text your model would otherwise see. If a personal-data masking guardrail runs earlier in the same policy chain, the judge sees the masked version; if it runs later, or isn't configured at all, the judge sees the raw value. Check the ordering of your guardrails if that matters for your data-handling requirements. For sensitive traffic, prefer a judge you host yourself — on-premises or inside your own VPC — over a third-party cloud judge.
Apinizer never writes the judge's raw response, the rendered prompt, or the underlying failure detail to the trace or to logs — only sanitized safety-category labels and, when the judge provides one, a numeric score are recorded.
Observability
The Guardrail Hits report breaks results down by provider (the built-in check or the external judge) and by verdict (blocked, timed out, errored, or skipped due to misconfiguration) — safe verdicts aren't included, since the report exists to surface what triggered, not every evaluation that passed. The Observe-Only comparison described above is metric-only and is not part of this report.
External DLP Integration
Personal Data Masking and Data-Loss Protection can, alongside their built-in (regex-based) rule list, ask an external provider for a verdict — either the same llm-judge judge described in External Provider Guardrails above, or an adapter type of their own (http-dlp) that connects to your organization's own external DLP / content-protection service: not a general-purpose LLM, but a vendor-neutral HTTP service you write yourself or already operate.
Evaluation Engine
The same three engine options apply here:
| Evaluation Engine | Behavior |
|---|---|
| Built-in only (default) | Only the rule list runs; no external call is made |
| Built-in + External provider | Both run; a built-in Block match short-circuits the external call (it is never made); the external service is called only when there is no built-in match |
| External provider only | Only the external service is evaluated; the rule list is ignored even if it has entries |
Personal Data Masking's normal job is masking. With External provider only selected, the rule list is disabled entirely — the policy no longer masks anything; it only blocks or passes the request based on the external service's verdict. Use Built-in + External provider if you want masking behavior preserved.
Whenever the external service returns an unsafe verdict, the request is blocked — even on Data-Loss Protection, this is a block, not a mask: the external judge can only say safe or unsafe, never "mask this".
Adapter Type: LLM Judge or HTTP DLP Service
Once the engine is set to anything other than Built-in, you separately choose which adapter gets called:
| Adapter | What it does |
|---|---|
LLM Judge (llm-judge) | The same adapter described for Prompt Protection and Off-Topic Protection in External Provider Guardrails above — the LLM provider reference, judge model name, prompt template (Native passthrough / Template mode), JSON mode, and max output tokens are all configured the same way. Everything in that section (including the reasoning-model warning) applies here too. |
HTTP DLP Service (http-dlp) | Connects to your organization's own vendor-neutral HTTP service, following the fixed contract in Connection and Request/Response Contract below. |
The adapter choice only changes who produces the verdict. The engine options (Built-in only / Built-in + External provider / External provider only), the External Failure Mode (Block/Allow on failure), and the three behavioral differences described just below are identical for both adapters.
Three Differences from Prompt/Off-Topic Protection
This integration shares the same underlying plumbing as External Provider Guardrails, but behaves differently in three ways:
- Execution mode is always Inline. The Inline / Asynchronous / Observe-Only choice available on Prompt Protection and Off-Topic Protection does not exist here — the external call always runs synchronously, on the thread handling the request; the adapter's own connection timeout is the only bound.
- The external service is never called for streamed responses. Chunk-by-chunk scanning always uses the built-in rule list only, regardless of the engine setting. The practical consequence: a policy configured with External provider only scans nothing at all (neither built-in nor external) on a streamed response — on non-streamed (unary) requests and responses the external service still runs normally.
- A configuration inconsistency is handled separately from a runtime failure. If the engine wants an external provider but the adapter is undefined or the connection reference can't be resolved, that's a save/deploy-time error — the engine automatically degrades to Built-in, and the failure behavior below does not apply. That failure behavior only kicks in once the external call is actually attempted and errors out or times out.
Failure Behavior
If the external call errors out or times out:
| Setting | Behavior |
|---|---|
| Block on failure (default) | The request is blocked |
| Allow on failure | The request continues; this is always recorded visibly — there is no silent fail-open |
Connection and Request/Response Contract
This contract is for the HTTP DLP Service adapter only — when LLM Judge is selected, the fields in External Provider Guardrails above apply instead.
The external service connection is set up through an existing Webhook connector — no separate connection type is introduced for this; the address, HTTP method, headers (including authentication), timeout, and TLS settings all come from that connection.
The fixed contract your own service must follow is:
Request (sent by Apinizer, application/json):
{
"text": "<text to scan, truncated to the configured max input characters>",
"direction": "REQUEST" | "RESPONSE",
"apiType": "AI" | "MCP" | "A2A" | null,
"apiProxyName": "<proxy name>" | null,
"apiProxyId": "<proxy id>" | null,
"correlationId": "<request correlation id>" | null
}
Response (your service must return HTTP 2xx with this body):
{
"safe": true,
"categories": ["secret-key"],
"score": 0.87
}
- Exactly one of
safe(boolean) or its inverseflagged(boolean) must be present — sending both, or neither, makes the call count as failed. categoriesis optional, taken as an array of strings.scoreis optional and taken only when it is a number between 0 and 1; otherwise it is ignored.- Any non-2xx status code, an invalid or missing JSON body, or a missing/non-boolean
safe/flaggedfield counts the call as failed and triggers the Failure Behavior setting above. There is no silent safe/unsafe assumption.
The default timeout is 2000 ms and the default max input characters is 8000; both are configurable on the policy screen.
Turning It On From the Screen
Personal Data Masking and Data-Loss Protection policy screens carry an External Provider section in the same place as the External Provider Guardrails section described above:
| Field | Corresponds to |
|---|---|
| Evaluation Engine | The three options in the Evaluation Engine table above |
| External Failure Mode | Shown once the engine is set to anything other than Built-in; the Block/Allow on failure choice from Failure Behavior |
| Adapter Type | The LLM Judge/HTTP DLP Service choice from the Adapter Type table above; depending on the choice, an LLM provider reference or a Webhook connection, timeout, and max input characters fields appear |
Fill in the section, save the policy, and click Deploy. The same settings are also reachable through the APIops REST API (API Reference: AI Gateway) or by setting engine, failMode, and externalGuardrail in the policy JSON.
Groundedness Protection
Checks whether an LLM's response is actually grounded in the context that a RAG policy injected into the same request — an external judge compares the generated response against the retrieved context and decides grounded or ungrounded/hallucinated. It runs on the response lane and can only be added to AI Gateways — it is not available on MCP or A2A Gateways.
This guardrail only checks groundedness against the RAG context this proxy injected into the same request. It is not a general factuality check against world knowledge — it never verifies anything against an external source. As a result:
- A claim that is factually true but absent from the retrieved context is still treated as ungrounded — the guardrail only judges support from the context, not truth.
- If the retrieved context itself is wrong (your source data is inaccurate), a response that faithfully repeats that wrong information is still judged grounded — the guardrail never questions the source's accuracy.
How RAG Context Is Detected — the Single Biggest Caveat
This guardrail does not read a dedicated signal published by the RAG policy — it looks for a literal <CONTEXT>...</CONTEXT> marker pair inside the request body actually sent to the backend LLM. RAG Injection's own default Injection Template already contains exactly this marker pair (see RAG Injection configuration).
RAG Injection's Injection Mode field defaults to Prepend to User Message — in that mode the context is inserted as plain text ahead of the user's request with no marker at all. In that default mode (or in Prepend to System Message / Append After System Message), Groundedness Protection finds no detectable context in the request and silently does nothing — it neither blocks nor flags, because it has nothing to evaluate. This guardrail only works when RAG's Injection Mode is set to Inject via Template and the template in use (default or custom) still contains the <CONTEXT>/</CONTEXT> pair. Adding this guardrail while leaving RAG on its default injection mode produces a configuration that looks active but never actually evaluates any request — if you plan to pair RAG with Groundedness Protection, set Injection Mode to Inject via Template.
A request where no detectable context is found (RAG never ran, or ran in an undetectable mode) is indistinguishable from one where the context happened to be empty — both fall through to the same silent no-op. This is the only behavior-preserving choice available without widening the guardrail's own scope.
Unary Responses Only
This guardrail does not run on streamed responses — when a response streams, evaluation is never triggered at all, neither blocking nor flagging. That is a deliberate design choice: a groundedness verdict needs the entire answer, and a streamed response's full text is only known once the stream ends; the buffer-and-scan machinery used by Off-Topic Protection's streaming support has not been wired up for this guardrail.
The Judge Is Mandatory
Unlike some other AI guardrails (Prompt Protection, Off-Topic Protection), Groundedness Protection has no built-in/algorithmic check — there is no cheap, deterministic way to answer "is this text supported by that text," so evaluation always goes through an llm-judge (see External Provider Guardrails). Two requirements follow from that, both enforced at save time:
- The policy cannot be saved without a judge (
externalGuardrail) configured. - The judge's prompt template cannot be left blank — unlike Prompt Protection, where a blank template falls back to Native passthrough, a blank template here would fall back to a generic safety judge that doesn't answer a groundedness question at all, producing a meaningless verdict. A new policy is pre-filled with a groundedness-specific starting template; adapt it to your needs, but it must always ask a groundedness question.
The judge's own settings (model name, timeout, JSON mode, max input/output tokens) are configured exactly like the llm-judge adapter in External Provider Guardrails — including the reasoning-model warning about raising Max Output Tokens.
Action, Evaluation Mode, and Failure Handling
| Setting | Default | Note |
|---|---|---|
| Action | Flag | Block rejects the ungrounded response |
| Evaluation Mode | Inline | Off-Topic Protection defaults to Asynchronous; here the default is Inline because this is a security gate that can Block a response — correctness is prioritized over latency. Asynchronous and Observe-Only are also available. |
| Judge Failure Mode | Block on failure | If the judge call errors or times out, the response is blocked. Allow on failure lets it continue, unverified, with a visible warning. |
| Max Context Characters | 6000 | Truncation cap applied to the retrieved context before it reaches the judge |
| Max Response Characters | 4000 | Truncation cap applied to the response text before it reaches the judge |
Ordering: After Personal Data Masking and DLP
This guardrail runs on the response lane, and the judge call itself sends response content out to an external LLM provider — so it must always be ordered after Personal Data Masking and Data-Loss Protection, for the same reason described in the Policy order warning at the top of this page. If you save the policy list in an order that breaks this rule, Apinizer corrects it automatically when you save.
Adding a Guardrail
Add the guardrail you need to the relevant API proxy or policy group.
For data-loss protection, choose the built-in pattern sets or define your own; for the other guardrails, set the relevant threshold and time window.
Pick Block, Flag, or Mask, and whether the guardrail runs inline, asynchronously, or in observe-only mode.
Saving and deploying are separate steps — a saved guardrail only takes effect after you deploy it.
Presets and Policies: Bound or Localized
The personal-data, prompt-guard and data-loss protection policies do not read their rules from the catalog at request time — each rule is a copy taken when you picked it. What decides whether that copy keeps tracking the catalog is its state, shown in the Source column of the rule row.
| State | What it means |
|---|---|
| Catalog-bound | The preset owns the content. Editing the preset and saving rewrites this rule in every policy using it and marks the affected deployments "redeploy required". The rule's catalog-owned fields are read-only in the policy screen. |
| Localized | A detached, independent copy. Preset edits never reach it, and you can edit it freely in the policy screen. |
A rule added from the catalog starts out catalog-bound. The rule row switches between the two states:
- Localize — cuts the link. The current content is kept, but the rule stops receiving catalog updates and becomes editable. Use this when you want to customise one policy only.
Which fields the catalog owns differs slightly per guardrail, and everything else stays per-policy:
| Guardrail | Refreshed from the preset | Always stays per-policy |
|---|---|---|
| Prompt guard / Data-loss protection | Pattern, action, category, description | Name and enabled/disabled |
| Personal-data masking | Field name, operation, pattern type, regex, masking settings | Enabled/disabled |
Personal-data masking has no separate per-policy label: its field name is functional (it is the field actually matched in field-name mode), so the preset owns it.
Because every project can pick a shared preset (a system administrator's record), editing one refreshes its bound rules in every project of the installation and marks the affected deployments "redeploy required". Only a system administrator can edit such a preset, and the usage list they see before saving covers the whole installation. A user viewing the same preset from a project screen sees only their own project's usages. For project-owned presets both the update and the list stay inside that project.
Rules created before this behaviour carry no preset reference, so they count as localized and will not sync on their own.
Localizing is one-way: there is no action that re-binds a localized rule to a preset. Matching by name would not be trustworthy (the name is editable, so name similarity risks binding to the wrong preset), and a bound rule takes its pattern and action from the preset on every later edit — so a wrong binding turns into a silent change in security behaviour. If you want a rule back in sync, delete it and add it again from the catalog.
Importing a configuration into another environment also lands its rules localized, because the target environment's preset records are different. The same applies there: delete and re-add from the catalog.
Deleting a Preset
Deleting is never blocked. If the preset is bound anywhere, a warning lists where before the delete goes through. On confirmation the preset is removed and its bound rules are localized — their content is preserved, no deployed policy breaks, and no redeploy is requested (only the link changed, not the effective content).
Renaming a preset does not affect bound rules: they track the preset by id, not by name.
OWASP LLM Top 10 and MITRE ATLAS Signature Pack (Prompt Protection)
Prompt Protection's built-in presets ship as a versioned, integrity-checked signature pack: each built-in rule is labeled with an OWASP GenAI LLM Top 10 id and a MITRE ATLAS (Adversarial Threat Landscape for AI Systems) technique id. Both labels appear as badges next to each rule in the rule list — the ATLAS technique shows in a tooltip on hover.
The built-in pack does not yet cover the full OWASP taxonomy — today it maps two categories: jailbreak / role-override / safety-bypass / injection patterns to LLM01 (Prompt Injection), and system-prompt-leak patterns to LLM08 (Hidden Context Exposure). The corresponding ATLAS techniques are AML.T0054 (LLM Jailbreak), AML.T0051.000 (LLM Prompt Injection: Direct), and AML.T0056 (Extract LLM System Prompt).
These labels are display/reporting metadata only — they never affect Prompt Protection's own matching logic at runtime; they exist to make each rule's threat classification visible from a compliance/audit angle. Your own custom rules can carry their own OWASP/ATLAS ids too — neither field is read-only.
Versioning and integrity: the pack carries an internal version counter that ships with the product — there is no network-based auto-update channel (a deliberate choice for air-gapped installations); a new pack version only arrives through an Apinizer upgrade. A status banner above the rule list shows:
| Field | Meaning |
|---|---|
| Signature pack vN installed | The oldest version stamped across the installation's built-in rows — if any row has never been stamped, the installed version is treated as unknown and the banner shows a warning color |
| this build ships vN | The pack version embedded in the running Apinizer build |
| checksum ... | A SHA-256 digest over every built-in row (name + rule + action + category + OWASP id + ATLAS technique, order-independent) — enabled/disabled state and description are not part of the digest |
| N version(s) behind | Shown when the installed version trails the version this build ships; replaced by an Up to date badge when they match |
A pack upgrade only ever targets builtIn=true rows — a custom rule you created, even one that happens to share a name with a built-in rule, is never touched by it.
Centralized Management
All AI guardrail policies are managed centrally from the Global Policies screen, alongside other global policy types. This page describes what each guardrail does; use Global Policies to create, update, and bulk-deploy them across the API proxies that use them.
Personal-data, prompt-guard, and data-loss protection presets can also be managed through the APIops REST API; see API Reference: AI Privacy Presets, AI Prompt-Guard Presets, and AI DLP Presets.
Guardrails and the Semantic Cache
When response masking or data-loss protection is active, the semantic cache stores the masked response rather than the raw one — so a cache hit can never bypass your masking rules.
Next Steps
Create, update, and bulk-deploy guardrail policies
See how cached responses interact with masking
The same protection principles apply to agent-to-agent traffic
Monitor guardrail trigger rates
Configure the RAG policy that Groundedness Protection checks responses against