Chat policy enforcement
A chat policy is a rule a system administrator writes that inspects assistant conversations
and decides whether they are allowed to proceed. Policies are configured under
Settings → AI → Policy Enforcement (#/manage/ai/policies). The whole page is system-admin
only, and a read-only administrator can review policies and verdicts but not change them.
Two kinds ship with the workspace: a forbidden-keywords list, and a local safety classifier. Each policy carries its own answer to three questions — when it runs, what it looks for, and what a match means.
The two outcomes
Every policy is configured with what a match produces, under On a match:
| Outcome | What happens | Where it shows |
|---|---|---|
| Flag | The conversation continues. The verdict is recorded. | The chat gets a warning icon in the chat list; the verdict appears in the Evaluations tab. |
| Reject | The turn is refused if the policy runs before the chat. Elsewhere the answer already exists, so the rejection is recorded rather than enforced. | The user gets the assistant's refusal; the verdict appears in the Evaluations tab. |
Policies run in order. A run stops at the first rejection — there is nothing left to decide — but carries on past a flag, so a reviewer sees every policy that had something to say about the same conversation.
Only a flagged or rejected outcome is stored. A conversation that passes every policy leaves nothing behind.
When a policy runs
The When to run dropdown offers three moments:
| Setting | What it sees | Pick it when |
|---|---|---|
| Before the chat | The exact prompt list about to be sent to the provider, including the user's new message. | You want the rejection to actually block the answer. This is the only moment where a rejection stops anything. |
| After the chat | The same list plus the assistant's reply. | The answer is what you are judging, or the check is too slow to keep a user waiting. |
| Batched, when idle | Whole conversations, swept in the background. | The check is expensive and you would rather pay for it when nobody is using the workspace. Nothing is blocked, and a verdict can arrive minutes or hours after the conversation. |
The system prompt is never matched against. It is workspace-authored — the thing policies protect, not something to test — and matching it would flag every conversation of an assistant whose prompt happens to quote a forbidden phrase.
The two policy kinds
The Policy type dropdown offers:
| Policy type | How it decides | Run modes offered |
|---|---|---|
| Forbidden keywords | Matches a list of words and phrases you write. No model call. | All three |
| Mistral Shieldstral 1.0 | A 3B safety classifier running on the workspace server, which judges intent rather than wording. | After the chat, Batched when idle |
Forbidden keywords
The editor takes one keyword or phrase per line under Forbidden keywords. Matching ignores
case and treats every run of whitespace as equivalent, so a phrase configured as
ignore previous instructions also matches text that wrapped it across lines or double-spaced it.
The keywords that matched are stored on the verdict, so a reviewer sees which line of the list
fired.
A new workspace is seeded with one example policy, Prompt Injection Attempts — a keyword list of the stock sentences a prompt-injection attempt is written with, set to Flag rather than Reject so a fresh workspace gets the visibility without any conversation suddenly being blocked. It is a built-in: you can edit it and switch it off, but not delete it, because the next boot would seed it again.
Mistral Shieldstral 1.0
Shieldstral is policy-adaptive rather than category-based: the policy itself carries the context and the question, and the model answers that question in a single token whose probability is the whole verdict. So the editor exposes three fields instead of a keyword list:
| Field | What it is |
|---|---|
| Question | The single yes/no question asked about the conversation, e.g. "Does this content promote physical violence?". The probability of "yes" is the score. |
| Evaluation context | What the model is reviewing and how strictly. This text is the policy — it is what makes one model policy differ from another. |
| Threshold | A score above this counts as a match. Must be between 0 and 1; the default is 0.4, a little under the model card's 0.5. Raise it to flag less, lower it to flag more. |
The model runs entirely on the workspace server through the LlmShield.Shieldstral package —
nothing is sent to an LLM provider. The weights are not shipped with the product: they are
downloaded on demand from the Enforcement tab (about 1.8 GB, the Q4_0 packing) into a
Models/Shieldstral folder under workspace storage. A container image can pre-bake them next to
the binaries instead, in which case the workspace starts ready.
A policy enabled before the weights are there passes
If the model has not been downloaded yet, the evaluator returns pass and logs a warning. The alternative — blocking every conversation on a file that is not on disk — is worse.
A forward pass of a 3B model is seconds rather than microseconds, which is why this kind is never offered Before the chat, and why the evaluator holds one model per process, scores strictly one conversation at a time, and caps itself at half the machine's cores.
What a rejected turn looks like to the user
A rejection is delivered as the assistant's own answer, not as an error. The user sees the refusal stream into the bubble they were already watching:
I'm sorry, but I can't help with this one — a workspace policy blocked it.
Reason:
<the policy's Reason field>If that looks wrong, your workspace administrator can tell you which policy applied.
The Reason you write on the policy is the part only the administrator who wrote it can explain, so it is worth writing a real sentence there.
The refusal is stored as an ordinary assistant message before the turn is closed, so it survives a reload and reads back as a normal transcript. The chat is not marked as failed and no error toast is shown — the workspace declined, it did not break. A message queued behind the refused turn still runs, and is evaluated on its own merits.
What a flagged conversation looks like
The chat list marks the conversation with an orange warning icon, tooltipped "A workspace policy flagged this conversation". That is all an end user sees; the detail lives in the admin page.
One evaluation is stored per chat, keyed off the chat, so a newer verdict replaces the previous one. It survives the chat being deleted — the evaluation is the audit trail, and deleting the conversation must not erase it.
The Policy Enforcement page
Four tabs, each addressable by URL (#/manage/ai/policies?show=policies, …?show=test, and so on).
Policies
The list of configured policies, each row showing its type, when it runs, whether a match flags or
rejects, and a toggle to enable or disable it. + opens the editor for a new one; clicking the
name opens it for editing. A built-in policy's delete button is disabled — disable it instead.
Test
Paste or type a message and every configured policy scores it, one card per policy: the verdict, the reason, what matched, and how long that evaluator took (which is where the cost of a model policy becomes visible).
The test deliberately differs from a real evaluation in three ways: it ignores the run mode, it runs policies that are switched off (marked Disabled on the card so a match is never mistaken for something live), and it writes nothing to the graph — a test is not an evaluation and must not land in the audit trail.
Evaluations
The stored verdicts, newest first (the 50 most recent), each row naming the policies that fired, the run mode, the user and the time. Opening one shows what each policy decided on the left, and on the right two tabs:
- Evaluated — the conversation exactly as the policies saw it, message by message. This is the evidence, and it outlives the chat.
- Current chat — the conversation as it stands now, or a note that it is no longer available.
The captured conversation is bounded: capture stops after 32,000 characters, the message that crosses the line is truncated, and a marker message records how many were left out.
Enforcement
Everything that governs the Batched, when idle sweep, plus the local model's state.
| Setting | Default | What it does |
|---|---|---|
| Idle when at most | 0 |
Users active in the last minute that still counts as idle. Zero means the workspace has to be completely empty. |
| Block a time window | Off | Also keep the sweep out of the UTC window below. |
| Blocked window (UTC) | 05:00 to 23:00 |
Whole hours, and it may wrap past midnight. The sweep never runs inside it. |
| Conversations per batch | 25 |
Evaluated before the gate is consulted again. Smaller batches react faster to the workspace getting busy. |
Two more conditions are not settings: the sweep also yields while any search is in flight, and it does nothing at all unless at least one enabled policy is set to Batched, when idle. A status line at the top says Running or Paused, and when paused it names the reason rather than leaving you to infer it from a queue that will not move. The gate is re-checked between conversations, so a batch that outlasts its window defers the remainder instead of finishing regardless.
Below the settings sits the Safety model section — download button, live progress while the
weights are fetched, and a Ready / Loaded badge afterwards — and a chart of the
indexes.chat_policy_queue_length metric over the last 30 minutes, so a queue that is not draining
is read next to the settings that decide whether it drains.
The window names the hours evaluation must NOT run
Blocked window (UTC) is a no-run window: the sweep is kept out of it. Reading it as "the hours evaluation is allowed to run" gets you the exact opposite schedule.
It used to be expressed that other way round — as an active window — and the defaults were
inverted along with the wording, so the current 05:00–23:00 blocked default describes the
same overnight sweep the old 23:00–05:00 active default did.
If you configured this window before the rewording, re-check it after upgrading. The
settings were renamed, so a stored window is not carried across: the tab reads back the
defaults (toggle off, 05:00 to 23:00) and your old schedule is no longer in force.
Conversations that are never evaluated
- The admin assistant's own chats are permanently exempt. Sudo
is the surface an administrator uses to read and rewrite the workspace — the policies included —
so a keyword list could otherwise lock an administrator out of the tool they would fix it with,
and a conversation about a forbidden phrase is exactly the conversation an admin needs to have
there. The exemption is a link on the chat, added when the chat is created; chats predating that
link are still recognised by the assistant they run. The marker itself
(
Built-In Bypass Policy Enforcement) is filtered out of the policy list, so it never appears as something to configure. - Generation the workspace drives itself is out of scope. Enforcement is opt-in and only the conversational endpoints ask for it. A summary, a draft or a tool invoking the model is the workspace talking to itself, not a user talking to an assistant.
What chat policies are, and what they are not
Chat policies are a workspace-level guardrail over conversations. They read the messages of a chat and decide flag, reject or nothing.
They are not access control on the underlying data. A policy cannot make a document readable or unreadable, does not change what the assistant is allowed to retrieve, and is not a substitute for permissions on the graph. What a user can reach through the assistant is decided by the permission model, exactly as it is everywhere else in the workspace — see Permissions, the Access control model and the Security baseline. Use policies for the layer above that: what may be asked, and what should be reviewed.
Two further limits worth stating plainly: a keyword list is exactly as good as the list, and a classifier returns a score, not a fact — both produce false positives and misses, which is why Flag is the default and the Evaluations tab exists.
Related
- Sudo, the admin assistant — the assistant that is exempt, and why.
- LLM Configuration — the providers the conversations being policed run against.
- Configuration sync — exported workspace definitions include your chat policies (one file each) and the enforcement settings; the seeded built-in and the bypass marker are skipped.
- Monitoring — the rest of the workspace's background-work and queue-depth signals.