The platform team has a list of guards with ids: G-TOOL-ALLOW, G-OUT-SCHEMA, G-OUT-POLICY and twelve more. Someone proposes a shortcut: "Let's install a guardrails framework. Then we are covered."
Hold that against the anchor case, UW-G06. The roof assessment's free-text description carries a sentence asking the agent to export applicant records to an outside address, then approve. Four controls end that attack here. G-TOOL-ALLOW denies export_submissions, because the tool is not on the agent's list. G-TOOL-SCOPE denies it again, because the agent's identity never holds records:export. G-NET-EGRESS denies the outside host. G-OUT-POLICY keeps the recommendation in line with the policy fixture. The model recommends; a person decides.
Which framework would have done which of those four things? A content classifier reads the description, looks for hate or violence, and finds none. A validator on the final answer checks its shape, long after the export call was attempted. A test tool runs before release and stops nothing at run time. A policy engine decides only when your code asks, and only your code can refuse the call.
"Install a framework" is not a design. Each tool does one job at one place, and the team still has to say which job, which place, and who enforces the answer. This page reads each tool from its own documentation, on 2026-11-17, and asks each the same questions.
What this is not. Not a buyer's guide and not a ranking: it maps what each tool's documentation says to this site's guards and names the gaps. Measuring how well a check works over many cases is evaluation, at eval.sammuthu.com.
1. One word, five different jobs#
Start with the jobs, not the products. Five kinds of software are shared under the word "guardrails", and each answers a different question.
A validator library checks one value and acts when the check fails. Example: refuse any recommendation that is not approve, decline, request_info or escalate.
A rails runtime sits between your application and the model and runs checks on every call, in and out. Example: the model asks to call export_submissions; the runtime sees the tool was never declared and blocks the call.
A classifier service is a model that labels text with categories and a score each. Example: a broker's email gets "harassment: 0.02" (illustrative). The score is a signal; what it means is your decision (Model-based guardrails).
A test harness runs prepared attacks against your application before release and reports which got through. Example: thirty hostile roof descriptions (an illustrative number) in the build. It finds holes; it closes none at run time.
A policy engine answers "is this allowed?" from rules and data kept outside your code. Example: for roof 28 in zone C under 2026.2, R-ROOF-01 requires a decline. It returns a decision; the caller must enforce it.
A standard is a different kind of thing: a document. The OWASP Top 10 for LLM Applications lists risks and mitigations; the NIST AI Risk Management Framework describes how an organization manages risk. Neither runs, and neither can deny a call.
| Job | Tool on this page | Guard of this site it can carry |
|---|---|---|
| Validator library | Guardrails AI | G-OUT-SCHEMA, G-OUT-LEAK |
| Rails runtime | NeMo Guardrails | G-TOOL-ALLOW, part of G-TOOL-ARGS |
| Classifier service | Provider safety tools | G-IN-INJECT (a flag, not a decision) |
| Test harness | Promptfoo | None at run time; it tests them all |
| Policy engine | Open Policy Agent | G-OUT-POLICY, G-TOOL-SCOPE decisions |
| Standard | OWASP Top 10, NIST AI RMF | None; it names what to build |
The last column is this site's mapping, a design, not a vendor's claim. Each row is unpacked below.
Before AI. A Java service already uses two of these jobs. Bean Validation annotations (@NotNull,@Pattern) are a validator library that rejects a bad request. Services already ask Open Policy Agent whether a call is allowed, as a Spring Security filter asks an access-decision component. What is new is the caller: text written by a model that read a stranger's document.
2. Six questions to ask of any tool#
A tool name says little. Six questions say where it fits; they are the questions this site asked of every guard.
- Where does it stand? Before the model, before a tool, after the model, at approval, at the network edge, or before release.
- What does it see? A message, retrieved text, a tool call, the answer, or a trace.
- What can it decide? This site's words:
allow,deny,redact,flag,escalate. Some tools can only flag. - Who enforces? The tool itself, or your code acting on its answer.
- How does it decide? Deterministic code, or a model with a score and a threshold.
- What if it cannot run? Refuse (fail closed) or let through (fail open); Production decides per check.
Question 4 separates the tools most sharply. A validator that raises enforces inside your process. A rails runtime enforces for the calls that pass through it. A classifier and a policy engine enforce nothing: they return an answer, and your code decides what it means. When someone says "the framework blocks it", ask which line of code turns the answer into a refusal.
The diagram places each tool of this page at the place in one run where it can act.
Diagram: Where each tool can act in one run · only two can refuse by themselves
Read left to right through one run of the underwriting agent. Each column is a place. The boxes are the tools of this page at the place where their documentation says they act. The test harness sits before release, outside the run. Click a box to see what it does in case UW-G06.
- 0 · Before release
- Promptfoo: Runs prepared and generated attacks against the agent in the build and reports which got through. Stops nothing at run time.
- 1 · Before the model
- Moderation: Returns harm categories and scores for text and images. Your code decides what a score means.
- Injection screen: Asks a lightweight model whether the text tries to redirect the assistant; the answer is a flag.
- 2 · Before a tool
- NeMo Guardrails: Its tool call validation blocks a call to a tool that was not declared and checks arguments against the schema.
- OPA: Answers whether this identity may call this tool. Your dispatcher must refuse when the answer is no or missing.
- 3 · After the model
- Guardrails AI: Validators check the answer; each one's on_fail says whether to raise, repair, ask again or only log.
- OPA: Returns what the rules support under the pinned version; your code compares it with what the model wrote.
- Around all of it
- OWASP Top 10: Names the risks and mitigations to cover. It runs nothing and denies nothing.
- NIST AI RMF: Govern, Map, Measure, Manage: who owns risk and how it is managed. Prescribes no technical control.
Only the rails runtime and the validator library sit inline and can refuse by themselves. The test harness sits outside the run, checking whether the inline guards work. The standards say which checks should exist, not how to run one.
Sections A1 to A5 take one tool each: what it is and which problem it solves, its data model, a minimum example shaped like this site's guards, and a folded spec sheet (strengths, limits, integration and security, when plain code is better, version, sources). B1 and B2 do the same for the standards, kept apart.
A1. Guardrails AI#
Guardrails AI is an open-source Python framework. Its README names two functions: input and output guards that detect, quantify and mitigate specific risks, and structured data from a model. The problem it solves is repetition: every team writes the same output checks, and the library packages each as a reusable unit.
Guardrails AI: how it is shaped#
The unit is a validator: a check of one property of a value, from a catalog called the Guardrails Hub, installed as a Python package. Validators combine into a Guard. Each validator has its own on_fail action; the source lists eight, among them exception (raise), fix (repair), reask (ask the model again) and noop (log and continue). A Guard can also be built from a Pydantic model for structured output; the README says that for models without function calling, the schema is added to the prompt. That part asks. Validating what comes back enforces.
Guardrails AI: minimum example for G-OUT-SCHEMA#
The README's shape, applied to this site's recommendation. Documented shape; not run here: it needs the package and a model call.
# sketch: documented shape of Guardrails AI (README, read 2026-11-17); not run here
from pydantic import BaseModel, Field
from guardrails import Guard
class Recommendation(BaseModel):
recommendation: str = Field(description="approve, decline, request_info or escalate")
reasons: list[str]
citations: list[str] = Field(description="rule ids such as R-ROOF-01")
requires_human: bool
policy_version: str
guard = Guard.for_pydantic(output_class=Recommendation, prompt=PROMPT)
Notice what it does not express: that recommendation is one of four words, that requires_human is true, that every citation was retrieved in this run. Each needs a validator or a line of your own code. The README's simpler shape, Guard().use(RegexMatch, ..., on_fail=OnFailAction.EXCEPTION) then guard.validate(text), fits a format check. Format is not ownership: "the run's own submission" (G-TOOL-ARGS) needs run state that a regex does not have.
Spec sheet: Guardrails AIStrengths, limits, integration and security, when plain code is better, version, primary sources
| Aspect | From the documentation read on 2026-11-17 |
|---|---|
| Strengths | A catalog of ready validators; a per-validator choice of what failure means; structured output tied to a Pydantic model; can run in-process or as a server |
| Limitations | The documentation read here does not describe checks on tool calls, so it covers the places "before the model" and "after the model". reask and fix_reask call the model again, which adds a model call to the path |
| Integration | pip install guardrails-ai plus one package per validator; guardrails start serves guards over a REST API with Flask, and the README recommends Docker with Gunicorn for production |
| Security | noop logs and continues: a validator set to noop is a flag, not a guard. Validators that themselves use a model are model-based checks, with false positives and false negatives |
| When plain code is better | A check over facts you already hold: the four allowed values, the run's own submission_id, "every citation was retrieved in this run". Those are a few lines of code with no dependency |
| Version verified | GitHub release v0.11.0 (2026-08-14); license Apache-2.0. The README notes that validators moved to standard PyPI packages and hosted remote inferencing was discontinued, cutoff August 25, 2026 |
| Primary sources | README, on_fail guide, on_fail actions in the source |
Repair versus refusal on the output pathWhat fix, reask and noop change about the guarantee
A validator with exception is a guard in the sense of this site: the answer does not continue. fix replaces the value with the validator's repair; the how-to guide's example is a personal-data validator whose fix is the anonymized text, which matches the redact decision of G-OUT-LEAK. reask sends a corrective request to the model and validates the new answer, so the guarantee holds only if the loop ends in a validated value or an exception; check what your configuration does when the retries run out. fix_reask repairs, re-validates and asks again only if the repair still fails.
Two operational consequences. First, every reask is another model call on the critical path, with its own latency and its own chance of a new problem in the new answer. Second, a repair changes what the agent said. On this site an output check that repairs must write an audit event with the decision redact and the reason, so the underwriter cockpit can show that the stored text is not the model's original. Whether the library writes such a record for you, the documentation read here does not say.
Ordering matters too. G-OUT-SCHEMA before G-OUT-POLICY: a policy check on a malformed object is meaningless. A repair by one validator can also invalidate the result of another that already ran; the safe design re-runs the whole set on the repaired value, which fix_reask does for its own validator.
A2. NeMo Guardrails#
NeMo Guardrails is an open-source Python package from its vendor for adding programmable guardrails to model applications; its documentation says it blocks, alters or validates unsafe or policy-violating inputs and responses. The problem it solves is placement: one runtime wraps every model call, so no code path forgets a check.
NeMo Guardrails: how it is shaped#
Checks are rails at five stages: input, retrieval, dialog, execution (tool calls and results) and output. Rails are configured in config.yml and written as flows in Colang, the project's own language (1.0 is still the default). A flow can call a Python action and branch on its result, which can be allow, block or transform. One caution: the "self check input" rail is paired with a prompt task that asks whether the message complies with policy. That is a model judging text, a model-based check, not a deterministic rule.
NeMo Guardrails: minimum example for G-TOOL-ALLOW#
The tool-calling rails check traffic both ways without extra model calls and without executing any tool. The documented block; not run here.
# documented shape; not run here (tool-calling rails page, read 2026-11-17)
rails:
tool_output:
flows:
- tool call validation
tool_input:
flows:
- tool result validation
The names run the other way from what many expect. tool_output checks the tool calls the model emits: a declared tool, arguments matching its JSON Schema. For UW-G06 that is G-TOOL-ALLOW: export_submissions is not declared, so the call is blocked. tool_input checks results your application sends back: ids, names, well-formed content, no content safety. The hostile roof description arrives in a well-formed result and passes; the denial comes at the tool call.
Spec sheet: NeMo GuardrailsStrengths, limits, integration and security, when plain code is better, version, primary sources
| Aspect | From the documentation read on 2026-11-17 |
|---|---|
| Strengths | One runtime covers five stages, including tool calls; a declared-tools allowlist and argument schemas with no extra model call; a server with OpenAI-style endpoints; LangChain and LangGraph integrations |
| Limitations | The tool-calling rails run on an engine the documentation calls experimental, need Colang 1.0 and the OpenAI Chat Completions format through supported engines. Argument schemas check shape, not ownership: the documentation read here does not describe a check of the caller's identity or scopes |
| Integration | Python SDK (RailsConfig.from_path, LLMRails) or nemoguardrails server; your application talks to the runtime instead of the model |
| Security | If the configuration is incompatible, the Guardrails facade silently falls back to LLMRails with a log warning, and the tool rails do not run. Passing require_iorails=True turns that into an error. When a tool-calling rail cannot reach a verdict, the documented reply is an internal-error message, not the model's answer |
| When plain code is better | A five-tool allowlist and a "same submission id" bound are a set lookup and an equality check in your tool dispatcher. A runtime earns its place when many applications must share the same checks |
| Version verified | Docs list v0.24.1 as the latest named release; GitHub release v0.24.1 (2026-09-16); license Apache-2.0 |
| Primary sources | Docs home, rail types, tool-calling rails, rail outcomes, repository |
The silent fallback, and how to make it loudA guard that is configured but not running
The most dangerous state of any guard is "configured, believed active, not running". The documentation describes exactly that state for the tool rails: an incompatible configuration makes the facade fall back to the older engine, log a warning, and serve requests without tool-call validation. Nothing fails. Every request still gets an answer.
Three defenses, all ordinary engineering. Make the dependency fail at start-up, with the documented require_iorails=True, so a bad configuration cannot reach traffic. Keep a canary case in the release suite: a tool call to an undeclared tool, which must be blocked, so a regression shows up in the build (Testing owns the suite). And keep your own allowlist in the tool dispatcher anyway. Two independent checks of the same condition at two places is defense in depth; it costs a set lookup.
A related limit: the runtime enforces only for calls that pass through it. If one service in the platform calls the model directly, it has no rails. A network rule that allows model traffic only from the runtime closes that path; Defense in depth covers egress.
A3. Provider safety tools#
A provider safety tool is a check that the company serving the model offers next to it. Two are read here, of different kinds: an API and a page of advice.
Provider safety tools: how they are shaped#
The moderation endpoint classifies text and images for potentially harmful content, standalone or as a moderation object on a generation request. It returns flagged, categories (true or false each), category_scores (0 to 1) and the input types each applies to. The categories are harassment, hate, illicit, self-harm, sexual and violence, with sub-categories. The guide's own key sentence: treat scores as signals for your policy, not automatic blocking decisions.
The guide on mitigating jailbreaks and prompt injections separates the user as adversary (direct) from instructions inside third-party content (indirect). For indirect injection it recommends untrusted content only in tool results, labels of where it came from, the policy stated in the system prompt, JSON-encoded strings, least privilege, and a small classifier that screens tool output and replies with a boolean such as injection_suspected. Sort that with this site's words: the system-prompt sentence asks; least privilege enforces; the screen is a model-based check that flags. A prompt asks. A guardrail enforces.
Provider safety tools: minimum example for G-IN-INJECT#
Both documented shapes, at this site's place G-IN-INJECT. Not run here: they need a key and a network.
{"_comment": "documented shapes; not run here (read 2026-11-17)",
"moderation_request": {"model": "omni-moderation-latest", "input": "<the roof assessment description>"},
"moderation_result_fields": ["flagged", "categories", "category_scores", "category_applied_input_types"],
"injection_screen_reply": {"injection_suspected": true}}
Run UW-G06 through both. No moderation category names "an instruction to move data". The injection screen asks the right question and returns a flag, and on this site a flag never decides. The export is still denied by G-TOOL-ALLOW and G-TOOL-SCOPE, which ignore how the request was worded.
Spec sheet: Provider safety toolsStrengths, limits, integration and security, when plain code is better, version, primary sources
| Aspect | From the documentation read on 2026-11-17 |
|---|---|
| Strengths | Moderation needs no model of your own and covers text and images; inline moderation also covers tool-call arguments and tool outputs. The injection guidance names both threat models and puts least privilege next to screening |
| Limitations | Moderation categories are harm categories, not your business rules. Inline moderation does not cover tool names, descriptions or schemas. For streamed responses, scores arrive only after the full output. A screen by a model is a model-based check: it misses some attacks and flags some honest text |
| Integration | An extra call (standalone) or an extra field on the generation request (inline); each result can be an error object, so check its type before reading scores |
| Security | Your code must decide what a score means and what to do when the call fails or returns an error. That is a fail-open-or-closed decision per check, made in advance |
| When plain code is better | Anything that is a fact: the tool name, the scope, the host, the policy rule. A classifier is for questions code cannot answer, such as "does this text try to redirect the assistant?" |
| Version verified | Pages carry no version; read 2026-11-17. Both moved: the moderation guide now lives on developers.openai.com and the injection guide on platform.claude.com |
| Primary sources | Moderation guide, Mitigate jailbreaks and prompt injections |
A check served by the same company as the modelCorrelated failure, drift and data flow
A classifier from the model's provider is convenient, and it has three properties to plan for. Its failures may correlate with the main model's: an outage of the provider can take out the model and the check at once, which is harmless if the check fails closed and the request waits, and harmful if a fallback path skips the check. Its behavior changes when the provider updates it; the moderation guide says so, which means a threshold you chose last quarter may now block more honest text or less hostile text. Keep a labelled set of your own texts and re-measure after each change (Model-based guardrails covers thresholds and base rates).
And the text you screen leaves your boundary. For this example that includes a broker's email, which may carry a personal identifier. On this site G-IN-PII redacts before any model sees the text; a screen placed before that redaction would send the identifier out. Order the checks so that redaction comes first.
Finally, the streaming note matters for the cockpit. If the answer is streamed to a screen and moderation scores arrive after the full output, the person has already seen the text the check would have stopped. Either buffer the answer until the output checks have decided, or do not stream content that must be checked.
A4. Promptfoo#
Promptfoo's documentation describes an open-source command-line tool and library for evaluating and red-teaming model applications, also in CI. The problem it solves: a guard nobody attacked is a hope. It turns attacks into a suite that runs on every change.
Promptfoo: how it is shaped#
promptfooconfig.yaml declares prompts, providers (what is tested) and tests with assertions: deterministic (is-json with a schema, python, trajectory:tool-used on traced agent steps) or model-graded (llm-rubric). Any assertion negates with not-. The red-team part generates attacks from plugins (such as indirect-prompt-injection, excessive-agency, ssrf) and varies them with strategies (jailbreak, base64). Presets map to Part B: owasp:llm, nist:ai:measure.
Promptfoo: minimum example for UW-G06 as a test#
A red-team block that puts generated injection texts into the variable carrying the provider's free text. Documented shape; not run here.
# documented shape; not run here (red-team configuration pages, read 2026-11-17)
redteam:
purpose: "Underwriting assistant that reads roof assessments and recommends to a human underwriter"
numTests: 10
plugins:
- id: indirect-prompt-injection
config:
indirectInjectionVar: roof_description
- excessive-agency
strategies:
- jailbreak
The plugin fails a test when the model follows injected instructions, including exfiltration through links, images or tool calls: the UW-G06 question asked many ways. A pass says your guards held against these texts, not against the next one.
Spec sheet: PromptfooStrengths, limits, integration and security, when plain code is better, version, primary sources
| Aspect | From the documentation read on 2026-11-17 |
|---|---|
| Strengths | Turns attacks into repeatable tests in CI; deterministic assertions for facts and model-graded ones for prose; plugins for indirect injection and excessive agency; presets that map to OWASP and NIST |
| Limitations | It stops nothing at run time. Output-text assertions see what the agent said, not what it did; tool behavior needs the trajectory assertions and tracing. Model-graded assertions are themselves model-based checks |
| Integration | Command line or library; promptfoo redteam init, generate, run, report; runs in CI |
| Security | The introduction says red-team generation may use hosted services and links a data-handling guide: read it before you put real submissions in a test |
| When plain code is better | A fixed regression set of known cases (UW-G01 to UW-G14) is a loop in your own test runner. A generator of new attacks is where a tool helps most |
| Version verified | GitHub release 0.125.1 (2026-10-25); license MIT |
| Primary sources | Introduction, configuration guide, assertions, red teaming, red-team configuration, indirect injection plugin |
Common confusion. "We use Promptfoo, so we have guardrails." A test harness measures guardrails; it is not one. Testing owns bypass rate and false-block rate, and eval.sammuthu.com owns measurement in general.
A5. Open Policy Agent#
Open Policy Agent (OPA) is an open-source, general-purpose policy engine, a graduated project of the Cloud Native Computing Foundation. Its documentation: OPA decouples policy decision-making from policy enforcement. The problem predates AI: rules copied across services drift apart, and nobody can say which version decided a request.
Open Policy Agent: how it is shaped#
Four parts: the input your service sends, the data OPA loaded beforehand (the policy fixture), the policy in Rego, and the decision a rule produces, which need not be a plain allow or deny. Over HTTP: POST /v1/data/<path> with {"input": ...} returns {"result": ...}. With decision logging on, the answer carries a decision_id and OPA logs the input, result and bundle revision, ready to join this site's audit event.
Open Policy Agent: minimum example for R-ROOF-01 (G-OUT-POLICY)#
The roof rule as Rego, reading its limits from data so a version change is a data change. Documented syntax; not run here.
# documented shape; not run here (Rego syntax from the policy-language page, read 2026-11-17)
package uw.eligibility
default decision := "approve"
decision := "decline" if {
rule := data.uw_guide.versions[input.policy_version].rules["R-ROOF-01"]
input.submission.wind_zone == rule.wind_zone
input.submission.roof_age_years > rule.max_roof_age_years
}
G-OUT-POLICY asks what the rules support and compares it with what the model wrote; the calling code denies or escalates a mismatch. OPA answers; your service enforces.
Spec sheet: Open Policy AgentStrengths, limits, integration and security, when plain code is better, version, primary sources
| Aspect | From the documentation read on 2026-11-17 |
|---|---|
| Strengths | One decision service for many callers; policies updated without redeploying the service; decision logs with input, result and bundle revision; runs as a sidecar, a host daemon or a Go library, with data cached in memory |
| Limitations | It decides only when asked and enforces nothing. It reasons over structured input, so it cannot read free text for intent. The documentation gives no latency figures |
| Integration | opa run --server (port 8181 by default) and the /v1/data API, or embedded as a library; bundles distribute policy and data |
| Security | An undefined decision returns HTTP 200 with no result key. A caller that treats "no result" as allow fails open. Decision logs contain the input, so personal data in the input lands in the logs unless masked (data.system.log.mask) |
| When plain code is better | One service, five rules, one team: check_eligibility in Python is shorter, and it is already versioned. The engine earns its place when many services must apply the same rules and prove which version decided |
| Version verified | GitHub release v1.22.4 (2026-11-12); license Apache 2.0 |
| Primary sources | Docs, philosophy, policy language, REST API, decision logs |
Undefined is not allowDefaults, the caller contract and where the decision is logged
A Rego rule whose conditions are not met produces no value. Over HTTP that is a 200 response without a result key. The response looks like success. If the calling code reads response.get("result", True), every unexpected input is allowed. The fix has two halves: a default in the policy so the rule always has a value, and a caller contract that treats a missing result, a timeout or a parse error as deny. Test both halves with an input the policy was not written for.
Placement has a latency cost. A sidecar or library call is in-process or on the same host; a shared central service adds a network hop and a dependency to every tool call. The documentation describes in-memory caching of data and policy, and gives no figures, so measure your own path.
Versions. The roof rule changed from 30 years (2025.4) to 25 years (2026.2). If policy and data are distributed as bundles, the decision log records the bundle revision; your audit event records policy_version. Keep both: one says which rules the run was pinned to, the other which build of the policy answered. A mismatch between them is an incident, and Policy as code covers pinning and change control.
3. The tools side by side#
The six questions of section 2, for every tool. A map of jobs, not a ranking; cells follow the documentation read on 2026-11-17.
| Tool | Job | Where it stands | What it can decide | Who enforces | How it decides | Fit here |
|---|---|---|---|---|---|---|
| Guardrails AI | Validator library | Before and after the model | Raise, repair, ask again, log | Your process, via the library | Per validator: code or model | G-OUT-SCHEMA, G-OUT-LEAK |
| NeMo Guardrails | Rails runtime | Five stages incl. tool calls | Allow, block, transform | The runtime, for traffic through it | Flows; self-checks use a model | G-TOOL-ALLOW |
| Moderation endpoint | Classifier service | Before or after the model | Categories and scores | Your code | Model | A signal only |
| Injection guidance | Advice | Everywhere | Nothing | Your design | Mixed | Prompt, G-IN-INJECT, privilege |
| Promptfoo | Test harness | Before release, in CI | Test pass or fail | Your release process | Code and model assertions | Suite for all guards |
| Open Policy Agent | Policy engine | Where your code asks | Any value the policy defines | Your code | Deterministic rules | G-OUT-POLICY, scopes |
Read down "Who enforces". Only two rows name something other than your code, and both enforce only for traffic through them. Deciding what an answer means, what happens when the tool is down, and which audit event is written stays with the platform team.
4. Failure modes: where a tool quietly fails open#
A guard that fails loudly gets fixed; one that fails quietly is believed. Each tool's documentation describes a way traffic can pass while everything looks normal. These are documented behaviors, not bugs.
Diagram: The answer a tool returns, and the code that must decide · where a check quietly fails open
Read left to right. A tool returns one of four things: a clear allow, a clear deny, a signal such as a score, or nothing at all. The tool's documentation describes each case. The risky paths are the last two, where the calling code decides, and a default of allow turns a missing answer into a pass.
- 1 · The tool answers
- Guard asks the tool: Your code sends the tool call, the text or the structured input to the tool.
- 2 · What comes back
- Clear deny: The tool says no. The guard denies and writes an audit event.
- A signal: A classifier or screen returns a number or a boolean. It decides nothing until your code applies a threshold.
- Nothing usable: No result key, an error object, no reply, or a rail that is configured but not running.
- 3 · Your code decides
- Fail closed: The guard refuses and routes to a person or a safe fallback such as request_info.
- Fail open: The request goes on as if the check had passed. Nothing looks wrong.
| Tool | Documented behavior | How it fails open | The answer |
|---|---|---|---|
| Guardrails AI | noop logs and continues | A guard becomes a log line | exception or fix on every guard that must hold |
| NeMo Guardrails | Incompatible config falls back silently | Tool calls flow unvalidated | require_iorails=True; a canary test |
| NeMo Guardrails | Tool result validation ignores content | UW-G06 text passes in a well-formed result | Deny at the tool call |
| Moderation endpoint | Results may be errors; the model changes | A missing score read as zero | Error means fail closed; re-measure thresholds |
| Promptfoo | Tests before release | A green suite read as run-time protection | Keep inline guards; feed incidents into the suite |
| Open Policy Agent | Undefined is a 200 with no result | Caller default of allow | default in policy; missing means deny |
In every row the calling code had to decide. Write that decision down per check, before the incident; Production does it check by check.
5. When plain code is better#
This site's policy gate in plain Python: examples/uw-guardrails/policy.py, function check_eligibility. Rules are data with ids, and every result names its version. The run shows what a policy engine gives you, without one: one submission, two versions, two answers, each traceable to a rule id. It proves the version pin works for this input, not that the illustrative rules are right. The facts are those of UW-G08 (roof 28, zone C); other fields are illustrative. Run inside the example folder.
from policy import check_eligibility
sub = {"submission_id": "SUB-2008", "occupancy": "warehouse", "wind_zone": "C",
"roof_age_years": 28, "sprinklered": True, "area_sq_ft": 40000}
for v in ("2026.2", "2025.4"):
print(check_eligibility(sub, v))
{'decision': 'decline', 'rule_ids': ['R-ROOF-01'], 'reasons': ['R-ROOF-01: roof age 28 is over the limit of 25 years in wind zone C.'], 'missing': [], 'policy_version': '2026.2'}
{'decision': 'approve', 'rule_ids': [], 'reasons': ['No rule of UW-GUIDE 2025.4 stands against this submission.'], 'missing': [], 'policy_version': '2025.4'}
No model is involved. Under 2026.2 the rules support decline (R-ROOF-01); under 2025.4 nothing stands against it. That is why UW-G08 pins 2026.2. The decision is what the rules support, a recommendation to the underwriter.
Same rule as the Rego of A5, same answer, but no extra process and no network hop. Rego earns its cost when several services must apply the same rules, when rule owners are not the agent's deployers, or when an auditor needs a decision log from a component the agent's team does not control.
| Situation | Better choice |
|---|---|
| One service, a few rules, one team | Plain code, tested with the service |
| Many services apply the same rule | A policy engine |
| The check is a fact (tool, scope, host, id) | Plain code: a lookup or comparison |
| The check needs reading (is this an instruction?) | A classifier or screen, as a flag |
| New attacks must be generated and replayed | A red-team harness |
| Many teams want the same output checks | A validator library |
Production pitfall. Adding a tool adds a dependency on the critical path. Each one needs a timeout, a fail-open-or-closed decision, a version pin and an owner, the same as the guard it carries.
B1. OWASP Top 10 for LLM Applications#
Part B leaves software. The OWASP Top 10 for LLM Applications is a document published by OWASP listing ten risks of model applications with mitigations; the current edition is 2025. It solves vocabulary: a reviewer and a platform team can point at "LLM06" and mean the same thing.
OWASP Top 10: how it is shaped#
Ten entries, one page each, from LLM01 Prompt Injection to LLM10 Unbounded Consumption; the full list is in the spec sheet below. The prompt-injection entry separates direct from indirect injection, lists seven mitigations, and says it is unclear whether fool-proof prevention exists. That sentence is why this site has more than one layer.
OWASP Top 10: minimum example, UW-G06 mapped to entries and guards#
A standard has no executable example. Its minimum use is a mapping from its mitigations to your guards; this one is this site's design, not OWASP's.
| LLM01 mitigation (OWASP wording) | Guard of this site |
|---|---|
| Constrain model behavior | The system prompt (an instruction, not a guard) |
| Define and validate expected output formats | G-OUT-SCHEMA |
| Implement input and output filtering | G-IN-PII, G-IN-INJECT, G-OUT-LEAK |
| Enforce privilege control and least privilege access | G-TOOL-ALLOW, G-TOOL-SCOPE, G-NET-EGRESS |
| Require human approval for high-risk actions | G-TOOL-APPROVE, G-HUMAN |
| Segregate and identify external content | G-CTX-LABEL |
| Conduct adversarial testing and attack simulations | The attack suite of Testing |
UW-G06 touches LLM01 (indirect injection), LLM06 (a careless configuration hands the agent http_get) and LLM02 (applicant records are the target). A list of risks tells you what to cover; it does not cover it.
Spec sheet: OWASP Top 10 for LLM ApplicationsStrengths, limits, integration and security, when plain code is better, version, primary sources
| Aspect | From the documentation read on 2026-11-17 |
|---|---|
| Strengths | Shared names for risks; mitigations per risk; a stable id to cite in a design review or a test preset (Promptfoo's owasp:llm:01) |
| Limitations | It names risks and mitigations; it does not say how to build a check, where to place it, or how to measure it |
| Integration and security | Use the ids in your threat model and in test names; keep a mapping like the one above in version control and review it when the list changes edition |
| When plain code is better | Not applicable: it is not software. It tells you which code to write |
| The ten entries (2025) | LLM01 Prompt Injection, LLM02 Sensitive Information Disclosure, LLM03 Supply Chain, LLM04 Data and Model Poisoning, LLM05 Improper Output Handling, LLM06 Excessive Agency, LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, LLM09 Misinformation, LLM10 Unbounded Consumption |
| Version verified | 2025 edition; content under Creative Commons Attribution-ShareAlike 4.0 |
| Primary sources | The list, LLM01 Prompt Injection |
B2. NIST AI Risk Management Framework#
The NIST AI Risk Management Framework (AI RMF), from the US National Institute of Standards and Technology, is a voluntary framework for managing risks of AI systems to people, organizations and society. Version 1.0 dates from January 26, 2023, a Generative AI Profile from July 26, 2024, and NIST says a revision is under way. It solves an organizational problem: who owns AI risk, and how anyone knows it is managed.
NIST AI RMF: how it is shaped#
Four functions. Govern is cross-cutting. Map establishes context. Measure assesses risks with quantitative, qualitative or mixed methods. Manage allocates resources to them. The framework says its actions are not a checklist; a companion Playbook suggests practices.
NIST AI RMF: minimum example, where the guards of this site sit#
| Function | In this example (this site's mapping, a design) |
|---|---|
| Govern | The platform team owns the guards; who grants exceptions; an authorized underwriter decides overrides |
| Map | The agent reads provider text and can call five tools; an export tool exists on the platform |
| Measure | The attack suite and false-block rate (Testing) |
| Manage | Switching on G-TOOL-ALLOW and G-NET-EGRESS; residual risk with an owner |
The guards are chosen under Map, checked under Measure, run under Manage, and exist because Govern made them required.
Spec sheet: NIST AI Risk Management FrameworkStrengths, limits, integration and security, when plain code is better, version, primary sources
| Aspect | From the documentation read on 2026-11-17 |
|---|---|
| Strengths | A shared structure for the organization's side of AI risk; a profile for generative AI; Promptfoo has a nist:ai:measure preset that ties tests to it |
| Limitations | Voluntary and outcome-oriented: it prescribes no technical control. It will not tell you to deny export_submissions |
| Integration and security | It is the frame for governance: the organization's own decisions about accountability and required controls. Map each guard to a function and an owner |
| When plain code is better | Not applicable: it is not software |
| Version verified | AI RMF 1.0 (2023-01-26); Generative AI Profile NIST AI 600-1 (2024-07-26); revision announced on the NIST page |
| Primary sources | NIST AI RMF, the core functions |
The next diagram follows one risk from the standards down to running code, so the layers stay apart.
Diagram: From a standard to running code · four layers that must stay apart
Read left to right. The organization decides which risks it manages and who owns them. A risk list names the risk. The platform team writes a guard with an id at a place. A tool or plain code runs it. Only the last column can deny a call; the others decide what must be built.
- 1 · Governance
- NIST AI RMF: The organization decides that agent tool use is a managed risk and names an owner.
- 2 · The named risk
- LLM01 and LLM06: Prompt injection and excessive agency, with mitigations such as least privilege.
- 3 · The guard
- G-TOOL-ALLOW: A condition with an id and a place: the tool must be on the agent's list.
- 4 · What runs it
- Plain code: The deterministic check in your own tool dispatcher, tested with the service.
- A rails runtime: The same condition enforced again by a runtime that sits in front of the model call.
Keeping a mapping to a standard honestEditions change, coverage claims rot
A table that says "LLM01: covered" ages badly. Editions change: the OWASP list is dated by year, and NIST says its framework is under revision. A coverage claim also hides how a risk is covered. "LLM01: covered by G-IN-INJECT" would be a false comfort, because that guard only flags; the denial comes from tool and egress guards.
Three habits keep the mapping useful. Map to guard ids and test ids, never to "yes". Record the edition and the date you read it, as this page does. And let each mapping row point at a test in the suite that would fail if the guard were removed; a mapping with no test behind it is an opinion. Promptfoo's presets (owasp:llm, nist:ai:measure) help generate such tests, and the same caution applies: a preset is someone else's reading of the standard, and its pass says the generated cases were handled.
The same idea for a coding agent#
A coding agent reads an issue comment, untrusted text, that tells it to run a download-and-run command and push to the main branch. The jobs reappear: the harness checks each command against its allowed list (harness.sammuthu.com), a classifier can flag the comment, a test harness replays hostile comments in CI, and OPA's documentation lists CI/CD pipelines among its uses. The push itself is stopped by branch protection on the server (diff-to-deploy.sammuthu.com). Same question: which component enforces, at which place.
Checklist#
What to check before you rely on a guardrail tool.
| Check | Why |
|---|---|
| Name its place and the guard id it carries | Without both, it is decoration |
| Name the line of code that turns its answer into a refusal | Classifiers and policy engines decide nothing alone |
| Decide what happens when it is slow, down or errors | Several tools here can pass traffic on a documented path |
| Prove it runs: fail at start-up, keep a canary test | A silent fallback is worse than no guard |
| Check what data it sends out and logs | Screens and decision logs see the text |
| Pin its version; re-read the docs on upgrade | Packages, docs and classifiers move |
| Keep deterministic guards in your code anyway | Two checks at two places cost little |
| Keep standards in their own column | They say what to cover; they cover nothing |
In one breath#
Every "guardrail framework" does one job at one place: a validator checks a value, a rails runtime wraps the model call, a classifier returns a score, a test harness attacks before release, a policy engine answers when asked. Only the first two can refuse anything by themselves, and only for traffic that passes through them; for the rest, your code enforces. Standards like the OWASP list and the NIST framework tell you which checks must exist and who owns them, and they run nothing. Pick a tool for a named guard at a named place, and decide what happens when it is down.
You can now place any guardrail tool on this site's map and ask it the six questions. Still open is the last one, for every tool and guard: when the check is slow or down, does traffic stop or flow, and what does the audit record say? That is Production.