Guardrails & Safety

Guardrails are the runtime controls that sit around a model and constrain what it is allowed to receive, say, and do. They are not a model setting and they are not a system prompt. A system prompt is an instruction the model may or may not follow; a guardrail is a check that runs independently of the model and can block the request regardless of what the model decided.

The distinction matters because the threats are adversarial. An attacker who can influence the text reaching the model can influence the model's behaviour, and no amount of prompt engineering closes that gap. The only durable defence is enforcement outside the model: classify the input before it reaches inference, validate the output before it reaches the user, and authorise every tool call before it touches a real system.

This page covers the threat model, the four enforcement points, the controls that belong at each, and the platform services that implement them. For the evaluation half of the picture - measuring whether the system is actually correct - see RAG for grounding and retrieval quality, and Autonomous AI for human-in-the-loop oversight patterns.

The Threat Model

Start from what actually goes wrong in production, not from a generic safety checklist.

Where Guardrails Sit

There are four enforcement points. Each catches a different class of failure, and none of them substitutes for another.

Every blocked request should be logged with the policy that fired and the offending segment. Guardrails that block silently give you no way to distinguish an attack from a false positive.

Input-Side Controls

Input guardrails run before inference and decide whether the request proceeds at all.

• Prompt attack detection. A dedicated classifier scores the input for injection and jailbreak patterns. Azure AI Content Safety exposes this as Prompt Shields; Bedrock Guardrails exposes it as the PROMPT_ATTACK content filter. Both are far more reliable than regex on "ignore previous instructions".
• Input content filtering. Score for hate, violence, sexual content, and self-harm before spending inference tokens. Blocking early is cheaper than blocking late.
• Topic and intent classification. A small, fast model routes out-of-scope requests before they reach the expensive model. This also enforces denied topics such as competitor comparisons or regulated advice.
• Structural separation. Keep user input and retrieved context in separate, clearly delimited message blocks from system instructions. This does not stop injection on its own, but it makes the boundary explicit and gives downstream classifiers something to scope to.

Treat retrieved content as untrusted

Prompt Shields and equivalent classifiers scan documents as well as user prompts, and this is the part most implementations skip. Apply the same checks to anything entering the context window:

• Scan documents at ingestion time, not only at query time. A poisoned document blocked at ingestion never reaches any user.
• Re-scan retrieved chunks at query time for sources that change outside your control (web results, shared drives, customer uploads).
• Enforce document-level access control in the retrieval query itself, so a user only ever retrieves what they are entitled to see. Filtering after retrieval leaks through logs, traces, and error messages.
• Strip or neutralise imperative content in retrieved text where the use case allows. Summarisation pipelines rarely need instructions from source documents.

Output-Side Controls

Output guardrails run after generation and before the response reaches the user or any downstream system.

• Content filtering. Same categories as the input filter, applied to generated text. Set output thresholds at least as strict as input thresholds; a model can produce harmful text from an entirely benign prompt.
• PII detection and redaction. Two actions matter and they are not interchangeable. Redact (anonymise) for identifiers the user legitimately needs in context, such as their own email or phone number. Block outright for high-severity identifiers such as national insurance numbers, SSNs, and card numbers, where partial exposure is still a breach.
• Grounding and faithfulness checks. Score whether each claim in the answer is supported by the retrieved context. Bedrock calls this contextual grounding and pairs it with a relevance score for the query itself. A grounding threshold around 0.7 is a reasonable starting point; tune it against a labelled set rather than guessing.
• Schema and injection validation. If the output feeds a downstream system, validate it as untrusted data. HTML-encode before rendering, validate against the expected schema before parsing, and check generated SQL against an allowlist of tables before execution. Never execute model-generated code outside a sandbox.

Grounding checks are the control that converts "the model sometimes hallucinates" from an unmanaged risk into a measurable one. When a response fails the grounding threshold, the correct behaviour is usually to return a refusal with citations rather than the unsupported answer.

Tool and Action Authorisation

For agents, the higher-severity question is not what the model says but what it is permitted to do. An agent with a send_email tool and an injection vulnerability is a data exfiltration channel regardless of how well its text is filtered.

Authorisation belongs in the application, evaluated before dispatch, and it should be driven by a policy the model cannot see or modify:

TOOL_POLICY = {
    "search_knowledge_base": {"decision": "allow",   "scopes": {"kb.read"}},
    "get_order_status":      {"decision": "allow",   "scopes": {"orders.read"}},
    "issue_refund":          {"decision": "allow",   "scopes": {"payments.write"},
                              "max_amount_gbp": 250},
    "delete_customer":       {"decision": "deny",    "scopes": set()},
}

def authorise(tool: str, args: dict, granted_scopes: set[str]) -> tuple[str, str]:
    """Returns (allow | approve | deny, reason). Evaluated before every dispatch."""
    policy = TOOL_POLICY.get(tool)
    if policy is None or policy["decision"] == "deny":
        return "deny", f"{tool} is not in this agent's tool policy"

    # Scopes are the calling user's delegated permissions, not the agent's.
    if not policy["scopes"] <= granted_scopes:
        return "deny", f"{tool} requires scopes the caller does not hold"

    ceiling = policy.get("max_amount_gbp")
    if ceiling is not None and args.get("amount_gbp", 0) > ceiling:
        return "approve", f"{tool} above {ceiling} GBP requires human sign-off"

    return "allow", "ok"

Three principles behind that policy:

• Deny by default. An unrecognised tool name is a denial, not a pass-through. Models hallucinate tool names.
• Scope to the user, not the agent. The agent should never hold broader permissions than the person it is acting for. A shared service identity with write access to everything turns any injection into a privilege escalation.
• Tier by reversibility. Reads are allowed, low-value writes are allowed with limits, irreversible or high-value actions require approval, destructive actions are denied outright.

Approval gates, escalation, and circuit-breaker patterns for the "approve" branch are covered in Autonomous AI. Iteration caps, token budgets, and allowed-tool enforcement inside the agent loop are covered in AI Agents, and the surrounding operational scaffolding in Workflow Automation.

Platform Tooling

Bedrock Guardrails is the most complete out-of-the-box policy engine. A single guardrail object covers input and output, and attaches to any model or agent invocation:

client = boto3.client("bedrock", region_name="eu-west-1")

guardrail = client.create_guardrail(
    name="production-safety",
    contentPolicyConfig={"filtersConfig": [
        {"type": "HATE",          "inputStrength": "HIGH",   "outputStrength": "HIGH"},
        {"type": "MISCONDUCT",    "inputStrength": "HIGH",   "outputStrength": "HIGH"},
        {"type": "INSULTS",       "inputStrength": "MEDIUM", "outputStrength": "HIGH"},
        {"type": "PROMPT_ATTACK", "inputStrength": "HIGH"},
    ]},
    topicPolicyConfig={"topicsConfig": [
        {"name": "financial-advice", "type": "DENY",
         "definition": "Requests for specific financial, investment, or tax advice"},
    ]},
    sensitiveInformationPolicyConfig={"piiEntitiesConfig": [
        {"type": "EMAIL", "action": "ANONYMIZE"},
        {"type": "SSN",   "action": "BLOCK"},
        {"type": "CREDIT_DEBIT_CARD_NUMBER", "action": "BLOCK"},
    ]},
    contextualGroundingPolicyConfig={"filtersConfig": [
        {"type": "GROUNDING",  "threshold": 0.7},
        {"type": "RELEVANCE",  "threshold": 0.7},
    ]},
    blockedInputMessaging="I cannot process this request due to safety policies.",
    blockedOutputsMessaging="I cannot provide this response due to safety policies.",
)

Enable trace mode during development so you can see exactly which policy fired and why. Azure-specific threat mappings, the OWASP LLM Top 10 controls, and the Agent Control Specification are covered in Microsoft Security & Guardrails.

What Guardrails Do Not Solve

Be honest with stakeholders about the limits, because overselling guardrails is itself a risk.

• They reduce risk, they do not eliminate it. Every classifier has a false negative rate. Novel jailbreaks appear faster than filters are updated. Guardrails buy you a much smaller attack surface, not a closed one.
• They cost latency and money. Each check is an extra call. Input filter, output filter, PII scan, and grounding check can add hundreds of milliseconds and 10-20% on top of inference spend. Budget for it and measure it.
• False positives erode trust faster than false negatives. An over-aggressive filter that blocks legitimate clinical, legal, or security questions trains users to route around the system. Tune thresholds per category against real traffic, not defaults.
• They do not make output correct. A grounding check confirms an answer is supported by the retrieved context. It cannot tell you the context was right, current, or complete.
• They do not replace human review. Where the cost of an error is high - clinical, legal, financial, or anything irreversible - a human approves before the action commits. Guardrails decide what reaches the reviewer; they do not decide instead of them.
• They need adversarial testing. Red-team the system on a schedule, including indirect injection through your own knowledge base. A guardrail configuration that has never been attacked is an assumption, not a control.

Key Takeaways

• Guardrails are enforcement outside the model. A system prompt asks; a guardrail blocks. Anything the model can be talked out of is not a guardrail.
• Treat retrieved content as untrusted input. Indirect prompt injection through the knowledge base is the failure mode most teams have not tested.
• Cover all four enforcement points: input classification, tool authorisation, output filtering, and audit logging. Each catches something the others miss.
• For agents, constrain what the system can do, not just what it can say. Deny by default, scope permissions to the calling user, and tier approvals by reversibility.
• Use the platform's managed guardrails rather than building filters yourself, then tune thresholds against real traffic and keep human review wherever the cost of an error is high.