The Microsoft Stack
Start Here
Overview and approach
Foundry
The production AI platform
Building Agents
Pro-code and low-code
Data and Retrieval
What grounds the agents
Operating in Production
Running it safely

Security & Guardrails

LLM applications introduce attack surfaces that traditional application security does not cover. Prompt injection, data leakage, and excessive agency are real threats in production. This article maps the OWASP LLM Top 10 to Azure-native controls, with MITRE ATLAS as the companion catalogue of adversary techniques.

Threat Landscape

Threats
Prompt Injection
Targets user input and data
Data Leakage
Targets the model and data
Excessive Agency
Targets app logic
Data Poisoning
Targets the data layer
Insecure Output
Targets app logic
Application Layers
User Input
App Logic
LLM
Data Layer

Top Threats and Mitigations

Prompt Injection

What it is

Attacker crafts input that overrides system prompt instructions.

  • Direct: User explicitly tries to override behavior ("Ignore previous instructions...")
  • Indirect: Malicious content embedded in retrieved documents

Azure Mitigations

from azure.ai.contentsafety import ContentSafetyClient

client = ContentSafetyClient(endpoint=endpoint, credential=credential)

result = client.analyze_text(
    text=user_input,
    categories=["PromptShield"],
    output_type="FourSeverityLevels"
)

if result.prompt_shield_result.attack_detected:
    return "Request blocked: potential prompt injection detected."

Defense layers:

  • Azure Content Safety prompt shields
  • System prompt design with explicit constraints
  • Segregate user input from system instructions
  • Validate LLM output before downstream use

Data Leakage

What it is

LLM reveals PII, credentials, or proprietary data from training data, fine-tuning data, or retrieved context.

Azure Mitigations

ControlImplementation
PII redactionPresidio or Content Safety before sending to LLM
Document-level securitySecurity filters in AI Search queries
Output filteringScan responses for PII patterns
Least-privilege RAGOnly retrieve docs the user has access to
results = search_client.search(
    search_text=query,
    filter=f"allowed_groups/any(g: g eq '{user_group}')",
    top=5
)

Excessive Agency

What it is

LLM-connected tools have more permissions than necessary. Model calls destructive APIs or accesses data outside scope.

Azure Mitigations

  • Limit tool permissions to minimum required
  • Human-in-the-loop for high-impact actions
  • Set iteration limits on agent loops
  • Azure RBAC to scope tool access per agent identity
def execute_tool(tool_name, params, risk_level):
    if risk_level == "high":
        approval = request_human_approval(tool_name, params)
        if not approval:
            return {"status": "blocked", "reason": "Approval denied"}
    return call_tool(tool_name, params)

Data Poisoning

What it is

Attacker manipulates training data, fine-tuning data, or RAG knowledge base to inject biased or malicious content.

Azure Mitigations

  • Validate data sources before indexing (Defender for Cloud malware scan)
  • Track data lineage with ML-BOM
  • Access controls on AI Search indexers and data sources
  • Content moderation on ingestion pipeline
  • Monitor for anomalous content in indexed documents

Insecure Output

What it is

Application trusts LLM output without validation and passes it to downstream systems (SQL, APIs, UI).

Azure Mitigations

import html

def safe_render(llm_response: str) -> str:
    return html.escape(llm_response)

def safe_sql_from_llm(llm_sql: str, allowed_tables: list[str]) -> str:
    for table in extract_tables(llm_sql):
        if table not in allowed_tables:
            raise ValueError(f"Unauthorized table: {table}")
    return llm_sql
  • Never execute LLM-generated code without sandboxing
  • Validate output against expected schema
  • Use parameterized queries even when LLM generates SQL

Security Architecture

Request Path
Every layer applies defence
User
Authenticated request
APIM
Authentication, rate limits
Input Safety
Content Safety prompt shields
Azure OpenAI
With the system prompt
Response Path
Checked twice before it reaches the user
Output Safety
Content Safety output filter
App Validation
Schema and business checks
User
Validated output

Security Checklist

Authentication and Network

  • Managed identity for all service-to-service auth (no API keys in code)
  • Private endpoints for Azure OpenAI and AI Search
  • RBAC with least-privilege roles

Input Protection

  • Content filtering enabled on all deployments
  • Prompt shields enabled for user-facing applications
  • Custom blocklists for domain-specific terms

Data Protection

  • Document-level security filters in search queries
  • PII redaction on ingestion and output
  • Data source validation and malware scanning

Output Protection

  • Schema validation before downstream execution
  • HTML encoding before rendering
  • Human-in-the-loop for destructive tool actions

Monitoring

  • Audit logging (Azure Monitor, Defender for Cloud)
  • Regular red-team testing against prompt injection
  • Anomaly detection on token usage patterns

Agent-Specific Security

Agent Control Specification (ACS)

Open industry standard for deterministic runtime controls at five agent checkpoints: input, LLM, state, tool execution, and output. Expressed as portable YAML - versionable, auditable, framework-agnostic.

Partners: Infosys, KPMG, IBM, Aviatrix, BigSpin, CrewAI.

Agent Control Specification | Agent Governance Toolkit

ASSERT

Open-source framework that converts written policies into executable agent evaluations. Generates targeted test scenarios and surfaces safety defects before production.

ASSERT on GitHub

Guided Guardrail Setup

Questionnaire in Foundry Agent Builder (public preview). Answers about audience, data access, and use case surface recommended controls (PII filters, jailbreak protection, task adherence) with one-click application.

AI Red Teaming Agent

Automated adversarial testing (preview). Probes your agent for prompt injection, jailbreak, and data exfiltration vulnerabilities.

See also: Observability & Evaluation for the full evaluation and trust toolchain.


Key Takeaways

  • LLM applications add attack surfaces traditional security does not cover: prompt injection, data leakage, excessive agency, data poisoning, and insecure output.
  • Retrieved documents are an attack vector. Indirect prompt injection arrives through RAG, not through the user.
  • Give tools least privilege, set iteration limits on agent loops, and put a human in the loop for destructive actions.
  • Treat model output as untrusted input. Validate it against a schema and use parameterised queries before anything downstream runs it.
  • Use managed identity and private endpoints throughout. No API keys in code.

Next: Databricks - the same stack on a lakehouse, where AI sits inside the data platform rather than beside it.