The Microsoft Stack
Start Here
Overview and approach
Foundry
The production AI platform
Building Agents
Pro-code and low-code
Data and Retrieval
What grounds the agents
Operating in Production
Running it safely

Foundry Models

Foundry Models is the catalogue and deployment layer for every model Foundry serves: more than 1,900 models from OpenAI, Anthropic, Meta, Microsoft, and partners behind a single Azure endpoint. The reason to go through it rather than to each vendor is what comes with the endpoint - enterprise SLA, RBAC, content filtering, and data residency.

Your prompts and completions are never used to train models.

Model Catalogue

As of July 2026.

ProviderModelsBest For
OpenAIGPT-5.5, GPT-5, GPT-4o, o1, o3-mini, DALL-E 3, WhisperGeneral purpose, reasoning, multimodal
AnthropicClaude Opus 5, Claude Sonnet 4Complex reasoning, long context
MetaLlama 4, Llama 3.3Open-weight, self-hosted flexibility
MistralMistral Large, Mistral SmallEuropean data residency
MicrosoftMAI-Thinking-1, MAI-Image-2.5, MAI-Transcribe-2, MAI-Voice-2, Phi-4First-party, cost-efficient, on-device
DeepSeekDeepSeek-R1Reasoning, code
Fireworks AIOpen-model inferenceLow latency, high throughput, custom weights
Embeddingstext-embedding-3-large, text-embedding-3-smallRAG, semantic search

Deployment Types

TypeBillingLatencyBest For
Pay-As-You-Go (PAYG)Per 1K tokens (input + output)VariableDevelopment, testing, variable workloads
Provisioned Throughput (PTU)Fixed monthly per PTUConsistentProduction with predictable traffic
GlobalPAYG pricing, cheapest regionHigherBatch processing, latency-insensitive
Managed ComputePer compute hourVariableRegional GPU constraints, custom model hosting
Data Zone (EU/APAC)PAYG with regional guaranteeRegionalData residency requirements

PTU is a fixed cost - unused capacity is wasted money. Size to handle P80 traffic and overflow to PAYG for spikes. EU Data Zone and APAC Data Zone pricing effective September 2026.

Getting Started

from openai import AzureOpenAI

client = AzureOpenAI(
    azure_endpoint="https://my-foundry.openai.azure.com/",
    api_key="your-key",
    api_version="2024-12-01-preview"
)

response = client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain RAG in 3 sentences."}
    ]
)

Production (Managed Identity)

from azure.identity import DefaultAzureCredential
from openai import AzureOpenAI

credential = DefaultAzureCredential()
token = credential.get_token("https://cognitiveservices.azure.com/.default")

client = AzureOpenAI(
    azure_endpoint="https://my-foundry.openai.azure.com/",
    azure_ad_token=token.token,
    api_version="2024-12-01-preview"
)

Fine-tuning

Fine-tuning customizes a base model on domain-specific data. Supports GPT-4o, GPT-4o-mini, and open models.

ApproachUse When
Prompt EngineeringQuick iteration, general tasks, no training data
RAGReference specific documents, data changes frequently, need citations
Fine-tuningDomain vocabulary, consistent output format, shorter prompts
Both (RAG + Fine-tune)Domain writing style AND current document grounding

Frontier Tuning

More than 10x more cost-efficient than GPT-5.5 for domain-specific tasks. Available via the Foundry portal with no hosting fees during experimentation (developer tier).

Cost Optimization

  1. Use max_tokens to cap response length
  2. Semantic caching with AI Search for frequent queries
  3. Smaller models (Phi-4, GPT-4o-mini) for classification/extraction
  4. Route batch workloads to Global deployments
  5. Monitor token usage per deployment with Azure Monitor
  6. Use Managed Compute when you need to route around regional GPU constraints

Multi-Region Architecture

Applications
App A
App B
Gateway
API Management
The GenAI API gateway
Deployments
Chosen by priority and residency
East US
PTU, priority 1
West Europe
PTU, priority 1
East US 2
PAYG, overflow only
EU Data Zone
PAYG, EU traffic only

Key Takeaways

  • One endpoint gives you 1,900+ models with SLA, RBAC, content filtering, and data residency, and your prompts are never used for training.
  • Use PAYG for development and variable load, and PTU for predictable production traffic. Size PTU to P80 and overflow to PAYG for spikes.
  • Send latency-insensitive batch work to Global deployments, and use Data Zone deployments when residency is a requirement.
  • Try prompting, then RAG, then fine-tuning. Fine-tuning earns its place for domain vocabulary and consistent output format.
  • Smaller models handle classification and extraction. Capping max_tokens and semantic caching are the cheapest cost levers.

Next: Foundry Agent Service - a deployed model answers one call; Agent Service is what runs it as an agent.