AI Play Book

Foundry Tools

Pre-built AI APIs that give agents the ability to act on data. Previously "Azure AI Services", now rebranded as Foundry Tools and integrated into the Foundry platform.

Service Catalogue

Tool	Capabilities	Use For
Document Intelligence	OCR, layout detection, table extraction, key-value pairs, custom models	Invoices, contracts, forms, receipts
Speech	Speech-to-text, text-to-speech, real-time transcription, voice cloning (MAI-Voice-2)	Voice agents, transcription, accessibility
Vision	Image analysis, OCR, spatial analysis, face detection	Image classification, document scanning
Language	NER, summarisation, PII detection, sentiment, key phrase extraction	Text analytics, compliance, content moderation
Content Understanding	Multimodal analysis (video, audio, documents)	Media processing, content indexing
Translator	Real-time translation, 100+ languages, document translation	Multilingual apps, content localisation

Document Intelligence

Extract structured data from unstructured documents at scale.

Feature	Description
Prebuilt models	Invoice, receipt, ID, tax form, health insurance
Layout	Tables, paragraphs, headers, page structure
Custom models	Train on your document types
Add-on features	Barcode, formula, font extraction

from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.identity import DefaultAzureCredential

client = DocumentIntelligenceClient(
    endpoint="https://my-foundry.cognitiveservices.azure.com/",
    credential=DefaultAzureCredential()
)

result = client.begin_analyze_document(
    "prebuilt-invoice",
    document_url="https://storage.blob.core.windows.net/invoices/sample.pdf"
).result()

for invoice in result.documents:
    print(f"Vendor: {invoice.fields['VendorName'].content}")
    print(f"Total: {invoice.fields['InvoiceTotal'].content}")

Document Intelligence docs

Speech

Feature	Status	Description
Speech-to-text	GA	Real-time and batch transcription
Text-to-speech	GA	Neural voices, custom voice
MAI-Transcribe-2	Preview	Microsoft's speech-to-text model with content biasing
MAI-Voice-2	Preview	Multilingual TTS with voice cloning
Voice Live	GA	End-to-end voice agent (integrated with Agent Service)
Multichannel	GA (July 2026)	Stereo input audio processing

Speech Service docs

Vision

Feature	Description
Image analysis	Captions, tags, objects, people
OCR	Text extraction from images
Spatial analysis	People counting, zone monitoring
Face	Detection, verification, grouping

Vision docs

Language

Feature	Description
Named Entity Recognition	People, places, organisations, dates
PII Detection	Personal data identification and redaction
Summarisation	Extractive and abstractive
Sentiment Analysis	Document and sentence level
Key Phrase Extraction	Important terms and concepts
Text Analytics for Health	Medical entity extraction

New in July 2026: Foundry playgrounds for language detection, text PII detection, conversation PII detection, and Text Analytics for Health.

Language Service docs

Content Understanding

Multimodal content analysis - process video, audio, and documents with a single API.

Input	Output
Video	Scene detection, transcription, visual descriptions
Audio	Transcription, speaker identification, content classification
Documents	Layout, entities, classifications

Content Understanding docs

Translator

Feature	Description
Text translation	100+ languages, real-time
Document translation	Preserve formatting
Custom translator	Domain-specific terminology
Transliteration	Script conversion

Translator docs

Integration with Agents

All Foundry Tools are available as agent tools via Toolboxes:

agent = client.agents.create(
    model="gpt-5",
    name="document-processor",
    tools=[
        {"type": "document_intelligence"},
        {"type": "language", "capabilities": ["pii_detection", "summarization"]}
    ]
)