The Microsoft Stack
Start Here
Overview and approach
Foundry
The production AI platform
Building Agents
Pro-code and low-code
Data and Retrieval
What grounds the agents
Operating in Production
Running it safely

Foundry Tools

Foundry Tools are pre-built AI APIs for documents, speech, vision, language, and translation, which an agent can call instead of you building those capabilities yourself. They were previously Azure AI Services, and are now part of the Foundry platform and available to agents as tools.

Service Catalogue

ToolCapabilitiesUse For
Document IntelligenceOCR, layout detection, table extraction, key-value pairs, custom modelsInvoices, contracts, forms, receipts
SpeechSpeech-to-text, text-to-speech, real-time transcription, voice cloning (MAI-Voice-2)Voice agents, transcription, accessibility
VisionImage analysis, OCR, spatial analysis, face detectionImage classification, document scanning
LanguageNER, summarisation, PII detection, sentiment, key phrase extractionText analytics, compliance, content moderation
Content UnderstandingMultimodal analysis (video, audio, documents)Media processing, content indexing
TranslatorReal-time translation, 100+ languages, document translationMultilingual apps, content localisation

Document Intelligence

Extract structured data from unstructured documents at scale.

FeatureDescription
Prebuilt modelsInvoice, receipt, ID, tax form, health insurance
LayoutTables, paragraphs, headers, page structure
Custom modelsTrain on your document types
Add-on featuresBarcode, formula, font extraction
from azure.ai.documentintelligence import DocumentIntelligenceClient
from azure.identity import DefaultAzureCredential

client = DocumentIntelligenceClient(
    endpoint="https://my-foundry.cognitiveservices.azure.com/",
    credential=DefaultAzureCredential()
)

result = client.begin_analyze_document(
    "prebuilt-invoice",
    document_url="https://storage.blob.core.windows.net/invoices/sample.pdf"
).result()

for invoice in result.documents:
    print(f"Vendor: {invoice.fields['VendorName'].content}")
    print(f"Total: {invoice.fields['InvoiceTotal'].content}")

Document Intelligence docs

Speech

FeatureStatusDescription
Speech-to-textGAReal-time and batch transcription
Text-to-speechGANeural voices, custom voice
MAI-Transcribe-2PreviewMicrosoft's speech-to-text model with content biasing
MAI-Voice-2PreviewMultilingual TTS with voice cloning
Voice LiveGAEnd-to-end voice agent (integrated with Agent Service)
MultichannelGA (July 2026)Stereo input audio processing

Speech Service docs

Vision

FeatureDescription
Image analysisCaptions, tags, objects, people
OCRText extraction from images
Spatial analysisPeople counting, zone monitoring
FaceDetection, verification, grouping

Vision docs

Language

FeatureDescription
Named Entity RecognitionPeople, places, organisations, dates
PII DetectionPersonal data identification and redaction
SummarisationExtractive and abstractive
Sentiment AnalysisDocument and sentence level
Key Phrase ExtractionImportant terms and concepts
Text Analytics for HealthMedical entity extraction

New in July 2026: Foundry playgrounds for language detection, text PII detection, conversation PII detection, and Text Analytics for Health.

Language Service docs

Content Understanding

Multimodal content analysis - process video, audio, and documents with a single API.

InputOutput
VideoScene detection, transcription, visual descriptions
AudioTranscription, speaker identification, content classification
DocumentsLayout, entities, classifications

Content Understanding docs

Translator

FeatureDescription
Text translation100+ languages, real-time
Document translationPreserve formatting
Custom translatorDomain-specific terminology
TransliterationScript conversion

Translator docs

Integration with Agents

All Foundry Tools are available as agent tools via Toolboxes:

agent = client.agents.create(
    model="gpt-5",
    name="document-processor",
    tools=[
        {"type": "document_intelligence"},
        {"type": "language", "capabilities": ["pii_detection", "summarization"]}
    ]
)

Key Takeaways

  • Check for a Foundry Tool before building a capability: document extraction, speech, vision, language, and translation are already solved.
  • Document Intelligence covers invoices, contracts, forms, and receipts, with custom models for your own document types.
  • Every tool is available to agents through Toolboxes, so extracting a document or redacting PII becomes a single tool call.
  • MAI-Transcribe-2 and MAI-Voice-2 are in preview. Speech-to-text, text-to-speech, and Voice Live are GA.

Next: Microsoft Agent Framework - Foundry hosts agents; Agent Framework is how you write the ones that need custom logic.