The 2 a.m. Question Nobody Wants to Answer Twice

It is the night before open enrollment closes. An employee in another time zone has a question about HSA eligibility. No HR representative is available, and the FAQ PDF she received three months ago is nowhere to be found. She does not want to submit a ticket and wait two days—she needs an answer now, delivered by an experience that feels familiar, trustworthy, and connected to her organization.

The Executive AI Avatar Platform was built to close this gap. It provides a branded, always-available AI avatar that can engage through voice or text and respond using the organization’s approved tone, policies, and knowledge. It is more than a chatbot added to a help page. It combines a face, a voice, and an AI-powered conversational experience that can support an employee at 2 a.m. or engage an audience of 10,000 people during a live town hall—without requiring a team to be available around the clock.

That is the promise. The rest of this post explains how the platform is built and why the architecture does not rely on a single frontier model for every task. In an enterprise deployment, cost, control, performance, and flexibility matter just as much as model capability.

1. What Is the Executive AI Avatar Platform?

The Executive AI Avatar Platform is a full-stack conversational AI platform that enables organizations to create branded, conversational digital humans. These photorealistic or custom-trained avatars can represent executives, HR personas, or subject-matter experts and allow employees or customers to interact naturally through text or voice.

The platform uses a Next.js 15 frontend connected to a FastAPI backend, which orchestrates two primary Microsoft capabilities:

Azure Speech in Foundry Tools (Avatar) renders the avatar’s photorealistic, lip-synchronized video and speech in both batch-based scripted experiences and real-time conversational experiences.

Microsoft Foundry provides the deployment platform for the language models that generate the avatar’s responses. The architecture can use a frontier model such as GPT-4o or an open model such as MiniMax M2.5, deployed through Microsoft Foundry and served by Fireworks AI.

┌─────────────────────────────────────────────────────────┐
│  Next.js 15 Frontend   — Library · Studio · Interact     │
└───────────────────┬───────────────────────────────────────┘
                     │  REST / WebSocket
┌───────────────────▼───────────────────────────────────────┐
│  FastAPI Backend — /avatars  /synthesis  /speech  /voice-live │
└───────────────────┬───────────────────────────────────────┘
                     │
        ┌────────────┴─────────────┐
        │                          │
  Azure Speech in Foundry Tools             Microsoft Foundry
  (TTS Avatar — batch           (GPT-4o realtime,
   & real-time rendering) MiniMax M2.5 via Fireworks AI, deployed through Microsoft Foundry)

An MCP server using Model Context Protocol, an open standard for connecting AI applications to external tools and data sources also wraps the platform API. This allows MCP-compatible clients, including Claude Desktop, GitHub Copilot in Visual Studio Code, and custom orchestration agents, to trigger avatar video generation without requiring each client to manage Azure credentials directly.

2. Use Cases: Live Interaction vs. Scripted Avatar

The platform supports two distinct interaction modes; each designed for different scenarios.

Live Interaction with Voice Live

Live Interaction enables a real-time spoken conversation. The user speaks naturally, and the avatar listens, processes the request, and responds with synchronized audio and video. Built-in turn detection allows the conversation to flow without requiring the user to press and hold a button to speak.

The experience is powered by the Voice Live API in Azure Speech in Foundry Tools and streamed through a persistent WebSocket connection, enabling low-latency, bidirectional communication between the user and the avatar.

Best suited for: live town halls, executive Q&A sessions, customer-support or help-desk kiosks, and other scenarios where responsiveness and a natural conversational experience are more important than extensive customization of the underlying reasoning model.

.

Scripted Avatar

This is a text-first interaction mode. The user enters a message or the system provides a predefined script and a language model generates the response. That text is then sent to Azure Speech in Foundry Tools, where text to speech (TTS) produces the avatar’s voice and lip-synchronized video.

The Web Real-Time Communication (WebRTC) session remains open throughout the interaction, so each new message requires only a lightweight text exchange followed by a speakTextAsync call, without establishing a new connection for every turn.

Best for: onboarding walkthroughs, policy explainers, FAQ bots, and scenarios where you want the flexibility to swap the underlying reasoning model based on cost, latency, or data-residency needs.

Model: MiniMax M2.5 via Fireworks AI, deployed through Microsoft Foundry, generates the text response. The response is then sent to Azure Speech in Foundry Tools to produce the avatar’s voice (TTS) and lip synchronization.

3. Models and Azure Services in the Stack

The key architectural decision is that the reasoning layer is pluggable. The backend exposes a single CHAT_MODEL_PROVIDER setting that routes each scripted-avatar chat completion to one of the supported model providers.

if settings.CHAT_MODEL_PROVIDER == “minimax”:

# Open-model path: MiniMax M2.5 via Fireworks AI, # deployed through Microsoft Foundry

    client = get_minimax_client()
    response = await client.chat.completions.create(
        model=settings.MINIMAX_DEPLOYMENT, messages=messages, …
    )
else:   

# Frontier-model path: GPT-4o via Microsoft Foundry

    client = get_client()
    response = await client.chat.completions.create(
        model=settings.AZURE_OPENAI_DEPLOYMENT, messages=messages, …
    )

Both model providers expose the same OpenAI-compatible chat.completions interface, allowing the rest of the pipeline including persona system prompts, conversation history, and avatar rendering to remain independent of the model generating the response. This interoperability makes the cost-and-performance trade-off described in the next section a simple configuration change rather than an architectural redesign.

4. The Trade-Off: Open Models via Fireworks AI vs. Frontier Models

Why route some workloads to an open model when GPT-4o is available through Microsoft Foundry? In practice, three considerations stand out.

Fast inference

Fireworks AI provides high-throughput, low-latency inference infrastructure for open models. For short, persona-specific responses—such as HR, IT, or policy-related answers—MiniMax M2.5 can provide a strong balance of latency, quality, and cost. Narrowly scoped tasks may not always require the full reasoning capabilities of a larger, general-purpose frontier model.

Fine-tuning and ownership of your own model & Intelligence 

This is the strategic differentiator. Frontier models are typically optimized through prompt engineering, system instructions, retrieval, and supported customization capabilities. With an open model deployed through Microsoft Foundry and served by Fireworks AI, organizations can gain additional flexibility to:

• Fine-tune the model using approved proprietary conversation data, policy content, or brand-specific examples.
• Adapt domain-specific behavior, such as a precise HR-policy tone or specialized customer-support workflow.
• Iterate on model behavior without depending entirely on a frontier model provider’s release cycle.
• Retain greater control over the customized model and its long-term deployment strategy, subject to the model’s license and platform capabilities.

The open-model path is therefore not simply a lower-cost alternative. It can help organizations build a continuously improving intelligence asset tailored to their data, workflows, and brand.

The Other Side of the Trade-Off

Open models also come with trade-offs. Frontier models such as GPT-4o generally remain stronger for:

• Complex, multi-step reasoning and ambiguous instruction following.
• Broad knowledge without additional adaptation or grounding.
• Advanced multimodal experiences, including the real-time audio interaction used by the Voice Live API.

That is why the platform supports both paths. Rather than choosing one model for every workload, the architecture allows each experience to use the model that best meets its requirements for capability, latency, cost, customization, and control.

The trade-off side

Frontier models such as GPT-4o still offer important advantages, particularly for:

• Complex, multi-step reasoning and ambiguous instructions.
• Broad knowledge without requiring additional fine-tuning or domain adaptation.
• Advanced multimodal experiences, including the real-time audio reasoning used by the Voice Live API in Azure Speech in Foundry Tools—an area where many open models have not yet reached the same level of production readiness.

That is precisely why the platform supports both paths instead of selecting a single model for every workload. This allows each use case to be matched with the right balance of capability, cost, latency, customization, and control.

5. Cost Differences at Scale

Token costs can compound quickly when an avatar platform is deployed across an organization, supporting thousands of daily conversations across HR, IT, onboarding, and other employee-service scenarios. The comparison below uses publicly available list pricing from the Microsoft Foundry pricing page and Fireworks AI’s serverless pricing documentation, verified at the time of writing. Always confirm the latest pricing before building a business case, as rates and service tiers may change.

Worked example — 50M input tokens + 10M output tokens in a month (a realistic volume for an org-wide FAQ/onboarding avatar handling tens of thousands of short conversations):

At this volume, routing scripted-avatar text generation through MiniMax M2.5 via Fireworks AI, deployed through Microsoft Foundry, costs approximately 88% less than using Foundry GPT-4o for the text-generation portion of the pipeline. As usage grows to more than 500 million tokens per month across a large enterprise deployment, the difference can become a meaningful operating-budget consideration.

This comparison does not include avatar-rendering costs in Azure Speech in Foundry Tools, including text-to-speech and video synthesis. These costs apply separately, regardless of which language model generates the response.

The comparison also excludes live spoken sessions powered by the Voice Live API in Azure Speech in Foundry Tools using GPT-4o through Microsoft Foundry. Voice Live is priced separately and is not part of this comparison because MiniMax M2.5 supports the scripted text-generation path rather than the real-time audio interaction path.

Get Started

Everything described in this post—including dual-model routing, WebRTC-based avatar rendering, and MCP server integration—is available as open-source code that you can clone and run today.

GitHub repository:

# Clone the repository

git clone https://github.com/ganachan/ai-executive-avatar-platform.git

cd ai-executive-avatar-platform

# Configure environment variables

cp backend/.env.example backend/.env # Add your Azure resource details

cp frontend/.env.example frontend/.env

# Install and run the FastAPI backend

cd backend

pip install -r requirements.txt

uvicorn main:app –reload –port 8000

# In a second terminal, install and run the Next.js frontend

cd frontend

npm install

npm run dev

Prerequisites: 

• Node.js 18 or later
• Python 3.11 or later
• An Azure subscription with Avatar enabled in Azure Speech in Foundry Tools
• A Microsoft Foundry resource with GPT-4o and MiniMax M2.5 deployed
• Azure CLI authentication configured through az login for Microsoft Entra ID authentication; static API keys are not required by default

Switching the reasoning model for your deployment requires only a one-line configuration change in backend/config.py or backend/.env:

# Route scripted-avatar chat through MiniMax M2.5 via Fireworks AI, # deployed through Microsoft Foundry CHAT_MODEL_PROVIDER=minimax # …or use GPT-4o via Microsoft Foundry CHAT_MODEL_PROVIDER=azure

Fork the repository, connect it to your own Azure resources, and customize the avatar personas, system prompts, knowledge sources, and fine-tuned models for your organization.

Summary: Open Models as Workflow Partners, Not Replacements

The key lesson from building this platform is not that open models outperform frontier models. It is that both belong in the same workflow, with each model routed to the tasks it handles best.

Use frontier models such as GPT-4o or GPT-5 through Microsoft Foundry when an interaction requires advanced reasoning, multimodal understanding, or real-time speech-to-speech fluency—such as the natural spoken conversations enabled by the Voice Live API in Azure Speech in Foundry Tools.

Use a fine-tuned open model such as MiniMax M2.5 via Fireworks AI, deployed through Microsoft Foundry, for high-volume, persona-specific, and cost-sensitive text interactions. These include FAQ and policy-related experiences in the Scripted Avatar mode, where organizations may also want greater control over model customization and the option to fine-tune using their approved data, subject to licensing and platform capabilities.

Because both model paths use the same OpenAI-compatible interface and are controlled through a single configuration setting, this is not merely a conceptual hybrid architecture. Switching between:

CHAT_MODEL_PROVIDER=minimax

and:

CHAT_MODEL_PROVIDER=azure

makes it possible to compare cost, latency, and response quality side by side using the same avatar, persona, and application workflow.

As avatar and agent platforms scale across the enterprise, this pattern—frontier models for complex interactions and customized open models for high-volume workloads—is likely to become a common architectural approach rather than an exception.

Built using Azure Speech in Foundry Tools, Microsoft Foundry, and Fireworks AI. Have questions about the architecture or want to explore the implementation? Share your questions in the comments or clone the repository linked above and try it with your own Azure resources.

Thank you, Binaka Sankaran, for volunteering to record the source video used to create the avatar and for helping us scale and accelerate this initiative.