ElevenLabs Realtime - Realtime voice agent models with 570ms to first audio

Eleven Realtime is ElevenLabs’ new family of speech-to-speech models for building voice agents that sound natural in conversation.

Problem: Voice agents are usually stitched together from separate parts: speech recognition, a language model, turn-taking logic and a voice. Each hand-off adds delay, and that’s where conversations start to feel stiff, with awkward pauses and talking over each other.

Solution: Eleven Realtime tunes speech recognition, turn-taking, the LLM and the voice end to end. It comes in two models. v1 Mini is built for natural conversation, and v1 Max is built for complex tasks that need precise, deliberate answers. You can use it in ElevenAgents or through a Realtime API.

What makes it different: The API is OpenAI-compatible, so if you already have a voice app built on OpenAI’s Realtime API, you change the endpoint and model name and your existing event handlers keep working. Per ElevenLabs, time to first audio is 620 ms for v1 Mini and 570 ms for v1 Max, across 90+ languages.

Key features:

• v1 Mini for assistants, tutors and interactive characters

• v1 Max for lead qualification, customer support and scheduling

• Pick the model from one dropdown in ElevenAgents, with transcription, turn-taking and voice preconfigured

• Deploy to phone, web widget, WhatsApp and contact center platforms

• Knowledge base, guardrails, scripted scenario testing and production monitoring in ElevenAgents

• Searchable, auditable conversation logs

• Per ElevenLabs, SOC 2, HIPAA and GDPR support, with EU Data Residency and Zero Retention modes

Who it’s for: Developers and teams building voice agents for support, sales, scheduling or education who want natural-sounding conversations without wiring the pieces together themselves.

Try it via Eleven Realtime, the docs and the OpenAI compatibility guide.