LiveKit Realtime Playground - Speech-to-Speech in Browser

TL;DR

TL;DR: The LiveKit Realtime Playground is a zero-install browser tool for experimenting with OpenAI’s voice Realtime API, powered by the open-source LiveKit Agents framework.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: playground.livekit.io
• LiveKit Agents source: github.com/livekit/agents
• LiveKit Agents PyPI: pypi.org/project/livekit-agents
• License: Apache-2.0 (verified via GitHub API license.spdx_id)
• HN launch thread: news.ycombinator.com/item?id=44183166 (Show HN, 4 pts)
• Stars: 11,100 (livekit/agents), 340 (livekit-examples/realtime-playground) — verified via GitHub API
• Source last checked: 2026-06-27

What Is the Realtime Playground?

The LiveKit Realtime Playground is a browser-based interface for experimenting with OpenAI’s Realtime API in speech-to-speech mode. You open a URL, grant microphone access, and start talking to GPT-4o — no pip install, no server process, no WebRTC boilerplate.

The playground is built on LiveKit Agents, an open-source Python framework for building realtime voice AI applications. LiveKit Agents handles the agent session, STT/TTS pipeline, and WebRTC transport; the playground provides a readymade client UI so you can skip the client-side scaffolding entirely.

What the Playground Actually Demonstrates

From the livekit-examples/realtime-playground repository and the livekit.io playground UI, the tool showcases:

• Voice-in, voice-out — speak to GPT-4o and hear its response, with no text intermediary
• Interruptible responses — LiveKit Agents uses a transformer-based semantic turn detector to know when the user is done speaking, so you can cut off the agent mid-response
• Browser-native — no app to download; works in any modern browser with WebRTC support
• OpenAI Realtime API — uses OpenAI’s native speech-to-speech models under the hood, with LiveKit Agents managing the connection lifecycle

The setup shown in the repository’s client-side code uses the LiveKit Agents JS SDK to connect to the Realtime API session.

Setup: How to Try It

There is no install step. Point your browser at playground.livekit.io and click Get started. Grant microphone access when prompted.

To build your own voice agent with the same underlying stack, LiveKit Agents requires Python 3.9+ and a LiveKit server (self-hosted or cloud):

pip install "livekit-agents[openai,deepgram,cartesia]"

The [openai,deepgram,cartesia] extras pull in the plugins for OpenAI’s Realtime API plus default STT (Deepgram) and TTS (Cartesia) choices. You can also swap in your own providers.

Under the Hood: How LiveKit Agents Works

LiveKit Agents is organized around four core concepts from the README:

• Agent — an LLM-based application with defined instructions
• AgentSession — manages interactions between an agent and end users
• entrypoint — the starting handler for an interactive session, similar to a web request handler
• AgentServer — the main process that coordinates job scheduling and launches agents

A minimal voice agent, as shown in the README, looks like this:

from livekit.agents import Agent, AgentSession, cli

session = AgentSession()

@session.on("user_input")
async def handle_input(ctx):
    await ctx.generate_response()

await session.add_agent(Agent(instructions="You are a helpful voice assistant."))
await session.start()

The framework handles turn detection, audio streaming, and the Realtime API connection lifecycle. You supply the agent instructions and the model providers.

Practical Evaluation Checklist

• Browser-based voice: open URL, talk immediately — no install required
• Interruptible responses: transformer-based turn detection pauses agent output cleanly
• Open-source stack: LiveKit Agents (Apache-2.0) runs on your own infrastructure
• Modular: swap STT, TTS, or LLM providers independently
• MCP support: LiveKit Agents has native MCP integration for tool use

Security Notes

• Voice data passes through OpenAI’s servers when using the Realtime API — review OpenAI’s data privacy policy for production use
• LiveKit Agents can be self-hosted on your own LiveKit server, keeping media off third-party infrastructure
• The playground is a demo tool; do not use it to discuss sensitive information in production

FAQ

Q: Is this the same as the OpenAI Realtime API playground? A: No. OpenAI released a web-based console for the Realtime API, but the LiveKit Realtime Playground specifically demonstrates voice-to-voice interactions using LiveKit Agents as the runtime framework. It also showcases LiveKit’s turn detection and interrupt handling, which are specific to the Agents framework.

Q: Do I need a LiveKit server to use the playground? A: No. The playground at playground.livekit.io runs on LiveKit’s infrastructure. To self-host a voice agent built on LiveKit Agents, you would deploy your own LiveKit server.

Q: What models does LiveKit Agents support? A: LiveKit Agents is model-agnostic and integrates with OpenAI, Anthropic, Google Gemini, and other providers via a plugin system. The playground specifically uses OpenAI’s Realtime API.

Q: Is this open source? A: Yes. The livekit/agents repository is Apache-2.0 licensed and the full stack — media server, agent framework, and client SDKs — can be self-hosted.

Conclusion

The LiveKit Realtime Playground is the fastest way to feel what voice-first GPT-4o is like in practice. It strips away the client boilerplate that normally makes realtime voice experiments time-consuming, giving you a direct conversation with the model in under a minute. Behind it sits the Apache-2.0 LiveKit Agents framework, which is worth exploring in depth if you want to ship voice AI products on your own infrastructure.