Pipecat - Open Source Voice AI Framework

TL;DR

TL;DR: Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents, with native support for 50+ speech, language, and transport services.

Source and Accuracy Notes

All links verified from primary source (README.md) on 2026-07-15.

• Project page: pipecat.ai
• Source repository: github.com/pipecat-ai/pipecat
• License: BSD-2-Clause (verified via GitHub API license.spdx_id)
• Latest release: v1.5.0 (2026-07-04)
• HN launch thread: news.ycombinator.com/item?id=40345696
• Docs: docs.pipecat.ai

What Is Pipecat?

Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents. It comes from the team at Daily.co, the video/WebRTC infrastructure company, and has accumulated 13,000+ stars on GitHub.

The framework handles the full pipeline: speech-to-text, language model inference, text-to-speech, and transport (WebSockets, WebRTC via Daily.co, LiveKit, and more). You compose these as a pipeline of processors that pass data through stages.

Key capabilities from the README:

• Voice Assistants — natural, streaming conversations with AI
• Multi-Agent Systems — specialists that hand off, fan out in parallel, or run as sidecars over a shared message bus
• AI Companions — coaches, meeting assistants, characters
• Multimodal Interfaces — voice, video, images, and more
• Interactive Storytelling — creative tools with generative media
• Business Agents — customer intake, support bots, guided flows
• Complex Dialog Systems — design logic with structured conversations

Setup Workflow

Step 1: Install uv

Pipecat requires uv, the fast Python package manager from Astral:

curl -LsSf https://astral.sh/uv/install.sh | sh

Step 2: Install the Pipecat CLI

uv tool install "pipecat-ai[cli]"

The CLI gives you the pipecat command for scaffolding, running, and deploying agents.

Step 3: Scaffold a New Project

pipecat init

This interactively creates a new phone or web/mobile bot project. The CLI can also integrate with an AI coding assistant (Claude Code, Codex) to build the bot automatically.

Step 4: Run the Quickstart

pipecat init quickstart

Or follow the quickstart guide in the docs. A basic voice agent looks like this:

from pipecat import PipecatPipeline, Airbreak

# Define your pipeline
pipeline = PipecatPipeline(
    stt="deepgram",
    llm="openai",
    tts="elevenlabs",
    transport="daily"
)

# Run it
await pipeline.run()

Step 5: Connect to Transport

Pipecat supports multiple transports for getting audio in and out:

# Daily.co WebRTC (recommended for production)
transport = DailyTransport()

# Or WebSocket server for custom clients
from pipecat.transports.websocket_server import WebSocketServerTransport
transport = WebSocketServerTransport()

Service Ecosystem

Pipecat ships with native integrations for 50+ services across every layer of the voice pipeline. This table covers the major categories:

| Category | Services | |---|---| | Speech-to-Text | Deepgram, AssemblyAI, Whisper, ElevenLabs, Google, Azure, AWS, Groq, Mistral, NVIDIA, OpenAI | | LLMs | OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama, DeepSeek, AWS, Azure, Together AI, OpenRouter | | Text-to-Speech | ElevenLabs, OpenAI, Azure, Google, AWS, Cartesia, Kokoro, LMNT, MiniMax, XTTS | | Speech-to-Speech | OpenAI Realtime, Gemini Multimodal Live, AWS Nova Sonic, Ultravox | | Transport | Daily.co, LiveKit, FastAPI WebSocket, Vonage, WhatsApp, WebSocket Server | | Video | HeyGen, Tavus, Simli, LemonSlice |

Client SDKs

Pipecat provides official client SDKs for connecting from any platform:

• JavaScript / React — docs.pipecat.ai/client/js/introduction
• React Native — docs.pipecat.ai/client/react-native/introduction
• iOS (Swift) — docs.pipecat.ai/client/ios/introduction
• Android (Kotlin) — docs.pipecat.ai/client/android/introduction
• C++ — docs.pipecat.ai/client/c++/introduction
• ESP32 — github.com/pipecat-ai/pipecat-esp32

Developer Tools in the Ecosystem

Pipecat ships several companion tools:

• Whisker — real-time debugger for Pipecat pipelines (github.com/pipecat-ai/whisker)
• Voice UI Kit — React components and templates for building voice AI frontends (github.com/pipecat-ai/voice-ui-kit)
• Pipecat Flows — structured conversation paths with state management (built-in, see docs.pipecat.ai/guides/features/pipecat-flows)
• Pipecat CLI — scaffold, monitor, and deploy agents to production

Practical Evaluation Checklist

• [ ] Installed via uv tool install "pipecat-ai[cli]" — verified from README
• [ ] Quickstart runs without errors on Python 3.10+
• [ ] STT + LLM + TTS pipeline connects and produces audio output
• [ ] Multi-agent handoff between two pipeline instances works
• [ ] Daily.co transport connects for WebRTC audio
• [ ] Claude Code plugin installs via claude plugin marketplace add pipecat-ai/skills

Security Notes

• Pipecat processes audio locally; no audio is stored by the framework by default
• API keys for STT/LLM/TTS services must be provided via environment variables
• The framework does not handle authentication — that is the developer’s responsibility in the transport layer
• For production deployments, use TLS on WebSocket transports and restrict access to the Daily.co or LiveKit room

FAQ

Q: Does Pipecat work without Daily.co? A: Yes. Daily.co is the default WebRTC transport, but Pipecat also supports LiveKit, FastAPI WebSockets, Vonage, WhatsApp, and a bare WebSocket server. You can run entirely without Daily.co.

Q: What Python version is required? A: Python 3.10 or later is recommended. The framework uses async/await extensively.

Q: Can I use local models with Pipecat? A: Yes. Ollama is a supported LLM service, and Whisper is supported for local speech-to-text. Both can run on the same machine as the Pipecat agent.

Q: How does multi-agent coordination work? A: Each Pipecat pipeline is an agent. You compose multi-agent systems by having agents hand off to each other, fan out in parallel, or use sidecar workers over a shared message bus. This is described in the README under “Multi-Agent Ready.”

Q: Is there a managed cloud offering? A: Pipecat Cloud is available for production hosting. The open-source framework can also be self-hosted on any server with Python 3.10+.

Conclusion

Pipecat stands out as the most comprehensive open-source voice AI framework in the Python ecosystem. With 13,000+ GitHub stars, 50+ service integrations, and a full suite of client SDKs spanning web, mobile, and embedded, it covers the entire pipeline from microphone input to AI response to audio output. The multi-agent architecture and structured conversation flows make it scalable beyond a single bot. Whether you are building a personal voice assistant, a customer support agent, or a distributed multi-AI system, Pipecat provides the building blocks.

The BSD-2-Clause license means you can use it freely in commercial projects. Start with pipecat init quickstart and have a running voice agent in minutes.