Pipecat - Open Source Voice AI Framework

TL;DR

TL;DR: Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents, with native support for 50+ speech, language, and transport services.

Source and Accuracy Notes

All links verified from primary source (README.md) on 2026-07-15.

• Project page: pipecat.ai
• Source repository: github.com/pipecat-ai/pipecat
• License: BSD-2-Clause (verified via GitHub API license.spdx_id)
• Latest release: v1.5.0 (2026-07-04)
• HN launch thread: news.ycombinator.com/item?id=40345696
• Docs: docs.pipecat.ai

What Is Pipecat?

Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents. It comes from the team at Daily.co, the video/WebRTC infrastructure company, and has accumulated 13,000+ stars on GitHub.

The framework handles the full pipeline: speech-to-text, language model inference, text-to-speech, and transport (WebSockets, WebRTC via Daily.co, LiveKit, and more). You compose these as a pipeline of processors that pass data through stages.

Key capabilities from the README:

• Voice Assistants — natural, streaming conversations with AI
• Multi-Agent Systems — specialists that hand off, fan out in parallel, or run as sidecars over a shared message bus
• AI Companions — coaches, meeting assistants, characters
• Multimodal Interfaces — voice, video, images, and more
• Interactive Storytelling — creative tools with generative media
• Business Agents — customer intake, support bots, guided flows
• Complex Dialog Systems — design logic with structured conversations

Setup Workflow

Step 1: Install uv

Pipecat requires uv, the fast Python package manager from Astral:

curl -LsSf https://astral.sh/uv/install.sh | sh
```bash

### Step 2: Install the Pipecat CLI

```bash
uv tool install "pipecat-ai[cli]"
```bash

The CLI gives you the `pipecat` command for scaffolding, running, and deploying agents.

### Step 3: Scaffold a New Project

```bash
pipecat init
```bash

This interactively creates a new phone or web/mobile bot project. The CLI can also integrate with an AI coding assistant (Claude Code, Codex) to build the bot automatically.

### Step 4: Run the Quickstart

```bash
pipecat init quickstart
```bash

Or follow the [quickstart guide](https://docs.pipecat.ai/getting-started/quickstart) in the docs. A basic voice agent looks like this:

```python
from pipecat import PipecatPipeline, Airbreak

# Define your pipeline
pipeline = PipecatPipeline(
    stt="deepgram",
    llm="openai",
    tts="elevenlabs",
    transport="daily"
)

# Run it
await pipeline.run()
```bash

### Step 5: Connect to Transport

Pipecat supports multiple transports for getting audio in and out:

```python
# Daily.co WebRTC (recommended for production)
transport = DailyTransport()

# Or WebSocket server for custom clients
from pipecat.transports.websocket_server import WebSocketServerTransport
transport = WebSocketServerTransport()

Service Ecosystem

Pipecat ships with native integrations for 50+ services across every layer of the voice pipeline. This table covers the major categories:

Category	Services
Speech-to-Text	Deepgram, AssemblyAI, Whisper, ElevenLabs, Google, Azure, AWS, Groq, Mistral, NVIDIA, OpenAI
LLMs	OpenAI, Anthropic, Gemini, Groq, Mistral, Ollama, DeepSeek, AWS, Azure, Together AI, OpenRouter
Text-to-Speech	ElevenLabs, OpenAI, Azure, Google, AWS, Cartesia, Kokoro, LMNT, MiniMax, XTTS
Speech-to-Speech	OpenAI Realtime, Gemini Multimodal Live, AWS Nova Sonic, Ultravox
Transport	Daily.co, LiveKit, FastAPI WebSocket, Vonage, WhatsApp, WebSocket Server
Video	HeyGen, Tavus, Simli, LemonSlice

Client SDKs

Pipecat provides official client SDKs for connecting from any platform:

• JavaScript / React — docs.pipecat.ai/client/js/introduction
• React Native — docs.pipecat.ai/client/react-native/introduction
• iOS (Swift) — docs.pipecat.ai/client/ios/introduction
• Android (Kotlin) — docs.pipecat.ai/client/android/introduction
• C++ — docs.pipecat.ai/client/c++/introduction
• ESP32 — github.com/pipecat-ai/pipecat-esp32

Developer Tools in the Ecosystem

Pipecat ships several companion tools:

• Whisker — real-time debugger for Pipecat pipelines (github.com/pipecat-ai/whisker)
• Voice UI Kit — React components and templates for building voice AI frontends (github.com/pipecat-ai/voice-ui-kit)
• Pipecat Flows — structured conversation paths with state management (built-in, see docs.pipecat.ai/guides/features/pipecat-flows)
• Pipecat CLI — scaffold, monitor, and deploy agents to production

Practical Evaluation Checklist

• Installed via uv tool install "pipecat-ai[cli]" — verified from README
• Quickstart runs without errors on Python 3.10+
• STT + LLM + TTS pipeline connects and produces audio output
• Multi-agent handoff between two pipeline instances works
• Daily.co transport connects for WebRTC audio
• Claude Code plugin installs via claude plugin marketplace add pipecat-ai/skills

Security Notes

• Pipecat processes audio locally; no audio is stored by the framework by default
• API keys for STT/LLM/TTS services must be provided via environment variables
• The framework does not handle authentication — that is the developer’s responsibility in the transport layer
• For production deployments, use TLS on WebSocket transports and restrict access to the Daily.co or LiveKit room

FAQ

Q: Does Pipecat work without Daily.co? A: Yes. Daily.co is the default WebRTC transport, but Pipecat also supports LiveKit, FastAPI WebSockets, Vonage, WhatsApp, and a bare WebSocket server. You can run entirely without Daily.co.

Q: What Python version is required? A: Python 3.10 or later is recommended. The framework uses async/await extensively.

Q: Can I use local models with Pipecat? A: Yes. Ollama is a supported LLM service, and Whisper is supported for local speech-to-text. Both can run on the same machine as the Pipecat agent.

Q: How does multi-agent coordination work? A: Each Pipecat pipeline is an agent. You compose multi-agent systems by having agents hand off to each other, fan out in parallel, or use sidecar workers over a shared message bus. This is described in the README under “Multi-Agent Ready.”

Q: Is there a managed cloud offering? A: Pipecat Cloud is available for production hosting. The open-source framework can also be self-hosted on any server with Python 3.10+.

Conclusion

Pipecat stands out as the most comprehensive open-source voice AI framework in the Python ecosystem. With 13,000+ GitHub stars, 50+ service integrations, and a full suite of client SDKs spanning web, mobile, and embedded, it covers the entire pipeline from microphone input to AI response to audio output. The multi-agent architecture and structured conversation flows make it scalable beyond a single bot. Whether you are building a personal voice assistant, a customer support agent, or a distributed multi-AI system, Pipecat provides the building blocks.

The BSD-2-Clause license means you can use it freely in commercial projects. Start with pipecat init quickstart and have a running voice agent in minutes.