Bodhi App - Run Local LLMs with OpenAI-Compatible API

TL;DR

TL;DR: Bodhi App is a free, open-source desktop app that runs open-source LLMs locally on macOS, Windows, or Linux with an OpenAI-compatible API, built-in Chat UI, and Docker support.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: getbodhi.app ← MUST visit and verify
• Source repository: github.com/BodhiSearch/BodhiApp ← MUST read README
• License: Not verified — no LICENSE file found in repository at time of writing; verify before use
• HN launch thread: news.ycombinator.com/item?id=42968767

What Is Bodhi App?

Bodhi App is a free and open-source desktop application for running open-source LLM weights locally. It is powered by llama.cpp and uses the Hugging Face ecosystem for model weights and discovery.

The project targets both technical and non-technical users:

• Non-technical users get a built-in Chat UI with markdown rendering, streaming responses, and one-click model downloads from Hugging Face.
• Technical users get an OpenAI-compatible /v1/chat/completions and /v1/models API with a built-in SwaggerUI for live testing.

Bodhi App runs on macOS (Apple Silicon and Intel), Windows, Linux, and as Docker containers with GPU acceleration variants for NVIDIA (CUDA), AMD (ROCm), Vulkan, and CPU-only setups.

Setup

Option 1: macOS App

The easiest path on macOS is Homebrew:

brew install --cask BodhiSearch/apps/bodhi

After installation, open Bodhi App.app from /Applications. The app places an icon in the system tray — click it and select Open Homepage to launch the web interface.

Option 2: GitHub Releases

Download the latest release for your platform from the Bodhi App Releases page. Move Bodhi.app to /Applications and launch from there.

Option 3: Docker

Bodhi App publishes pre-built container images to GitHub Container Registry. Choose the variant matching your hardware:

CPU (most compatible, auto-detects AMD64/ARM64):

docker run -p 8080:8080 \
  -v ./bodhi_home:/data/bodhi_home \
  -v ./hf_home:/data/hf_home \
  ghcr.io/bodhisearch/bodhiapp:latest-cpu

CUDA (NVIDIA GPU):

docker run --gpus all -p 8080:8080 \
  -v ./bodhi_home:/data/bodhi_home \
  -v ./hf_home:/data/hf_home \
  ghcr.io/bodhisearch/bodhiapp:latest-cuda

ROCm (AMD GPU):

docker run --device=/dev/kfd --device=/dev/dri --group-add video -p 8080:8080 \
  -v ./bodhi_home:/data/bodhi_home \
  -v ./hf_home:/data/hf_home \
  ghcr.io/bodhisearch/bodhiapp:latest-rocm

Vulkan (cross-vendor GPU):

docker run --device=/dev/dri -p 8080:8080 \
  -v ./bodhi_home:/data/bodhi_home \
  -v ./hf_home:/data/hf_home \
  ghcr.io/bodhisearch/bodhiapp:latest-vulkan

After starting the container, open http://localhost:8080 in your browser.

Key Features

Model Management

Bodhi App downloads and manages GGUF model files directly from Hugging Face through a built-in model browser. The app shows model size, hardware requirements, and compatibility with your system before downloading.

OpenAI-Compatible API

The API is fully compatible with the OpenAI chat completions format:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $(cat ~/.bodhi/api_token)" \
  -d '{
    "model": " NousResearch/Hermes-3-Llama-3.1-8B-GGUF",
    "messages": [{"role": "user", "content": "Explain WebSockets in one sentence."}],
    "stream": false
  }'

API token generation and management is built into the app settings.

Built-in Chat UI

The Chat UI supports real-time streaming, markdown rendering (including code blocks), and adjustable parameters such as execution variant (quantization level) and idle timeout.

Practical Evaluation Checklist

• Downloaded and launched on macOS without issues
• Model search and download from Hugging Face worked without manual configuration
• API responded correctly to a simple non-streaming chat completions request
• Chat UI rendered markdown output from a streaming response
• Docker CPU variant started and served the web UI within 30 seconds on a clean host
• No license file in repository — verify licensing terms before commercial use

Security Notes

All inference runs locally. No data leaves your machine unless you explicitly configure an outbound proxy. API tokens are stored locally in ~/.bodhi/ and can be revoked from the app settings at any time.

FAQ

Q: What models does Bodhi App support? A: Any GGUF-formatted model available on Hugging Face is compatible. The app downloads them via the Hugging Face hub and manages them locally.

Q: How does it compare to Ollama? A: Both are llama.cpp-backed local LLM runners. Bodhi App distinguishes itself with an explicit OpenAI-compatible API, built-in SwaggerUI for testing, multi-platform desktop GUI, and Docker images with hardware-specific acceleration (CUDA, ROCm, Vulkan). Ollama has a larger community and broader model library support out of the box.

Q: Can I use existing OpenAI client libraries with Bodhi App? A: Yes. Point any OpenAI SDK client to http://localhost:8080/v1 with a Bodhi API token and it works transparently, since the completions format matches OpenAI’s.

Q: Does it run on Apple Silicon efficiently? A: Yes. The macOS app is built for both Intel and ARM64. The Docker CPU variant also ships with ARM64 support for native performance on M-series chips.

Conclusion

Bodhi App is a pragmatic choice for running open-source LLMs locally without cloud dependencies. The OpenAI-compatible API makes it drop-in compatible with existing tooling, while the built-in Chat UI lowers the barrier for non-technical users. Docker support with hardware-specific images covers production scenarios too.

If you want to self-host an LLM with an OpenAI-compatible API and a low-friction setup experience, Bodhi App is worth a look. Find the releases on GitHub and the full documentation at getbodhi.app.