AgentLens: Chrome DevTools for AI Agents

TL;DR

TL;DR: AgentLens is an open-source observability platform that traces every step of your AI agent’s execution, lets you replay failures step-by-step, and uses AI to find root causes — all self-hosted and free.

What Is AgentLens?

When an AI agent fails in production, traditional logging leaves you blind. You see API calls but not why the agent chose tool A over B, or where its reasoning went wrong.

AgentLens solves this with a full execution trace: every tool call, every LLM response, every span of work — recorded and replayable. It also ships with an AI-powered “Autopsy” feature that analyzes failed traces and suggests fixes.

Setup Workflow

Step 1: Install the SDK

AgentLens supports Python, TypeScript, and .NET. Choose your language:

# Python
pip install agentlens-observe

# TypeScript
npm install agentlens-observe

# .NET
dotnet add package AgentLens.Observe

Step 2: Start the Server

pip install agentlens-observe
docker compose up

The dashboard starts at http://localhost:3000. The server runs on port 8000 by default.

Step 3: Configure Your Agent

import agentlens

agentlens.configure(server_url="http://localhost:8000", api_key="your-key")

@agentlens.trace
def my_agent(query):
    with agentlens.span("search") as s:
        results = search(query)
        s.set_output(str(results))
    return results

Step 4: Trace MCP Calls Automatically

If you use the Model Context Protocol, a one-line patch enables full auto-tracing:

from agentlens.integrations.mcp import patch_mcp
patch_mcp()  # All MCP tools/calls/resources/prompts are traced

Step 5: Integrate with Frameworks

AgentLens ships official integrations for LangChain, CrewAI, AutoGen, LlamaIndex, Google ADK, and Semantic Kernel (stub):

# LangChain example
from agentlens.integrations.langchain import AgentLensCallbackHandler
callback = AgentLensCallbackHandler()

Deeper Analysis

AI Failure Autopsy

On any failed trace, click Autopsy — AgentLens uses your own API key (OpenAI, Anthropic, or Gemini) to identify the root cause and suggest a concrete fix. This is the feature that most separates it from LangSmith or Langfuse.

Replay Sandbox

The Replay Sandbox lets you step through a trace one span at a time. You can edit inputs at any span, save the modified session, and compare original vs. modified runs side-by-side.

LLM-as-Judge Evaluation

Define custom scoring rubrics (numeric 1-5 or pass/fail) and run automated evaluations against stored traces. Useful for regression testing after prompt changes.

Prompt Versioning

AgentLens version-controls your prompt templates. Compare any two versions with a unified diff, and track which version produced which trace result.

Alerting

Set rule-based alerts on cost, latency, and error rate. Alerts fire via webhook when thresholds are exceeded.

Go CLI and VS Code Extension

Beyond the dashboard, AgentLens ships a Go CLI:

agentlens traces list
agentlens traces show <id>
agentlens traces tail
agentlens traces diff <id1> <id2>

The VS Code extension adds a sidebar with trace history, a detail webview, and a status bar.

Practical Evaluation Checklist

• Self-hosted: runs entirely on your own infrastructure
• Multi-SDK: Python, TypeScript, .NET all stable
• MCP tracing: zero-config auto-patch for Model Context Protocol
• AI Autopsy: bring your own API key for root-cause analysis
• Replay Sandbox: time-travel through execution steps
• LLM-as-Judge: automated evaluation with custom rubrics
• Prompt versioning: diff-based version tracking
• Framework integrations: LangChain, CrewAI, AutoGen, LlamaIndex, Google ADK
• Alerting: rule-based with webhook notifications
• Free forever (self-hosted), no seat limit

Security Notes

AgentLens is self-hosted, so all trace data stays on your infrastructure. The AI Autopsy feature requires you to provide your own API key (OpenAI, Anthropic, or Gemini) — the server never stores it. The Go CLI supports stdin piping for sensitive data.

FAQ

Q: How is this different from LangSmith or Langfuse? A: LangSmith is SaaS-only and charges per seat. Langfuse supports self-hosting but has limited features there. AgentLens is fully self-hosted and free, with unique features like AI Autopsy, MCP protocol tracing, and a Replay Sandbox.

Q: Does it work with closed-source agents? A: Yes. The SDK decorator and span API work with any Python/TypeScript/.NET code, not just specific frameworks.

Q: Can I use it without Docker? A: The dashboard uses Docker Compose by default, but the server is a FastAPI app you can run directly with pip install and python -m agentlens.server.

Source and Accuracy Notes

• Project page: agentlens.io
• Source repository: github.com/tranhoangtu-it/agentlens
• License: MIT (verified via LICENSE file on main branch)
• HN launch thread: news.ycombinator.com/item?id=47565649
• Latest release: v1.0.0 (verified via GitHub Releases API, published 2026-03-29)
• Stars: 4 (verified via GitHub REST API)