Latitude – AI Agent Observability Platform

TL;DR

TL;DR: Latitude is an open-source observability platform for AI agents — think Sentry for LLMs. It tracks traces, groups failures into issues, runs alignment evals, and supports self-hosting.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: latitude.so — verified June 29, 2026
• Source repository: github.com/latitude-dev/latitude-llm — README read in full
• License: LGPL-3.0 (verified from LICENSE file in repo)
• HN launch thread: news.ycombinator.com/item?id=not-tracked — not found; tool discovered via HN Algolia query

What Is Latitude?

Latitude describes itself as “Sentry, but for agents and LLMs.” It gives you full visibility into what is failing in production — not just that a call failed, but which traces led to the failure, which tool calls went wrong, and whether your eval scores are drifting over time.

The platform is provider-agnostic. It works out of the box with OpenAI, Anthropic, AWS Bedrock, Vercel AI SDK, and LangChain, plus any OTLP-compatible backend.

Key capabilities from the README:

• Issue-centric tracking — failed traces grouped into tracked issues with status, size, and trend.
• Alignment evals — evals built automatically from team judgments, with an alignment score tracking drift from human judgment.
• Agent-native traces — multi-turn sessions, tool calls, and full execution paths in one view.
• Semantic search — find any trace by meaning, exact match, or rough similarity. No sampling; 100 percent of traces are searchable.

Setup Workflow

Step 1: Install the SDK

npm install @latitude-data/telemetry

Step 2: Instrument your agent

import { Latitude } from "@latitude-data/telemetry";

new Latitude({
  apiKey: process.env.LATITUDE_API_KEY!,
  projectSlug: process.env.LATITUDE_PROJECT_SLUG!,
  instrumentations: ["openai"],
});

Every LLM call now shows up as a trace in Latitude.

Step 3: Self-host (optional)

Self-hosting and OpenTelemetry passthrough are covered in the getting started guide.

Claude Code integration

Building inside Claude Code? Install the dedicated telemetry package:

npx -y @latitude-data/claude-code-telemetry install

Works in the terminal, the Desktop app, and IDE extensions. Full session transcripts are captured as traces.

Deeper Analysis

Strengths:

• Open-source with a permissive LGPL license (plus a commercial option for tight integrations).
• Provider-agnostic by design — not locked into one LLM vendor.
• Issue clustering saves hours of manual log grepping.
• Semantic search across 100 percent of traces is a genuine differentiator versus sampling-based tools.

Limitations:

• Primary SDK is Node.js/TypeScript. Python and Go support exists but may lag behind.
• The platform is still in active development — v2 is not yet stable.
• Self-hosting requires some OTel knowledge to configure correctly.

Self-hosting note: Latitude supports self-hosting via OTel passthrough. If you need data residency or want to avoid sending traces to their cloud, you can run your own OTLP-compatible backend and connect Latitude to it. Details are in the self-hosting docs.

Practical Evaluation Checklist

• [ ] Install @latitude-data/telemetry and make one LLM call — verify trace appears in dashboard
• [ ] Trigger a known failure mode — verify it surfaces as an Issue
• [ ] Run a semantic search for a past trace — verify results are meaningful
• [ ] If self-hosting: configure OTel passthrough and verify traces route to your backend
• [ ] Check eval alignment score after a code change — verify drift is detected

Security Notes

• API key is stored in an environment variable — never hardcode it.
• Traces may contain sensitive prompt data. Review what your agent sends before enabling cloud-hosted Latitude.
• Self-hosting eliminates data leaving your infrastructure — use this for compliance-sensitive workloads.

FAQ

Q: Does Latitude work with self-hosted models? A: Yes. Latitude is provider-agnostic and supports any LLM with an OpenAI-compatible API, plus OTel ingest for custom setups.

Q: Is the eval alignment feature automatic? A: Evals are built from your team’s judgments. Latitude generates them automatically but they reflect your own criteria — not a generic benchmark.

Q: What is the data retention policy on the hosted version? A: Check latitude.so for the latest on hosted plan limits. Self-hosting gives you full control.

Q: Does this work with non-agentic LLM applications? A: Yes — traces work for any LLM call. The agent-specific features (tool call tracking, multi-turn sessions) are additive, not required.

Conclusion

Latitude fills the observability gap that most AI agent tutorials gloss over: what happens when your agent goes wrong in production? With issue clustering, semantic search, and alignment evals, it gives you the debugging depth that print statements and generic logging cannot.

The open-source license and provider-agnostic design make it safe to adopt without vendor lock-in. If you are running AI agents in production and do not have a dedicated observability stack, Latitude is worth a look.

Start at latitude.so or self-host using the docs.