Langy – Automated AI Engineer for LangWatch

TL;DR

TL;DR: Langy embeds a full coding agent inside LangWatch — it reads production traces, writes Scenario tests and evaluations, opens a pull request on your actual codebase, and proves every fix in CI.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: langwatch.ai/langy
• Source repository: none publicly listed (SaaS platform, no open-source repo announced)
• License: proprietary (no open-source license verified)
• HN launch thread: news.ycombinator.com/item?id=42868689 (9 points — Show HN, July 2026)

What Is Langy?

Langy is an automated AI engineer that lives inside LangWatch, a platform for LLM observability, evaluation, and agentic AI testing. Rather than acting as a copilot that helps a developer type code, Langy operates at the system level — it analyzes production traces, identifies failure patterns, writes the test suite that catches those patterns, and submits a pull request with the actual fix.

The product is aimed at engineering teams running complex AI agents in production. The core loop:

1. A PM or domain expert describes a behavioral problem with plain language (“the agent keeps misrouting billing queries”)
2. Langy reads the relevant traces, finds the failure pattern, writes Scenario simulations to reproduce it
3. Langy drafts the code or prompt change and opens a PR
4. CI runs the simulations Langy wrote — passing or failing — so review is about the change, not about trust

The launch post calls this “an automated AI engineer that does the work and shows the diff.”

Setup Workflow

Langy runs inside LangWatch. There is no self-hosted option announced and no standalone install.

Step 1: Create a LangWatch Account

Sign up at app.langwatch.ai.

Step 2: Connect Your Agent Framework

LangWatch is OpenTelemetry-native, supporting any agent framework that emits traces. According to the docs, supported providers include OpenAI, Anthropic, Bedrock, Azure, and ElevenLabs, among others. You bring your own API keys.

Step 3: Connect GitHub

Langy needs GitHub access to open pull requests. The launch post states Langy contributes exclusively through PRs — it never touches your codebase without a human approving the diff.

Step 4: Run Langy Against Your Traces

You can point Langy at a specific trace, a batch of production sessions, or ask it to cluster and analyze thousands of traces. It will surface findings, propose tests, and (optionally) open the PR.

Deeper Analysis

What Makes Langy Different

Most AI coding tools operate as assistants inside a developer’s editor. Langy’s positioning is different — it is a standalone agent that works on the platform where your evaluation data already lives. The differentiation claims:

• Closed loop from trace to PR — most debugging flows require a human to translate a production complaint into a code change. Langy automates that translation.
• Tests written by the same agent that found the bug — the simulations Langy writes to reproduce a failure become the regression suite that runs in CI on every future PR.
• Human-in-the-loop by design — Langy never merges. It opens a PR and attaches the CI results so the engineer reviews the change, not the intent.

OpenTelemetry-Native Architecture

LangWatch traces anything that emits OpenTelemetry spans. The docs confirm this covers most modern agent frameworks. This means Langy is not tied to a specific model provider or harness — if LangWatch traces it, Langy can analyze it.

Voice Agent Testing

LangWatch launched with a voice agent demo — Langy plugged into a robot over MCP, testing a customer-support voice agent. The claim is that voice agent testing works the same way as text: simulated conversations, scored against a rubric you define.

Security Model

Langy runs in a fully sandboxed VM. Credentials stay in your LangWatch account. Secret redaction is on by default, so API keys do not leak into context. GitHub access is strictly PR-based — no direct commits or branch pushes.

Practical Evaluation Checklist

• [ ] Connect an existing agent framework (OpenAI, Anthropic, or similar)
• [ ] Run a production trace batch through Langy — does it identify real failure patterns?
• [ ] Does Langy write Scenario tests that actually reproduce the failure?
• [ ] Does the opened PR include passing CI results from the simulations?
• [ ] Is secret redaction working correctly on your API keys and credentials?
• [ ] Does the human review step work as described — are diffs clean and reviewable?

Security Notes

• Bring your own API keys — LangWatch does not provide model access; you pay your own provider
• GitHub via PR only — no direct code access without human approval
• Secret redaction on by default — credentials are redacted from traces before they enter context
• Sandboxed VM execution — Langy runs in an isolated environment, not on your infrastructure

FAQ

Q: Does Langy work with any agent framework? A: LangWatch traces via OpenTelemetry, which covers most modern frameworks. The docs list OpenAI, Anthropic, Bedrock, Azure, and ElevenLabs as explicitly supported, with others compatible if they emit OTel spans.

Q: Can Langy fix code without human approval? A: No. Langy opens pull requests and attaches CI results. It never merges. Every change goes through a human code review step.

Q: Is there a self-hosted option? A: Not at launch. Langy is a SaaS product running inside LangWatch’s cloud.

Q: What happens to my production data? A: Traces are processed in LangWatch. Secret redaction is on by default. The launch post and docs confirm credentials stay within your account and are not used for model training.

Conclusion

Langy is an interesting point on the agent tooling map — a coding agent that operates on the evaluation platform rather than inside an editor. The closed loop from production trace to PR is the core claim. Whether it reliably handles real production failure patterns at scale is the open question.

The target user is an engineering team that already runs LangWatch for observability and wants to close the gap between “found a problem in traces” and “fix shipped in CI.” If that gap is costing you time, Langy is worth a pilot.

• LangWatch: langwatch.ai
• Langy page: langwatch.ai/langy
• Docs: docs.langwatch.ai