Compliant LLM – Security Testing for AI Agents

TL;DR

TL;DR: Compliant LLM is an open-source Python toolkit that stress-tests AI agents and MCP servers against NIST AI RMF, OWASP, HIPAA, and GDPR — using a built-in dashboard to surface attack results and compliance gaps.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: fiddlecube.ai/compliant-llm — verified product page
• Source repository: github.com/fiddlecube/compliant-llm — README read end-to-end
• License: MIT (verified via GitHub API license.spdx_id)
• HN launch thread: news.ycombinator.com/item?id=44126670 — Show HN
• Source last checked: 2026-06-28 (commit main branch)

What Is Compliant LLM?

Compliant LLM is a Python toolkit built by Fiddlecube (YC W23) for infosec, compliance, and gen AI teams who need to verify their AI agents behave correctly under attack and comply with internal policies.

The core workflow: point it at a prompt, agent, or MCP server, pick a compliance framework, and run a suite of adversarial tests. Results appear in a visual dashboard with actionable findings.

Key capabilities from the README:

• Security Testing — 8+ attack strategies including prompt injection, jailbreaking, and context manipulation
• Compliance Analysis — against NIST AI RMF, ISO, OWASP, GDPR, HIPAA, and EU AI Act
• Provider Support — OpenAI, Anthropic, Gemini, Mistral, Groq, Deepseek, Azure, vLLM, Ollama, Nvidia Nim, Meta Llama (via LiteLLM)
• Visual Dashboard — interactive UI for browsing test results
• MCP Server Auditing — test MCP servers directly for security and compliance gaps
• Detailed Reporting — exportable findings with remediation guidance

Setup Workflow

Prerequisites

• Python 3.10 or higher
• pip or uv

Step 1: Install

pip install compliant-llm

Step 2: Configure an LLM Provider

The toolkit uses LiteLLM under the hood. Set your provider key as an environment variable:

# OpenAI example
export OPENAI_API_KEY="sk-..."

# Or use a models.csv file for multiple providers
# See: https://docs.litellm.ai/docs/providers

Step 3: Launch the Dashboard

compliant-llm dashboard

This starts a local web UI. Open the URL printed in the terminal to configure tests and browse results.

Step 4: Run a Compliance Audit

From the dashboard, select your target (prompt string, agent endpoint, or MCP server URL) and choose a compliance framework. Click Run to execute the test suite.

For headless operation, the CLI also accepts direct arguments:

compliant-llm audit --target "your-agent-prompt" --framework NIST

Deeper Analysis

Attack Strategies Tested

The README documents 8+ built-in attack types. The exact list grows with each release, but the categories include:

• Prompt injection via user input manipulation
• Jailbreaking via instruction override
• Context window exhaustion
• System prompt extraction
• Multi-turn context poisoning
• Tool call injection (for agentic systems)
• Role confusion attacks
• Data exfiltration via response manipulation

Compliance Frameworks

Each framework maps test results to specific controls. The supported set from the README:

• NIST AI RMF — the primary target, maps to the full risk management framework
• OWASP — LLM Top 10 coverage
• HIPAA — for healthcare AI applications
• GDPR — EU data protection requirements
• ISO 42001 — AI management system standard
• EU AI Act — risk classification and obligations

MCP Server Testing

Compliant LLM can connect directly to an MCP server and test it without wrapping the target in additional code. This is useful for auditing third-party MCP integrations before deploying them inside your agentic workflow.

Provider Flexibility

Using LiteLLM as the abstraction layer means you can swap providers without rewriting tests. If your organisation uses Azure OpenAI internally but you want to test against Anthropic for a second opinion, change one environment variable.

Practical Evaluation Checklist

• Does the dashboard launch without errors on a fresh Python 3.10+ environment?
• Are all 8+ attack strategies accessible from the UI, or only via CLI?
• Does the NIST AI RMF mapping cover all four tiers (Govern, Map, Measure, Manage)?
• Is the MCP server test compatible with the official modelcontextprotocol/servers registry?
• Does the telemetry respect an opt-out flag?
• Are there pre-built CI/CD integrations (GitHub Actions, etc.)?
• Does the report export to PDF/JSON, or HTML only?

Security Notes

Telemetry: The README explicitly states that Compliant LLM tracks anonymised usage statistics. If you are deploying this inside a regulated environment (HIPAA, GDPR), review their privacy policy and set up network-level blocking if needed.

Local testing: The toolkit runs entirely locally by default — your prompts and test data do not go to Fiddlecube’s servers unless telemetry is active.

Scope: Compliant LLM is a testing and auditing tool, not a runtime guard. It identifies weaknesses; it does not patch them. Findings should feed into your SDLC before production deployment.

FAQ

Q: What is the minimum viable test — just one prompt? A: Yes. Point --target at any string and run against one framework. A full audit against all 8+ attack strategies takes longer but does not require a full agent setup.

Q: Does it work with self-hosted models? A: Yes, via the Ollama or vLLM LiteLLM integrations. Set the appropriate OLLAMA_BASE_URL or VLLM_API_BASE environment variable.

Q: How is this different from a standard penetration test? A: A pen test targets your infrastructure. Compliant LLM targets the AI layer — prompt injection, context manipulation, and compliance against AI-specific frameworks like NIST AI RMF that most pen test tools do not cover.

Q: Is there a hosted/cloud version? A: Fiddlecube offers a demo booking via their site. The open-source version is fully self-hosted with no artificial limits.

Conclusion

Compliant LLM fills the gap between generic security tooling and the specific risks that AI agents introduce — prompt injection, context poisoning, and compliance drift across frameworks. The built-in dashboard makes it approachable without a security background, while the CLI supports CI/CD integration for teams that want automated gate checks.

If you are building agentic systems today, auditing them against NIST AI RMF and OWASP before production is a sensible baseline. Compliant LLM makes that tractable without requiring a dedicated red team.

Install it with pip install compliant-llm, point it at your agent, and run your first compliance report.