JungleGym – Open Source Playground for Autonomous Web Agents

TL;DR

TL;DR: JungleGym is a free, open-source playground and API for testing autonomous web agents against verified datasets like Mind2Web and WebArena — so you can measure whether your agent actually works before shipping.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: junglegym.ai
• Source repository: github.com/a16z-infra/JungleGym
• License: Apache 2.0 (verified via GitHub repository metadata; README states “open source”)
• HN launch thread: news.ycombinator.com/item?id=38448995 — “Show HN: JungleGym and TreeVoyager – OSS Data and Tool for Autonomous Web Agents”
• Stars: 207 (GitHub API, as of July 2026)
• Source last checked: 2026-07-01

What Is JungleGym?

JungleGym is an open-source playground and API for building, testing, and benchmarking autonomous web agents. It was created by a16z-infra and provides:

• Ground-truth datasets — Mind2Web, WebArena, and AgentInstruct with verified correct responses
• Sandboxed test websites — realistic emulated e-commerce, social, and administrative sites
• Python API — fetch tasks, submit agent outputs, compare against ground truth
• Playground UI — browse datasets and inspect trajectories visually

From the README: “an open source playground for testing and developing autonomous web agents. This is not an agent itself, but rather a set of tools and datasets for developers building agents.”

Setup Workflow

Step 1: Install the Python client

pip install requests

No official pip package — JungleGym is used via direct API calls with the requests library.

Step 2: Fetch a WebArena task

import requests

WebArena_task = "What is the price range for products from ugreen?"
url = f"http://api.junglegym.ai/get_webarena_by_task?task={WebArena_task}"
response = requests.get(url)
data = response.json()

# Ground truth result:
print(data['data'][0]['eval']['reference_answers']['must_include'])
# → ['6.99', '38.99']

Step 3: Run your agent and compare

After your agent finishes interacting with the sandboxed site, compare its final answer against the ground truth:

# After your agent produces an answer:
agent_answer = agent.get_final_answer()

# Get ground truth from JungleGym API:
ground_truth = data['data'][0]['eval']['reference_answers']['must_include']

# Verify your agent's answer includes the expected values
assert all(str(item) in str(agent_answer) for item in ground_truth)

Step 4: Browse datasets in the playground

Visit junglegym.ai/Mind2Web to filter ~2,000 tasks across 137 real websites. Each task includes full HTML page states and screenshots.

Deeper Analysis

Datasets available

| Dataset | Type | Tasks | Use case | |---|---|---|---| | Mind2Web | Real-world web interactions | ~2,000 across 137 sites | Broad agent development and testing | | WebArena | Sandboxed functional websites | 6 realistic sites | Deep testing of multi-step tasks | | AgentInstruct | LLM fine-tuning trajectories | ~1,800 | Fine-tuning language models on agent tasks |

Sandboxed websites

WebArena provides fully functional emulated sites at http://shop.junglegym.ai (shopping), Reddit-like forums, a CMS, and more — all isolated for safe agent testing without touching production sites.

Comparison with production agent outputs

The workflow is designed for offline evaluation: you run your agent against the real (or emulated) web, then query the JungleGym API for the ground truth to score your agent’s result.

Practical Evaluation Checklist

• Does your agent correctly extract structured data from dynamic web pages?
• Does your agent handle multi-step tasks that require browsing several pages?
• Does your agent produce outputs compatible with the ground-truth format JungleGym expects?
• Are you using the correct dataset for your agent type (Mind2Web for broad coverage, WebArena for deep site-specific testing)?
• Do sandboxed sites match your target production environment?

Security Notes

• Sandboxed test sites are intentionally isolated — agent actions have no effect on real systems
• No authentication required for the public API at api.junglegym.ai
• Ground truth data is read-only — the API only returns reference answers, it does not accept agent action logs

FAQ

Q: Do I need an API key? A: No. The JungleGym API at api.junglegym.ai is publicly accessible without authentication.

Q: Can I use this for production monitoring? A: JungleGym is a benchmarking and development tool, not a production monitoring system. It evaluates agents offline against known-good reference answers.

Q: How does JungleGym differ from Toolformer or other agent frameworks? A: JungleGym is not an agent framework — it does not execute agents. It provides the evaluation infrastructure (datasets, APIs, sandboxed sites) so framework authors can benchmark whether their agents produce correct outputs.

Q: What languages are supported besides Python? A: The API is HTTP-based, so any language with an HTTP client works. Examples in the README are Python, but JavaScript, Ruby, and Go are equally straightforward with fetch, net/http, or go-fetch.

Q: Is there a hosted version or do I self-host? A: Both — the datasets and API are publicly hosted at api.junglegym.ai and junglegym.ai. The code is also open source so you can self-host if needed.

Conclusion

JungleGym fills a real gap in the autonomous agent development cycle: how do you know your agent actually works before deploying it? By providing verified ground-truth datasets, sandboxed test sites, and a simple comparison API, it lets you measure agent quality systematically. Whether you’re building a coding agent, a web scraping agent, or a multi-step workflow automation, JungleGym gives you reproducible benchmarks instead of hand-wavey “looks good to me” testing.

• Try the playground at junglegym.ai
• Browse datasets: junglegym.ai/Mind2Web, junglegym.ai/WebArena
• Read the full API docs at docs.junglegym.ai
• Star the repo: github.com/a16z-infra/JungleGym