Julie – Screen-Aware Desktop AI Assistant with Computer Use

TL;DR

TL;DR: Julie is an open-source, screen-aware desktop AI assistant that sees your screen, answers by voice or text, and can autonomously drive a browser or control your mouse and keyboard — no cloud required.

Source and Accuracy Notes

⚠️ This section is MANDATORY. All links must be verified from actual source, not guessed.

• Project page: github.com/Luthiraa/julie
• Official releases: github.com/Luthiraa/julie/releases
• License: MIT — verified from README
• HN launch thread: news.ycombinator.com/item?id=46825935 (Show HN: Julie Zero)
• Also discussed: news.ycombinator.com/item?id=46415057
• Latest release: v1.2.0 (verified via GitHub Releases API)
• Source last checked: 2026-07-16

What Is Julie?

Julie is a lightweight, transparent desktop AI assistant built in a weekend and open-sourced for anyone who wants to explore or extend it. The core idea: Julie lives on top of your workspace, understands what is on your screen, and responds via voice or text — without forcing you to switch tabs, copy context, or break focus.

Unlike cloud-based assistants, Julie runs locally and uses Groq (Llama 3 70B and Llama 4 Scout) for fast, reasoning-heavy responses. It is also agentic — it can autonomously control your browser, execute terminal commands, and interact with your computer using mouse and keyboard actions.

From the README:

Julie is a lightweight, transparent desktop AI assistant built to reduce context switching. It lives on top of your workspace, understands what’s on your screen, and responds via voice or text without forcing you to switch tabs, copy context, or break focus.

Key Features

Transparent, Floating Interface

Julie appears as a transparent window that blends into your desktop. Ghost Mode (activated with Cmd+Shift+I) makes it click-through so it stays visible while you work without blocking interaction.

Screen Awareness

One-click analysis of your current screen content — Julie reads what is displayed and reasons about it alongside your conversation, eliminating the need to describe context manually.

Voice and Text

Responds via voice or text. This is designed to keep your hands free and your focus on the task rather than on typing.

Agentic Browser Automation (Puppeteer)

Julie can autonomously control a web browser using Puppeteer to:

• Navigate to any URL
• Click on elements using CSS selectors
• Type text into input fields
• Scroll pages up or down
• Read page content and interactable elements
• Execute custom JavaScript

Computer Use on macOS (JXA)

Native system control via JavaScript for Automation (JXA):

• Mouse Movement — move cursor to specific screen coordinates
• Click Actions — left click, right click, double click
• Drag and Drop — drag from one point to another
• Terminal Commands — execute shell commands directly

Setup

Prerequisites

• macOS (Apple Silicon M1/M2/M3 or Intel) or Windows (x64 or ARM64)
• Groq API key — get one at console.groq.com (free tier available)

Download

Get the latest installers from the Releases page:

| OS | Architecture | Installer | |---|---|---| | macOS | Apple Silicon (M1/M2/M3) | Julie-1.2.0-arm64.dmg | | Windows | x64 | Julie-Setup-1.2.0-x64.exe | | Windows | ARM64 (Surface/Snapdragon) | Julie-Setup-1.2.0-arm64.exe |

Running

1. Download and install the .dmg or .exe for your OS.
2. Launch Julie.
3. Enter your Groq API key when prompted.
4. Start chatting or use voice.

For self-host or development (if you want to run from source):

git clone https://github.com/Luthiraa/julie.git
cd julie
# Install dependencies (see README for full dev setup)
npm install
npm run dev

The README notes the project is a pnpm + Turborepo monorepo (Node.js), but exact build steps should be confirmed from the current README as the project evolves.

How It Works Under the Hood

Julie connects to Groq’s API for model inference — specifically Llama 3 70B or Llama 4 Scout. Groq provides the speed needed for real-time, interactive use. Screen content is captured and sent as vision input to the model.

Browser automation uses Puppeteer (headless Chrome), giving Julie full DOM access for navigation, interaction, and content extraction. On macOS, native JXA scripts handle mouse, keyboard, and system-level actions without requiring elevated permissions beyond normal app access.

Practical Evaluation Checklist

• [ ] Installs cleanly on macOS Apple Silicon
• [ ] Groq API key configuration works
• [ ] Transparent window renders and floats correctly
• [ ] Ghost Mode click-through works
• [ ] Screen capture sends correct context to model
• [ ] Browser automation (Puppeteer) navigates and interacts
• [ ] JXA computer use executes mouse movements and clicks
• [ ] Voice response latency is acceptable
• [ ] No data leaves local machine (verify network calls)

Security Notes

• API calls go directly to Groq’s servers — no intermediary. Your prompts and screen context are processed by Groq.
• Browser automation is local — Julie controls a browser on your machine.
• JXA runs on-device for macOS system control.
• No reported security vulnerabilities as of July 2026 (project is new; treat accordingly).
• See SECURITY.md for the reporting policy (note: file was not found in the repository at time of writing — confirm its existence before filing a report).

FAQ

Q: Is my screen data sent to external servers? A: Yes — screen captures are sent to Groq’s API for vision processing. Local-only operation would require a local vision model, which is not currently built in.

Q: Does this work on Linux? A: Not officially listed. Releases cover macOS and Windows only. Linux may work from source but is not a supported target.

Q: What models does Julie use? A: Groq-hosted Llama 3 70B and Llama 4 Scout by default. You can configure which Groq model to use.

Q: Is this related to Apple Intelligence or Microsoft Copilot? A: No — Julie is an independent open-source project built by Luthiraa. It is not affiliated with Apple, Microsoft, or Groq.

Q: Can I self-host the backend? A: Not currently — inference runs through Groq’s hosted API. A fully self-hosted variant would require swapping in a local model and self-hosting the Puppeteer/JXA layer.

Conclusion

Julie is a scratch-your-own-itch weekend project that punches above its weight: a transparent, screen-aware AI desktop assistant with real browser automation and native macOS computer use — all open-source and running on Groq’s fast inference layer.

The agentic capabilities (Puppeteer browser control, JXA mouse/keyboard) are the most differentiated part. If you want a local AI assistant that can actually do things on your machine — not just chat — Julie is worth a look. The MIT license and downloadable installers make it trivial to try.

As a young project (28 stars, v1.2.0), it is early stage. Watch the GitHub for rapid iteration if you plan to integrate it into a workflow.