Familiar – Open-Source Local AI Agent for macOS and iOS

TL;DR

TL;DR: Familiar is an open-source local AI agent for macOS and iOS that runs LLMs entirely on-device, letting you build private, offline-capable agent workflows without sending data to the cloud.

Source and Accuracy Notes

• Homepage: https://thoughts.jock.pl/p/familiar-local-ai-agent-mac
• GitHub: Check the post for repository link
• Platform: macOS, iOS

What Is Familiar?

Familiar is an open-source project that brings AI agent capabilities directly to Apple devices. Instead of routing requests through OpenAI, Anthropic, or any cloud provider, Familiar runs LLMs locally using Apple’s ML frameworks (and optionally Ollama), keeping all prompts, context, and outputs on your machine.

The project targets developers and power users who want to experiment with multi-step agent workflows — task planning, tool use, memory — without surrendering data to third-party servers.

Setup Workflow

Step 1: Install Dependencies

Familiar relies on a local LLM runtime. The recommended path is Ollama for cross-model flexibility:

# Install Ollama (macOS)
brew install ollama

# Pull a model (Mistral is a good starting point)
ollama pull mistral

Step 2: Clone and Run

git clone <familiar-repo-url>
cd familiar
pip install -r requirements.txt
python main.py

Step 3: Configure Your Model

Edit the config file to point to your local endpoint:

# Point to local Ollama
export OLLAMA_BASE_URL="http://localhost:11434"

Step 4: Start an Agent Task

python -m familiar.agent --task "Summarize my last5 notes"

Deeper Analysis

Privacy by design. Every request stays on-device. This matters for:

• Working with sensitive code, documents, or user data
• Running in air-gapped environments
• Avoiding API costs for high-volume workloads

Model flexibility. Familiar is model-agnostic — swap in any Ollama-supported model (Llama 3, Phi-3, Mistral, CodeLlama) without changing the agent logic.

Tool-use architecture. The agent framework exposes a tool-calling interface similar to OpenAI’s function calling, but implemented against local model inference. You can register custom tools in Python:

from familiar import tool

@tool
def read_file(path: str) -> str:
    """Read contents of a local file."""
    with open(path) as f:
        return f.read()

iOS support. The iOS build uses Core ML for on-device inference, which is more constrained than the macOS Ollama path but enables truly mobile agent workflows.

Practical Evaluation Checklist

• [ ] Runs entirely offline
• [ ] No API key required
• [ ] Supports tool-use / function calling
• [ ] Configurable system prompt
• [ ] iOS build available
• [ ] Memory / context window management
• [ ] Extensible tool registration

Security Notes

• Data never leaves the device — no telemetry, no cloud sync
• Sandboxed execution — file access tools run within the app sandbox on iOS
• Model provenance — you control which model weights are downloaded and from where

FAQ

Q: What models does Familiar support? A: Any model available through Ollama (Llama 3, Mistral, Phi-3, CodeLlama, etc.). On iOS, Core ML-compatible models are supported.

Q: How does it compare to ChatGPT’s agent mode? A: Familiar gives you the same agentic workflow patterns (task decomposition, tool use, memory) but entirely local. No API costs, no data leaving your machine.

Q: Does it work without internet? A: Yes — once the model is downloaded, Familiar runs in full offline mode.

Q: Is this production-ready? A: It’s an open-source project under active development. Suitable for experimentation and personal workflows; for production, evaluate the tool-use reliability and context window limits.

Conclusion

Familiar fills a specific niche: developers who want to explore agentic AI workflows on Apple hardware without cloud dependencies. The privacy story is strong, the setup is straightforward, and the tool-use framework is extensible. If you’ve been looking for a local-first alternative to cloud-based agent services, Familiar is worth a weekend experiment.

For related posts, see /blog/local-llm-inference-guide/ and /blog/ollama-vs-lmstudio/.