dev-tools
Soul AI: An Open-Source Consciousness Simulator
Soul AI pairs a consciousness simulator with an Apache-2.0 model on Hugging Face, runnable through llama.cpp, plus a private offline iPhone companion app.
#ai#local-llm#llama-cpp
Tag
Local LLM Posts
11 posts tagged #local-llm
Browse 11 posts tagged Local LLM. Setup notes, reviews, comparisons, and workflow patterns.
All Posts
dev-tools
llmspeed: Feel Local LLM Speed Before You Buy Hardware
llmspeed simulates tokens per second and time-to-first-token for local LLMs across 30 GPUs, Macs and CPUs, so you can judge hardware before buying.
#dev-tools#local-llm#benchmark
ai-setup
AirLLM: Running a 70B Model on a 4GB GPU
AirLLM runs 70B-parameter models on a single 4GB GPU. Here is what the repo claims, what the numbers imply, and where the approach breaks down.
#github-trending#local-llm#llm-inference
dev-tools
ds4: DeepSeek 4 inference engine written in C
ds4 is an MIT licensed C implementation of DeepSeek 4 Flash and PRO local inference, targeting Metal, CUDA and ROCm with 23,453 GitHub stars.
#github-trending#c#llm-inference
ai-setup
Wolffish - Brain-Inspired Desktop AI Agent
Wolffish is a local-first AI desktop agent that runs entirely on your machine with no cloud dependency. Built with a 15-module brain architecture, markdown-driven behavior, and full system access on
#ai-setup#self-hosted#local-llm
ai-setup
Moltis – Secure Persistent Agent Server in Rust
Moltis is a self-hosted AI agent server written in Rust. One binary, sandboxed tool execution, multi-provider LLMs, voice, memory, and integrations with Telegram, Discord, and MCP tools.
#ai-agent#self-hosted#rust
ai-setup
Colibri – Run GLM-5.2 on Your Machine (Pure C, 25 GB RAM)
Colibri is a pure-C MoE runtime that streams 744B-parameter GLM-5.2 from disk to consumer hardware. Zero Python, zero GPU required — ~25 GB RAM and you are running a frontier model locally.
#ai-setup#local-llm#open-source
ai-setup
Phi-3-MLX: Run Phi-3 Vision and Language Models on Your Mac
Phi-3-MLX brings Microsoft's Phi-3.5-vision and Phi-3.5-mini models to Apple Silicon Macs via MLX. Install with pip and run locally in minutes.
#ai-setup#apple-silicon#local-llm
ai-setup
LLMtary - Local LLM-Powered Penetration Testing
Run autonomous penetration tests with local LLMs. LLMtary chains recon, exploitation, and reporting—Ollama, Claude, GPT-4, and Gemini supported.
#pentesting#security#local-llm
ai-setup
Bodhi App - Run Local LLMs with OpenAI-Compatible API
Bodhi App is a free, open-source desktop app that runs open-source LLMs locally on macOS, Windows, or Linux with an OpenAI-compatible API and built-in Chat UI.
#ai-setup#local-llm#open-source
ai-setup
CanIUseLLM - Check If Your PC Can Run Local LLMs
Instantly check whether your PC can run open-source LLMs like Llama, Mistral, and Gemma locally. Get personalized GPU/VRAM requirements and quantization recommendations.
#ai-setup#dev-tools#local-llm