ai-setup
BitNet: the official 1-bit LLM inference framework
BitNet is Microsoft's official inference framework for 1-bit LLMs, written in C++, MIT licensed, with 40,931 GitHub stars and 3,810 forks.
#github-trending#ai-setup#llm-inference
Tag
LLM Inference Posts
4 posts tagged #llm-inference
Browse 4 posts tagged LLM Inference. Setup notes, reviews, comparisons, and workflow patterns.
All Posts
ai-setup
AirLLM: Running a 70B Model on a 4GB GPU
AirLLM runs 70B-parameter models on a single 4GB GPU. Here is what the repo claims, what the numbers imply, and where the approach breaks down.
#github-trending#local-llm#llm-inference
dev-tools
ds4: DeepSeek 4 inference engine written in C
ds4 is an MIT licensed C implementation of DeepSeek 4 Flash and PRO local inference, targeting Metal, CUDA and ROCm with 23,453 GitHub stars.
#github-trending#c#llm-inference
ai-setup
Cactus – Ollama for Smartphones
Run LLMs locally on iOS and Android with an OpenAI-compatible API layer, custom quantization kernels, and zero-copy edge-cloud handoff.
#ai-setup#mobile#llm-inference