dev-tools
llmspeed: Feel Local LLM Speed Before You Buy Hardware
llmspeed simulates tokens per second and time-to-first-token for local LLMs across 30 GPUs, Macs and CPUs, so you can judge hardware before buying.
#dev-tools#local-llm#benchmark
Tag
LLM Posts
39 posts tagged #llm
Browse 39 posts tagged LLM. Setup notes, reviews, comparisons, and workflow patterns.
All Posts
tutorial
llm-course: A Free Roadmap Into Large Language Models
mlabonne's llm-course is a free, Apache-2.0 licensed LLM curriculum with roadmaps and Colab notebooks. 84,071 stars and counting.
#github-trending#llm#machine-learning
dev-tools
LiteLLM: One OpenAI-shaped gateway for 100+ LLM APIs
LiteLLM puts 100+ LLM providers behind one OpenAI-shaped API, adding cost tracking, guardrails, load balancing, and logging. 60.7k stars.
#github-trending#ai-gateway#llm
ai-setup
Tencent WeKnora: An Open-Source LLM RAG Knowledge Platform
WeKnora is Tencent's open-source LLM knowledge platform for turning raw documents into a queryable RAG, reasoning agent, and self-maintaining Wiki. Go, 25k stars trending.
#github-trending#ai-setup#rag
ai-setup
ECC: A Performance Harness for Claude Code Agents
ECC is an MIT-licensed agent harness performance system bundling skills, instincts, memory, security, and research-first development for Claude Code, Codex, and Opencode. 260k stars.
#github-trending#ai-setup#claude-code
dev-tools
Colibri: Run Frontier MoE Models on Hardware You Already Own
Colibri is a pure-C inference engine for frontier mixture-of-experts LLMs with zero external dependencies. It streams expert weights from disk, runs on commodity CPUs, and ships in a tiny binary.
#github-trending#dev-tools#c
ai-setup
MathModelAgent: An LLM Agent That Writes Your Math Modeling
Open source AI agent for mathematical modeling competitions. Ingests a problem, plans an approach, runs the analysis, and produces a submission-ready paper.
#github-trending#ai-setup#llm
dev-tools
LLM Wiki: Turn Your Documents Into an Interlinked Knowledge
Cross-platform desktop app that ingests your documents and turns them into an organized, interlinked knowledge base you can query with an LLM.
#github-trending#dev-tools#llm
dev-tools
Fleak – Serverless Data API Builder with No-Code IDE
Build and deploy serverless data API backends in minutes with Fleak's no-code IDE. Chain SQL, LLM, and Lambda steps into production endpoints — no infrastructure config needed.
#api-builder#serverless#no-code
ai-setup
UpTrain – Open-Source LLM Evaluation Platform
UpTrain is an open-source platform with 20+ pre-configured LLM evaluation checks, root cause analysis, and a local dashboard. Apache-2.0 licensed.
#ai-setup#dev-tools#llm
ai-setup
Kolosal AI – Lightweight Local LLM Runner (LM Studio
Kolosal AI is an open-source C++ desktop app for running LLMs offline on any AVX2 CPU or AMD/NVIDIA GPU. ~20MB compiled, Apache-2.0 licensed.
#ai-setup#local-ai#llm
ai-setup
PromptMage – Self-Hosted LLM Workflow Management
A Python framework that simplifies building and managing multi-step LLM applications. Self-hosted, open source, with version control for prompts and built-in FastAPI.
#ai-setup#dev-tools#llm
ai-setup
Tokenless – Cut Your LLM Inference Bill in Half
Tokenless is a YC S26 model router that fans out API calls to multiple models, selects the best responder, and cancels the rest — dropping costs by 50 percent with no quality loss.
#llm#inference#cost-optimization
self-hosted
LLMKube – Kubernetes Operator for Local LLM Inference
LLMKube is an open-source Kubernetes operator that manages local LLM inference across NVIDIA, Apple Silicon Metal, and AMD GPUs from a single YAML spec, with an OpenAI-compatible API.
#llm#kubernetes#self-hosted
ai-setup
Cactus – AI Inference on Your Smartphone
Open-source edge AI engine that runs local LLMs on iOS, Android, and wearables using custom quantization, CPU/GPU kernels, and an OpenAI-compatible API.
#ai-setup#mobile#edge-ai
dev-tools
RΞASON - TypeScript Framework for LLM Apps
RΞASON is a minimalistic TypeScript framework for building LLM apps with structured outputs, function-based agents, streaming, and zero-config OpenTelemetry observability.
#llm#typescript#dev-tools
ai-setup
RapidFire AI – 16–24x Faster LLM Experimentation
RapidFire AI is an open-source experiment execution framework for LLM customization — RAG, fine-tuning, and post-training — delivering 16–24x throughput over sequential pipelines.
#ai-setup#dev-tools#open-source
ai-setup
Forge – Guardrails for Self-Hosted LLM Tool-Calling
Forge is a Python reliability layer that wraps self-hosted LLM backends with tool-calling guardrails, pushing an 8B model from 53% to 84%+ success on agentic tasks.
#ai-setup#self-hosted#llm
ai-setup
FlexLlama – Run Multiple Local LLMs with a Dashboard
A lightweight self-hosted tool that runs multiple llama.cpp server instances simultaneously, with OpenAI v1 API compatibility and a real-time GPU dashboard.
#ai-setup#self-hosted#llm
ai-setup
Firecrawl – Web Data API for AI Agents
Open-source web scraping API that converts any URL into clean Markdown or structured data for AI agents. Supports search, scrape, crawl, and map endpoints with LLM-ready output.
#ai-tools#web-scraping#llm
ai-setup
Sudo - Unified LLM API for Developers
One API to route across OpenAI, Anthropic, Google, and more. Sudo handles context management and monetization so developers can focus on building AI products.
#ai-setup#dev-tools#llm
ai-setup
Vocode – Build Voice Conversations with LLMs
Vocode is an open-source Python library for building real-time voice-based LLM applications. Deploy to phone calls, Zoom meetings, and more. MIT licensed, 3.7k stars.
#ai-setup#python#voice
dev-tools
prompttools - Open-Source LLM and Vector DB Evaluation
Self-hostable tools for testing prompts, LLMs, and vector databases from code or a local Streamlit playground. Apache-2.0 licensed.
#developer-tools#llm#open-source
dev-tools
Unstract – LLM Document Extraction Platform
Open-source platform for extracting structured data from unstructured documents using LLMs. Deploy as API or ETL pipeline.
#document-ai#llm#etl
ai-setup
LocalScore - Open Local LLM Benchmark
Measure how well your hardware runs local AI models. Open-source benchmark tests prompt speed, generation speed, and time-to-first-token for LLMs.
#ai-setup#dev-tools#llm
dev-tools
Parastore: Isometric 3D Retail Simulator with LLM Shoppers
Parastore is a synthetic-consumer retail sandbox where you draw a store, generate LLM personas, and watch them shop an isometric 3D space with pathfinding and per-aisle analytics.
#github-trending#dev-tools#agent-simulation
dev-tools
journal-adapt-writing-skill: Journal Style Adaptation
Learn how journal-adapt-writing-skill turns a manuscript and a target journal's papers into an auditable dynamic writing skill for Claude Code and Codex revision work.
#github-trending#dev-tools#academic-writing
dev-tools
Skyvern - Open-Source AI Browser Automation
Skyvern is an open-source AI agent that automates complex browser-based workflows using LLMs, supporting multi-step tasks, form filling, and API integrations.
#browser-automation#ai-agent#open-source
ai-setup
AI Agent Tools Directory – 316 Tools for Building Agents
A curated directory of 316 tools for building AI agents, LLMs, and RAG pipelines. Browse by category, use case, and stack.
#ai-setup#dev-tools#directory
ai-setup
TokenSpeed for Agentic LLM Inference
TokenSpeed targets agentic LLM inference with static-compiler parallelism, KV-safe scheduling, and aggressive Blackwell-era performance goals in its preview release.
#github-trending#dev-tools#llm
ai-setup
Karpathy CLAUDE.md: Four Principles to Fix Claude Code
Drop Karpathy's viral CLAUDE.md skill into Claude Code or Cursor — 4 behavioral principles that kill over-engineering, drive-by refactors, and assumption bugs.
#claude-code#ai-coding#karpathy
tutorial
How to Train Your GPT From Scratch
raiyanyahya/how-to-train-your-gpt teaches modern decoder-only LLM internals with a 12-chapter walkthrough, runnable notebooks, a commented `main.py`, and fine-tuning guides aimed at Python developers.
#github-trending#dev-tools#python
dev-tools
Airbolt - Drop an AI Chat Into Your React App in 3 Lines
Airbolt is a zero-backend LLM SDK that drops a working AI chat into any React app in three lines. Bring your own OpenAI keys; the SDK encrypts them with AES-256-GCM and rate-limits per user.
#ai-sdk#react#openai
dev-tools
Parastore - Simulating Retail Stores With AI Shoppers
Parastore turns shelf plans and product catalogs into 3D store simulations where shopper personas move, browse, and buy for synthetic consumer testing.
#agent-simulation#retail-tech#threejs
ai-setup
Langtail – Prompt Management Platform for AI Teams
Langtail is a collaborative prompt management platform that lets teams build, test, and deploy AI prompts with a spreadsheet-like interface, evaluation tools, and an AI firewall.
#ai-tools#prompt-engineering#dev-tools
ai-setup
Embedding.io - Turn Websites Into Knowledge Bases for LLMs
Crawl, chunk, and vectorize any website into a queryable knowledge base. REST API with collections, scheduled updates, and built-in RAG for LLM applications.
#rag#llm#ai-setup
dev-tools
Rivet - Visual AI Agent IDE for Complex LLM Apps
Rivet is an open-source desktop IDE from Ironclad for building, debugging, and deploying LLM prompt graphs. Ships with a TypeScript runtime library.
#rivet#ai-agents#llm
dev-tools
Future AGI – Open Source LLM Observability Platform
Future AGI is a self-hostable platform for evaluating, observing, and improving LLM apps. Includes tracing, evals, simulation, datasets, an AI gateway, and.
#github-trending#dev-tools#ai-agent
dev-tools
Raindrop Workshop Agent Debugging Guide
Set up Raindrop Workshop for local agent traces, tool-call debugging, replay workflows, SQLite storage, instrumentation, and eval repair loops.
#github-trending#dev-tools#typescript