dev-tools 5 min read

nanochat: training a small ChatGPT on a single node

nanochat is Andrej Karpathy's MIT-licensed Python project for training a ChatGPT-style model end to end, pitched as the best ChatGPT $100 can buy, with 59,915 stars.

#github-trending #llm #model-training #python #chatgpt
By
Share: X in
GitHub social preview card for karpathy/nanochat

TL;DR: nanochat is Andrej Karpathy’s Python implementation of training a ChatGPT-style model from scratch, built around the idea that a useful model should cost about $100 to train. It is MIT licensed and has 59,915 stars and 8,414 forks.

What it is

The repository description is one sentence: “The best ChatGPT that $100 can buy.”

That is a deliberately specific claim, and it sets the frame for everything else. nanochat is not a wrapper around someone else’s model and it is not a fine-tuning recipe for an existing checkpoint. It is a compact, from-scratch training codebase that covers the full pipeline for building a small chat model, including the tokenizer, pretraining on raw text, adaptation on instruction data, and training on conversation data.

The reason a codebase like this exists is pedagogical as much as practical. Most “train your own LLM” material is either a set of shell commands around someone else’s library or a research paper you need a cluster to reproduce. nanochat is positioned as the readable middle path: small enough to follow in one sitting, real enough to produce a model you can actually chat with.

The $100 framing

A budget claim like this is only meaningful if you know what it buys, so it is worth being precise about the shape of the setup.

The dollar figure assumes commodity cloud GPU time, not hardware you already own. The premise is that a small model, trained on a modest dataset for a few hours on a single multi-GPU node, can reach a quality where talking to it feels like a real product rather than a toy. The code is organized around that goal, which is why the repository is small and the steps are explicit.

What the framing gets you is a working end-to-end reference you can modify. Change the data, change the model size, change the training schedule, and you can see what happens because you are reading the actual training loop rather than a config file full of magic.

What the metadata says

FieldValue
LanguagePython
LicenseMIT
Stars59,915
Forks8,414
Open issues139
Default branchmaster
Created2025-10-13
Last push2026-09-07

Two details stand out. The repository declares no topics at all, which is unusual for a project this visible and means topic-based discovery will not lead you to it. And the last push was on September 7, 2026, roughly two months before this writing, so treat it as a stable reference rather than a project receiving daily attention.

MIT is the permissive baseline, with no copyleft and no patent grant to speak of. For code you plan to read, adapt, and build on, that is about as frictionless as licensing gets.

How to try it

git clone https://github.com/karpathy/nanochat.git
cd nanochat
pip install -e .

The package installs as a module with nanochat as the entry point, and the workflow is split into stages you run in order. Training from raw text:

# Pretrain on raw text
python3 -m nanochat.train

Then chat with what you produced:

# Talk to the trained model
python3 -m nanochat.chat

Check the documentation for the exact module names and required dataset paths before you rely on that syntax. The repository also carries a speedrun-style path that targets the budget claim directly, and the accompanying scripts download and prepare the data the training stages expect.

Budget for the hardware story to be the hard part, not the code. The instructions assume access to GPU instances, and the sensible first move is to work through the smallest configuration available to you rather than trying to reproduce the headline run.

When to use it

nanochat fits when you want to understand how a chat model gets built rather than just call an API, when you need a reference implementation you can debug line by line, or when you are prototyping a training change and want a baseline that trains fast enough to iterate on.

It does not fit when you need a frontier-quality model, since the entire premise is a small model on a modest budget. It also does not fit if you want a production serving stack, because the goal is the training pipeline rather than a hardened inference service.

Source and Accuracy Notes

  • Project page: https://github.com/karpathy/nanochat
  • Source repository: karpathy/nanochat (verified via GitHub API html_url)
  • License: MIT (verified via GitHub API license.spdx_id)
  • Stars at time of writing: 59915
  • Forks at time of writing: 8414
  • Open issues at time of writing: 139
  • Repository created: 2025-10-13
  • Last push: 2026-09-07