Guides
Short, plain-English guides to the projects around microgpt.
- What is a token in AI?
What a token is in a language model like ChatGPT, how a tokenizer turns text into numbers, and how byte-pair encoding (BPE) builds its vocabulary, with a worked example.
- nanochat explained
What Andrej Karpathy's nanochat is, what it trains, what it costs and needs, how it relates to nanoGPT and microgpt, and how to understand it from scratch first.
- micrograd explained
What micrograd is, how its backward pass works, how it differs from the Value class in microgpt, and how to build it yourself in Python.
- Attention explained with real numbers
How the attention mechanism in a GPT works, step by step, with a worked example you can check by hand and the real numbers from a trained model.
- nanoGPT vs nanochat vs microgpt
What nanoGPT, nanochat and microgpt each are, how big they are, what they need to run, and which one to start with if you want to understand how GPTs work.