A 5-part primer
How do LLMs work?
A plain-English, five-part primer on how large language models like ChatGPT actually work, distilled from Andrej Karpathy's deep dive. Read it in order, or jump to the part you're curious about.
- 1
How an LLM is trained: the internet in a box
Pretraining is the first stage of building something like ChatGPT. By the end of this piece you will understand how raw internet text becomes a model that can continue any document, and why that model is not yet an assistant.
8 min read → - 2
Inside the model: predicting the next token
What actually sits behind the chat box is one giant fixed math function. This piece explains what that function is, how it generates text one token at a time, and why that shape sets a hard limit on what it can do in a single step.
8 min read → - 3
From base model to assistant
A base model is an internet-document simulator. Post-training turns it into ChatGPT by teaching it to imitate human labelers who write ideal answers. Once you see the model as a simulation of those labelers, its behavior stops feeling magical.
8 min read → - 4
Why LLMs make things up, and what fixes it
A language model always emits a plausible next token, even when it has no idea. Once you see why, you can tell when to trust it, when to hand it a tool, and when to check its work.
8 min read → - 5
Teaching models to think: RL and reasoning
After a model has read the internet and copied expert answers, one stage is left. Reinforcement learning lets the model practice, discover its own way to solve problems, and start to think before it answers.
9 min read →
Understand it, then build with it.
Knowing how these models work makes you far better at directing them. Bring your questions to a free Oslo Vibe Coding drop-in.