SESSION 1 · ~45M
What an LLM actually is
Strip away the marketing and a large language model is a statistical engine that predicts the next token — almost everything it can and can't do follows from that one idea.
A large language model is, at heart, a machine that does one thing extremely well: given some text, it predicts what should come next. There is no fact database it searches and no rulebook it consults. The 'knowledge' lives in billions of numerical parameters tuned during training to capture the statistical structure of human language.
Tokens, not words
Models read and write in tokens — fragments that can be a whole common word, a piece of a rare word, or a single character. Tokenization is why a model can rhyme but miscount letters, why it bills and measures length in tokens rather than words, and why an 8,000-token context window holds roughly 6,000 English words rather than 8,000.
- Common words are usually single tokens; rare or technical terms split into several.
- Code and non-English text consume more tokens per idea, so they fill the context window faster.
- Every token you send is both a cost and a slice of the model's limited working memory.
Temperature is the single dial between reliable and creative. Low temperature picks the most probable token each step — consistent, good for facts and code. High temperature spreads chance across less-likely tokens — varied and creative, but more error-prone. Most failures that look like 'the model got confused' are really a temperature or context problem.
Why scale keeps surprising us
Capabilities emerge with size: a model too small to do arithmetic will suddenly manage it once it crosses a scale threshold, even though no one taught it arithmetic directly. This emergent behavior is why the field keeps producing surprises and why bigger, better-trained models reliably unlock new use cases.
TRY IT
Ask any chat model to continue this exactly: 'The supply chain is'. Run it three times, clearing the reply each time. The differing completions are probability and temperature at work — direct evidence that you are steering a statistical process, not querying an encyclopedia.
CHECK YOUR UNDERSTANDING
Why does an 8,000-token context window hold roughly 6,000 English words instead of 8,000?
What happens when you lower the temperature of a model's output?
What does 'emergence' mean in the context of LLM capabilities?
NOTEBOOKLM
Connect your notebook
Paste a NotebookLM share link to embed your notebook for this module.