---
title: "How language models read text"
lesson_id: "02"
---

# How language models read text

- **Lesson ID:** 02
- **Goal:** Explain tokens, next-token prediction, and the context window without transformer math.
- **Human lesson:** [02-language-models.html](02-language-models.html)

## Prerequisites

- Lesson 01, or a plain understanding of architecture as input, shape, and output.
- Familiarity with a chat prompt and a written reply.

## Inputs, outputs, and artifacts

- **Inputs:** A text prompt represented as an ordered sequence of tokens.
- **Outputs:** A next token, repeated into a text reply.
- **Artifacts:** Progressive `.flow` sequence from words to tokens to next token to reply.

## Agent build steps

1. Explain that tokens are small text pieces and that the model processes them in order.
2. Draw the first `.flow` from words to tokens.
3. Redraw it with context and next-token prediction added.
4. Redraw it again with the reply as repeated next-token work.
5. Explain the context window as the amount of recent text available in recent context.
6. Contrast this job with [the Fashion-MNIST ClothingCNN course](../pytorch-core-ai-xcode/), which reads pixels and returns a class.
7. Link [From the metal to the model](../core-ai-from-metal-back/) to explain that token work still becomes runtime work on silicon.

## Constraints

Use plain language and the words tokens, next token, and context window consistently. Do not explain transformer math, papers, CUDA, or quizzes. Keep both prior-course links relative. Do not use em dashes. Use HTML `.flow` diagrams, not Mermaid.

## Key concepts

- A language model reads tokens in order and predicts the next token.
- The context window is how much recent text the model can keep for one reply.
- A reply is many next-token steps, not one big jump.
- ClothingCNN reads a pixel grid and returns a class. A language model continues or changes text.

## Takeaways

- You can trace words to tokens to context to next token to reply.
- You can explain why a short reply still takes many steps.
- You can name the context window as a hard limit on recent text.

## Acceptance checks

- Words become tokens, then a next token, then a reply through progressive diagrams.
- Context window is explained as usable recent text, not as a math definition.
- The lesson clearly differs from a photo-to-clothing-label classifier.
- Both prior courses are linked and the Agent brief link is present.
- [ ] Proceed to [lesson 03 brief](03-image-models.llms.md).