A language model takes in tokens. A token might be a whole short word, part of a longer word, punctuation, or a space pattern. The important part is the job. The model works through a list in order.
Context is the recent text
The context window is how much recent text the model can keep in recent context for one response. Your prompt, earlier turns, and documents you give it all share that space. When that space is full, older text may no longer be there for the next guess.
This is different from the Fashion-MNIST clothing classifier course. ClothingCNN takes in an image grid and chooses a class. A language model takes in token context and continues or changes text.
Why a reply takes many steps
The model chooses one next token, adds it to the working text, then predicts again. A short reply is a chain of these small jobs.
From the metal to the model follows a different path through the same kind of system. Its Metal lesson shows why token work later becomes instructions running on the chip. The architecture changes the work. The hardware still has to do it.
Next, compare that sequence job with models that read and make images.
Key concepts
- A language model reads tokens in order and predicts the next token.
- The context window is how much recent text the model can keep for one reply.
- A reply is many next-token steps, not one big jump.
- ClothingCNN reads a pixel grid and returns a class. A language model continues or changes text.
Takeaways
- You can trace words to tokens to context to next token to reply.
- You can explain why a short reply still takes many steps.
- You can name the context window as a hard limit on recent text.