---
title: "Set the context and size the memory"
lesson_id: "02"
---

# Set the context and size the memory

- **Lesson ID:** 02
- **Goal:** Fix the context size for the app's real endpoint and confirm the memory fits.
- **Human lesson:** [02-set-context-and-size-memory.html](02-set-context-and-size-memory.html)

## Prerequisites

- Lesson 01 done.

## Inputs, outputs, and artifacts

- **Inputs:** Model, longest prompt, number of parallel requests.
- **Outputs:** A model or server config that passes `ctx_guard.py`.
- **Artifacts:** Modelfile, `gguf_arch.py` output.

## Agent build steps

1. Create a Modelfile with `PARAMETER num_ctx` or start Ollama with `OLLAMA_CONTEXT_LENGTH`.
2. Run `ctx_guard.py` and confirm PASS.
3. Run `gguf_arch.py <file.gguf> <context>` to estimate KV memory.
4. Multiply by `OLLAMA_NUM_PARALLEL` and compare with free memory.
5. Confirm with `ollama ps` that CONTEXT is right and PROCESSOR is not split to CPU.

## Constraints

Keep the language simple, friendly, and literal. No quizzes, no em dashes. Brand Edge FDE only. Use small local models. Do not invent flags, endpoints, or issue numbers. 

## Key concepts

- `OLLAMA_CONTEXT_LENGTH` and a Modelfile work for every endpoint.
- Per-request `num_ctx` works on `/api/chat` only.
- KV cache memory grows with layers, KV heads, head size, context, and parallel slots.
- `ollama ps` shows context and CPU/GPU split.

## Takeaways

- Use a Modelfile or `OLLAMA_CONTEXT_LENGTH` if your app uses the OpenAI endpoint.
- Estimate KV memory from the GGUF before you raise the context.
- Run `ctx_guard.py` again after every change.

## Acceptance checks

- `ctx_guard.py` prints PASS.
- The KV estimate is written down with the formula.
- [ ] Proceed to [lesson 03 brief](03-measure-speculative-decoding.llms.md).