---
title: "Keep the prompt cache warm"
lesson_id: "01"
---

# Keep the prompt cache warm

- **Lesson ID:** 01
- **Goal:** Make llama-server reuse the prompt cache in an agent loop, and prove it with numbers.
- **Human lesson:** [01-keep-the-prompt-cache-warm.html](01-keep-the-prompt-cache-warm.html)

## Prerequisites

- llama.cpp installed (`llama-server --version`).
- uv and Python 3.11+.

## Inputs, outputs, and artifacts

- **Inputs:** A system prompt, a tool list, and a user question.
- **Outputs:** A table of `cache_n`, `prompt_n`, and wall time for three prompt layouts.
- **Artifacts:** `cache_check.py` and the server log.

## Agent build steps

1. Do the environment setup from the human lesson.
2. Start `llama-server -hf Qwen/Qwen3-1.7B-GGUF:Q8_0 --port 8080 -c 8192 -np 1`.
3. Save `cache_check.py` from the lesson and run `uv run python cache_check.py`.
4. Confirm case A and C reuse most tokens after the first call, and case B does not.
5. Grep the server log for `selected slot by LCP similarity` and `prompt eval time`.
6. Move any timestamp, file tree, or changing state in your own agent to the end of the prompt.
7. For hybrid models, check the llama.cpp build against issues #20225, #21912, #24055 before blaming the prompt.

## Constraints

Keep the language simple, friendly, and literal. No quizzes, no em dashes. Brand Edge FDE only. Use small local models. Do not invent flags, endpoints, or issue numbers. 

## Key concepts

- The prompt cache reuses tokens that match from the start of the prompt.
- `timings.cache_n` is tokens reused. `timings.prompt_n` is tokens read again.
- One changed token near the top forces a re-read of everything after it.
- Hybrid models use context checkpoints, and some llama.cpp builds had bugs there.

## Takeaways

- Put instructions and tools first. Put time, state, and the new turn last.
- Run the three-case script and compare `prompt_n` before you blame the model.
- If a hybrid model re-reads on every turn, update llama.cpp and test again.

## Acceptance checks

- Case A second call has `cache_n` close to the full prompt and a small `prompt_n`.
- Case B has a large `prompt_n` on every call.
- Environment setup appears before the first runnable example.
- [ ] Proceed to [lesson 02 brief](02-thinking-output-in-the-wrong-field.llms.md).