---
title: "A recall test and the speed trap"
lesson_id: "04"
---

# A recall test and the speed trap

- **Lesson ID:** 04
- **Goal:** Measure search quality and embedding speed before you ship.
- **Human lesson:** [04-recall-test-and-speed.html](04-recall-test-and-speed.html)

## Prerequisites

- Lesson 01 server running.

## Inputs, outputs, and artifacts

- **Inputs:** Labelled query to document pairs.
- **Outputs:** Recall@1 per setup and a timing table.
- **Artifacts:** `recall_test.py`, `length_timing.py`.

## Agent build steps

1. Save `recall_test.py` and replace DOCS and QUERIES with 50 or more real pairs.
2. Run `uv run python recall_test.py` and record recall for each setup.
3. Run `uv run python length_timing.py` against the server.
4. On CUDA with flash attention, use a build after PR #30416 or set `-fa off`.
5. Add the recall test to CI.

## Constraints

Keep the language simple, friendly, and literal. No quizzes, no em dashes. Brand Edge FDE only. Use small local models. Do not invent flags, endpoints, or issue numbers. 

## Key concepts

- Recall@1 counts how often the right document ranks first.
- A small, easy test set cannot show differences between setups.
- llama.cpp #30398 made inputs that are not a multiple of 256 tokens much slower on CUDA with flash attention.
- Speed depends on build, backend, and input length.

## Takeaways

- Build a recall set of 50 or more real queries, with hard cases.
- Run it in CI on every model, prefix, dimension, or runtime change.
- Time embeddings at lengths around multiples of 256 on your own hardware.

## Acceptance checks

- The toy result (all 1.0) is reported honestly with the reason.
- llama.cpp #30398 is linked with the fix date.
- [ ] Return to the [course SKILL.md](SKILL.md).