---
title: "Thresholds, human fallback, latency"
lesson_id: "04"
---

# Thresholds, human fallback, latency

- **Lesson ID:** 04
- **Goal:** Turn test results into a safe routing rule.
- **Human lesson:** [04-thresholds-fallback-latency.html](04-thresholds-fallback-latency.html)

## Prerequisites

- Lessons 02 and 03.

## Inputs, outputs, and artifacts

- **Inputs:** Labelled set and chosen quant.
- **Outputs:** A threshold, a routing function, and latency numbers.
- **Artifacts:** Threshold sweep output and `route()` function.

## Agent build steps

1. Run the threshold sweep for 0.5, 0.7, 0.9.
2. Pick the lowest threshold with no unacceptable automatic mistakes.
3. Implement `route()` with a human fallback.
4. Record p50 and p95 latency on the target machine.
5. Run with `-np 1`, or split questions that share a prefix (llama.cpp #30482).
6. Log answers with confidence in production.

## Constraints

Keep the language simple, friendly, and literal. No quizzes, no em dashes. Brand Edge FDE only. Use small local models. Do not invent flags, endpoints, or issue numbers. 

## Key concepts

- A confidence threshold splits automatic actions from human review.
- Higher thresholds mean fewer mistakes and more human work.
- Latency depends on hardware, quant, and slots.
- llama.cpp #30482: multi-slot servers can fail on questions with shared prefixes.

## Takeaways

- Pick the threshold from your labelled data, not from a guess.
- Always have a human fallback.
- Measure p50 and p95 on the target machine.
- Use `-np 1` for decision models until #30482 is fixed.

## Acceptance checks

- The sweep table is from a real run.
- There is a human fallback path.
- llama.cpp #30482 is linked and reproduced or cited.
- [ ] Return to the [course SKILL.md](SKILL.md).