---
title: "Compare quantizations"
lesson_id: "03"
---

# Compare quantizations

- **Lesson ID:** 03
- **Goal:** Choose a quantization with data.
- **Human lesson:** [03-compare-quantizations.html](03-compare-quantizations.html)

## Prerequisites

- Lesson 02 eval script and labelled set.

## Inputs, outputs, and artifacts

- **Inputs:** Two quants of the same decision model.
- **Outputs:** A comparison table.
- **Artifacts:** `q8.txt`, `bf16.txt`.

## Agent build steps

1. Start a server per quant on different ports.
2. Run `eval_decision.py` against each and save the output.
3. Compare accuracy, flips, per-ticket confidence, and latency.
4. If confidences shrink toward uniform on the smaller quant, do not use it (see llama.cpp #30064).
5. For Clef in Ollama, use `PARAMETER num_ctx 4096` if loading fails (Ollama #18865).

## Constraints

Keep the language simple, friendly, and literal. No quizzes, no em dashes. Brand Edge FDE only. Use small local models. Do not invent flags, endpoints, or issue numbers. 

## Key concepts

- Quantization can change probabilities even when the pick stays right.
- Run the same labelled set on each quant.
- Compare accuracy, flips, confidence, and latency.
- Some decision model quants and runtimes have known bugs.

## Takeaways

- Pick the smallest quant whose confidences match the bigger one on your set.
- Check confidence spread, not only accuracy.
- Set `num_ctx` small for Clef in Ollama if it fails to load.

## Acceptance checks

- The table has both quants on the same set.
- The confidence check is stated, not only accuracy.
- [ ] Proceed to [lesson 04 brief](04-thresholds-fallback-latency.llms.md).