---
title: "The loop that improves the harness"
lesson_id: "04"
---

# The loop that improves the harness

- **Lesson ID:** 04
- **Goal:** Teach the production → traces → golden case → harness change → eval loop. Harness engineering before fine-tuning.
- **Human lesson:** [04-improve-loop.html](04-improve-loop.html)

## Prerequisites

- Lesson 03: a verifier that can fail on purpose.

## Inputs, outputs, and artifacts

- **Inputs:** A bad production outcome and a trace log.
- **Outputs:** One golden row and one harness change.
- **Artifacts:** One Mermaid loop diagram.

## Agent build steps

1. Say the harness is the moat. Fine-tune later.
2. Draw Mermaid: Production run → Bad outcome → Read traces → Add golden case → Change harness → Re-run eval → Production.
3. Say traces are receipts. Link [Trace every step of an AI call](../on-device-llm-tracer/).
4. Say a golden case plus rubric becomes a test. Link [Test AI behavior before you ship](../on-device-eval-harness/).
5. List harness changes: prompt or instructions, tool definition, guardrail, model-choice config.
6. Reuse the secret-margin miss from lesson 03 as a golden case.
7. Say change one piece, then re-run. Do not start a training job to fix a missing cite.

## Constraints

Keep the language simple, friendly, and literal. No quizzes, no em dashes, no invented WWDC sessions. Do not sell a cloud eval vendor. Cross-link tracer and eval-harness.

## Key concepts

- The harness is the moat. Model hype does not replace a loop you can rerun.
- The loop is production, bad outcome, traces, golden case, harness change, eval.
- Traces are receipts for the wrong hop. Use them before you change weights.
- A harness change is a prompt, a tool definition, a guardrail, or a model-choice config.
- Fine-tune after the suite already encodes the miss. Not before.

## Takeaways

- Turn one bad run into one golden row and one harness change.
- Keep traces next to evals: tracer for receipts, eval harness for the gate.
- Do not start a training job to fix a missing cite or a wide context pack.

## Acceptance checks

- Loop Mermaid is present.
- Tracer and eval-harness links are present.
- Fine-tune is last, not first.
- [ ] Proceed to [lesson 05 brief](05-write-what-good-looks-like.llms.md).