--- name: The agent harness (and how to improve it) description: >- Teach an Edge FDE to define an agent harness, name its six layers, sketch a tiny Python harness, and run a production-to-trace-to-eval improve loop before fine-tuning. --- # The agent harness (and how to improve it) Use this course when someone wants a safer agent and reaches for a new model first. The harness is everything wrapped around the model. The model is the smallest, swappable part. You improve the harness from real runs. Cite [Mark Ajzenstadt (@mardehaym) on harnesses](https://x.com/mardehaym/status/2097736766245499226) once on the course home. Do not repeat the link in every lesson. ## End-to-end agent recipe 1. Start with [lesson 01](01-harness-not-the-model.llms.md). Easy mode: harness vs model. An agent is a controlled workflow: trusted context, bounded tools, evaluation, human judgment, and ops ownership. Recipe: instructions + scoped context + tools + verifier + guardrails. Research-agent checks: who asked, plan, approved data, trusted calc, cite, stop if confidence is low. 2. Use [lesson 02](02-six-layers.llms.md) for the six layers: Trigger, Orchestration, Tools, Trusted context (about 80% of success), Control, Runtime. Model swap is runtime config. Use three progressive Mermaid diagrams, not one mega diagram. 3. Use [lesson 03](03-sketch-a-harness.llms.md) to run a tiny Python harness with `uv` and Python 3.11+. Environment setup first. Fake KPI tool with fixed logic, scoped context pack, verifier (tool called + answer cites evidence), loop limit. CPU or Mac. No CUDA. 4. Use [lesson 04](04-improve-loop.llms.md) for the improve loop: production run → bad outcome → read traces → add a golden case / rubric → change the harness (prompt, tool def, guardrail, or model choice) → re-run eval. Link [Trace every step of an AI call](../on-device-llm-tracer/) and [Test AI behavior before you ship](../on-device-eval-harness/). Harness engineering before fine-tuning. 5. Use [lesson 05](05-write-what-good-looks-like.llms.md) for the highest leverage: humans write what good looks like. Golden set, approvals, a named owner, and a stop-and-ask rule. Tie this to the control layer. 6. Keep these links exact: [Run an agent with a state machine](../on-device-agent-orchestrator/), [Test AI behavior before you ship](../on-device-eval-harness/), [Trace every step of an AI call](../on-device-llm-tracer/), [Stop prompt attacks and leaks](../on-device-guardrails/), and [Build a context packer for on-device AI](../on-device-context-assembler/). 7. Use the human HTML pages for the reader. Use the `.llms.md` briefs for agent steps and acceptance checks. ## Constraints - Plain spoken English for a non-native reader. - Literal sentences. No flourish, no quizzes. - Do not use em dashes. - Brand Edge FDE only. Do not name a course author. - Small Mermaid diagrams. Progressive layer-by-layer diagrams in lesson 02. Do not build one mega diagram. - Do not invent WWDC sessions. Do not use YouTube for Apple talks. - Teach the sketch in Python 3.11+ with `uv`. No CUDA. Keep it runnable on CPU or a Mac. - Mark the model function as a stand-in you can swap. The harness stays. - Cite the X post only on the course home. ## Key concepts - The harness wraps the model. An agent is trusted context, bounded tools, evaluation, human judgment, and ops ownership. - Six layers: Trigger, Orchestration, Tools, Trusted context, Control, Runtime. Trusted context does most of the work. - A small harness has recipe instructions, a scoped context pack, fixed tools, a verifier, guardrails, and a loop limit. - Improve from production: read traces as receipts, add a golden check, change the harness, re-run the eval. Fine-tune later. ## Takeaways - You can name what the harness owns and swap the model without rewriting the product. - You can sketch a tiny Python harness on a Mac or any CPU and grade it with a verifier. - You can turn a bad run into a golden case and a harness change, not a training job. - You can write the bar in the control layer, with a named owner and a stop-and-ask rule. ## Acceptance checks - Every human lesson HTML has **Key concepts** and **Takeaways** sections before lesson-nav. The course home has the same two headings for the whole course. - Five human lessons exist with the requested basenames. - Five matching `.llms.md` briefs exist and link to their human HTML pages. - Every human lesson has its Agent brief link and sibling-course links where the lesson calls for them. - The course index lists all five lessons, the course SKILL.md, and all five briefs. - The course home has a short What you'll learn section (Easy, Hard, God) and cites the Ajzenstadt harness notes once. - Lesson 01 has a small model-vs-harness Mermaid diagram. Lesson 02 has two or three progressive Mermaid diagrams. Lesson 04 has a loop Mermaid diagram. - There is no em dash, no quiz, and no invented WWDC session. ## Environment setup Lesson 03 needs Python 3.11 or newer and `uv`. The sketch uses the standard library only. Train or infer nothing. No CUDA.