Agent Harness: Build and Improve the Loop

Ship agents that stay safe and get better from real runs, not from model hype.

Define a harness, name its six layers, sketch a small harness, and run an improve loop using traces + golden checks.

The model gets the hype. The harness gets the results.

What you'll learn

Easy: a harness is everything around the model. An agent is a controlled workflow. The model is the smallest, swappable part. The recipe is instructions, scoped context, tools, a verifier, and guardrails.
Hard: six layers. Trigger, Orchestration, Tools, Trusted context (about 80% of success), Control, and Runtime. Model swap is a config change.
God: the harness is the moat. Production to traces to golden checks to a harness change. Write what good looks like before you fine-tune.

Notes that shaped this framing: Mark Ajzenstadt (@mardehaym) on harnesses. The rest of the course stays on a small harness you can improve from real runs.

Lessons

Exercises

Evolve your agent's prompt with GEPA

Key concepts

The harness wraps the model. An agent is trusted context, bounded tools, evaluation, human judgment, and ops ownership.
Six layers: Trigger, Orchestration, Tools, Trusted context, Control, Runtime. Trusted context does most of the work.
A small harness has recipe instructions, a scoped context pack, fixed tools, a verifier, guardrails, and a loop limit.
Improve from production: read traces as receipts, add a golden check, change the harness, re-run the eval. Fine-tune later.

Takeaways

You can name what the harness owns and swap the model without rewriting the product.
You can sketch a tiny Python harness on a Mac or any CPU and grade it with a verifier.
You can turn a bad run into a golden case and a harness change, not a training job.
You can write the bar in the control layer, with a named owner and a stop-and-ask rule.

Agent briefs

FAQ

What is an agent harness?

The harness is everything wrapped around the model. An agent is a controlled workflow: trusted context, bounded tools, evaluation, human judgment, and ops ownership. The model is the smallest, swappable part.

What are the six harness layers?

Trigger, Orchestration, Tools, Trusted context, Control, and Runtime. Trusted context does about 80% of the success. Model choice is config in the runtime layer.

What is in a tiny harness?

Recipe instructions, a scoped context pack, fixed tools such as calc_kpi, a verifier, guardrails, and a MAX_STEPS loop limit. A missing metric stops the loop.

When do I fine-tune the model?

After the suite already encodes the miss. The loop is production, bad outcome, traces, golden case, harness change, eval. Change the harness before you fine-tune.

Who writes what good looks like?

A human. The control layer holds the golden set, guardrails, and approvals. A named owner decides, and the agent stops and asks when evidence is missing or confidence is low.

Run an agent with a state machine · Test AI behavior before you ship · Trace every step of an AI call · Stop prompt attacks and leaks · Build a context packer for on-device AI · AI Full-Stack Operator · Reliable Local Agents (llama.cpp + Ollama)