---
title: "Grade the tool trajectory"
lesson_id: "03"
---

# Grade the tool trajectory

- **Lesson ID:** 03
- **Goal:** Grade tool-calling with TrajectoryExpectation and ToolCallEvaluator. Name AllPass and PercentagePass.
- **Human lesson:** [03-grade-the-trajectory.html](03-grade-the-trajectory.html)

## Prerequisites

- Lesson 02, or a running Evaluation type.

## Inputs, outputs, and artifacts

- **Inputs:** Prompts that should call tools.
- **Outputs:** Strict and partial tool metrics.
- **Artifacts:** Progressive `.flow` diagrams and a ToolCallEvaluator sketch.

## Agent build steps

1. Define a trajectory as the expected tool path. Grade steps and side effects, not only the last sentence.
2. Use an on-device notes example: log the meeting, write a draft, pause before send.
3. Name ToolCallEvaluator and TrajectoryExpectation.
4. Name ordered, unordered, and disallowed.
5. Link a real bug from the tracer course.

## Constraints

Keep the language simple, friendly, and literal. iOS 27 and macOS 27 only. No quizzes, Mermaid, or em dashes. Do not invent API names. Mark sketches as sketches and tell the reader to confirm them in the Xcode 27 SDK.

## Key concepts

- A good final sentence can hide a wrong tool path. Grade the steps and side effects.
- `TrajectoryExpectation` lists expected tool calls in the transcript.
- `ToolCallEvaluator` writes `ToolsAllPass` and `ToolsPercentagePass`.
- Write samples from real bugs found in traces.

## Takeaways

- If the feature has tools, ship a trajectory eval.
- One trace bug becomes one sample row with the expected tool path.
- Confirm enum cases in the installed Evaluations docs.

## Acceptance checks

- Both official types are named.
- AllPass and PercentagePass are named.
- Sibling links, progressive `.flow` diagrams, and an Agent brief link are present.
- [ ] Proceed to [lesson 04 brief](04-gate-ci.llms.md).