---
title: "Meet Apple's Evaluations framework"
lesson_id: "02"
---

# Meet Apple's Evaluations framework

- **Lesson ID:** 02
- **Goal:** Introduce Evaluation, ModelSample, Metric, ModelJudgeEvaluator, and the .evaluates Swift Testing trait.
- **Human lesson:** [02-evaluations-framework.html](02-evaluations-framework.html)

## Prerequisites

- Lesson 01, or the grade-not-equals idea.

## Inputs, outputs, and artifacts

- **Inputs:** A small sample set.
- **Outputs:** A named Evaluation and a threshold.
- **Artifacts:** Progressive `.flow` diagrams and a test sketch.

## Agent build steps

1. Name Evaluation, dataset, evaluators, and Metric.
2. Map agent + system + tasks + verifier onto Evaluation, samples, and evaluators.
3. Show .evaluates and #expect on a mean.
4. Mention rubrics and ModelJudgeEvaluator for non-binary "good."
5. If the agent fails after a rubric: incomplete checklist, weak model, bad context, or weak prompt.
6. Mark the sketch as a sketch.
7. Warn that reports can hold prompts.

## Constraints

Keep the language simple, friendly, and literal. iOS 27 and macOS 27 only. No quizzes, Mermaid, or em dashes. Do not invent API names. Mark sketches as sketches and tell the reader to confirm them in the Xcode 27 SDK.

## Key concepts

- Recipe: agent + system + tasks + verifier. Samples are tasks. Evaluators are verifiers.
- An `Evaluation` has `ModelSample` rows, evaluators, and `Metric` values.
- Attach with `@Test(.evaluates(...))` and read the aggregate from `EvaluationContext`.
- Start with 20 to 30 focused samples.
- Treat evaluation reports like traces. They can hold sensitive text.

## Takeaways

- Keep one Evaluation type per feature and one threshold the team can defend.
- Pin the prompt version on every sample.
- Use code evaluators when Swift can measure the fact. Use a rubric and `ModelJudgeEvaluator` when "good" is not binary.

## Acceptance checks

- Official type names are used.
- .evaluates is shown.
- Sibling links, progressive `.flow` diagrams, and an Agent brief link are present.
- [ ] Proceed to [lesson 03 brief](03-grade-the-trajectory.llms.md).