Meet Apple's Evaluations framework

Import Evaluations. Conform a type to Evaluation. Give it samples and evaluators. Attach that type to a @Test with the .evaluates trait. Read the aggregate in the test body and #expect a threshold.

The recipe is agent + system + tasks + verifier. Apple's Evaluation type is that recipe in Swift. The Evaluation and the session under test are the agent plus the system. Each ModelSample row is a task. Evaluators are the verifiers. A verifier can be a simple Swift function or a content check. For work that is not a yes or no, a rubric tells the verifier what good looks like. ModelJudgeEvaluator is the official path for those soft traits.

If the agent still fails after you write the rubric, the usual causes are an incomplete checklist, a model that is not strong enough, bad or incomplete context, or a weak prompt.

import Evaluations
import Testing

struct TagQuality: Evaluation {
    var dataset = samples
    let tagCount = Metric("TagCount")

    var evaluators: Evaluators {
        // Code-based or ModelJudgeEvaluator sketches go here.
        Evaluators()
    }
}

struct TagQualityTests {
    @Test(.evaluates(TagQuality()))
    func tagsStayUseful() async throws {
        let result = EvaluationContext.current.result
        let evaluation = TagQuality()
        let mean = result.aggregateValue(.mean(of: evaluation.tagCount))
        #expect(mean >= 0.8)
    }
}

That sketch follows WWDC26 session 298. Confirm initializers and Evaluators syntax in the installed docs. A Metric can pass, fail, score, or ignore. ModelJudgeEvaluator is the official model-as-judge path when a code check cannot see quality.

Start with 20 to 30 focused samples. Use a code evaluator when you can measure the fact in Swift. Use a judge only for tone or similar soft traits. Pin the prompt version on every sample.

Xcode 27 shows an evaluation report with per-sample prompts and replies. Treat that report like an Instruments trace: it can hold sensitive text.

Pin the prompt version from Version prompts like product copy on every sample. If a hop was wrong, start from Trace every step of an AI call and add that prompt as a row.

Next, grade tool paths.