Measure TTFT and inter-token gaps

Apple's Instruments template reports time to first token, tokens per second, and total latency. You still want the same numbers in your own debug build, so a product log and a CI check can read them.

import Foundation

struct StreamMetrics {
    var startedAt = ContinuousClock.now
    var firstTokenAt: ContinuousClock.Instant?
    var lastTokenAt: ContinuousClock.Instant?
    var yieldCount = 0

    mutating func markYield() {
        let now = ContinuousClock.now
        if firstTokenAt == nil { firstTokenAt = now }
        lastTokenAt = now
        yieldCount += 1
    }

    var ttft: Duration? {
        firstTokenAt.map { $0 - startedAt }
    }

    var interTokenMean: Duration? {
        guard let first = firstTokenAt, let last = lastTokenAt, yieldCount > 1 else {
            return nil
        }
        return (last - first) / (yieldCount - 1)
    }
}

Create the struct, then call streamResponse. Call markYield() on every snapshot. Log TTFT, mean inter-token gap, yield count, and total time. ContinuousClock is monotonic. Do not use wall-clock Date for this.

A yield is a snapshot of aggregated text, not always one model token. Say that in the log. Tokens per second from Instruments is the better token rate. Your yield gaps still show when the UI stalled.

Write the numbers next to the model name, the profile name, and the prompt version. A faster TTFT after a shorter instruction is a real win. A faster TTFT after you dropped a required fact is not.

Use Test AI behavior before you ship so a speed win cannot hide a quality drop.

Next, compare the on-device model with Private Cloud Compute.