← All courses ← Course home

Lesson 05 of 05

Measure and fall back

Time the first token. If the pick fails or is too slow, drop to the next legal model. Never fall toward a privacy violation.

Agent brief (llms.md)

A route that always picks PCC will look smart in a demo and drain a commute. Measure on a device, not only the simulator.

TryFirst pick
WatchError or timeout
NextLegal smaller path
func fallbackOrder(from route: String, privacy: PrivacyBound) -> [String] {
    switch (route, privacy) {
    case ("pcc", .applePCCAllowed):
        return ["pcc", "system"]
    case ("coreai", _):
        return ["coreai", "system"]
    default:
        return ["system"]
    }
}

Do not add PCC to a mustStayOnDevice fallback list. If the system model is down and privacy blocks PCC, show a clear offline state. That is a product win, not a silent cloud hop.

Record milliseconds to first useful token, input tokens, output tokens, cachedTokenCount if present, and whether you fell back. Session 241 shows usage fields including cached and reasoning tokens. Those are not a semantic cache. See Reuse answers that are almost the same for that layer.

Sources: WWDC26 241, WWDC26 242, Foundation Models, Core AI.

Done. You can pick the smallest Apple model that still works, prove it with a trace, and fall back without breaking privacy.

Key concepts

  • Fallback order must stay legal for the privacy bound.
  • Never add PCC to a mustStayOnDevice fallback list.
  • Record time to first token, token counts, cached tokens, and a fallback flag.
  • Measure on a device, not only the simulator.

Takeaways

  • If the system model is down and privacy blocks PCC, show a clear offline state.
  • A route that always picks PCC drains battery.
  • cachedTokenCount is not a semantic cache.