EDGE FDE
IntermediateOn-Device Streaming Metrics (TTFT)
Users feel the app is faster when the first token arrives sooner. You can prove it with numbers.
Stream LanguageModelSession tokens and measure TTFT and inter-token latency for on-device and Private Cloud Compute.
Paint partial text, clock first token and gaps, and compare SystemLanguageModel with PrivateCloudComputeLanguageModel. iOS 27 and macOS 27 only.
Lessons
Key concepts
- Users judge speed from the first token, not total generation time.
- Stream with
LanguageModelSession.streamResponseand paint partial text in SwiftUI. - Clock TTFT and inter-token gaps with
ContinuousClock. - Compare
SystemLanguageModelon-device withPrivateCloudComputeLanguageModel.
Takeaways
- You can stream tokens into the UI and prove speed with TTFT, gaps, and total time.
- You can log the same four numbers for on-device and PCC with a model field.
- You can confirm in-app clocks in the Xcode 27 Foundation Models instrument.
Agent briefs
Related courses
Trace every step of an AI call · Test AI behavior before you ship · Version prompts like product copy · Core AI vs Core ML vs MLX · Local LLM Runtime Traps (Ollama + llama.cpp)
Official references
Foundation Models · WWDC26 243 Instruments · WWDC26 319 Private Cloud Compute