Why the first token matters

A language model can take a few seconds to finish. The user does not wait for the finish. The user waits for the first token. If that first token is late, the app feels stuck.

Time to first token (TTFT) is the time from the moment you start the request to the moment the first piece of text is ready. Total time is the time until the last token. They are different numbers. An Edge FDE reports both.

import Foundation
import FoundationModels

func clocks(session: LanguageModelSession, prompt: String) async throws {
    let start = ContinuousClock.now
    var first: Duration?
    let stream = session.streamResponse { prompt }
    for try await partial in stream {
        if first == nil { first = ContinuousClock.now - start }
        _ = partial
    }
    let total = ContinuousClock.now - start
    print("ttft:", first ?? .zero, "total:", total)
}

Streaming does not make the first token faster. Streaming shows tokens as they arrive. Waiting for the full string hides a model that is already working. Show the stream. Then measure the two clocks.

This course uses Apple's Foundation Models API on iOS 27 and macOS 27. The same session type can run on the device or on Private Cloud Compute. You will stream both paths and compare the numbers.

Read Trace every step of an AI call when a slow first token is not the model, but context packing or a tool. Read Core AI vs Core ML vs MLX if you still mix Foundation Models with a file you trained.

Next, stream the tokens into the UI.