Where the first-token wait goes

xcodebuild -downloadComponent MetalToolchain

curl -LsSf https://astral.sh/uv/install.sh | sh
git clone https://github.com/apple/coreai-models.git && cd coreai-models
uv venv
# The repo's uv project pins its own deps (coreai-torch, PyTorch).
# PyTorch uses MPS on Apple silicon and falls back to CPU.
uv sync
uv run python -c "import torch; print('mps' if torch.backends.mps.is_available() else 'cpu')"
# Export a small language model for iOS.
uv run coreai.llm.export Qwen/Qwen3-0.6B --platform iOS

The four events

Apple's Instruments guide splits Core AI activity into four event types. In the order they usually appear:

• Specialization turns the portable .aimodel into code for this device's hardware and OS version. Apple calls it often the most time-intensive operation. Each model produces at most one, and none when it is already cached.
• Load happens when the app first loads the model into memory. Many Load events during a session mean the app reloads the model.
• Setup comes before each inference.
• Inference is the run itself. For a language model, the first token waits for the prefill of the whole prompt.

Look with Apple's tools

1. Debug gauge: run from Xcode and open the Core AI gauge in the Debug navigator. It shows median Inference, Load, and Specialization times, plus a table of events. See Monitoring model performance with the debug gauge.
2. Instruments: Product > Profile, choose the Core AI template, record while you trigger the model. The trace shows which compute units run your model and whether specialization happens when you expect. Profile on a real device, with no other heavy apps running.

Time it in your own code too

Tools are for the lab. In the field you need logs. Wrap the steps you control: model init (which includes specialization when needed), loadFunction(named:), and the first run.

import CoreAI
import os

struct FirstTokenTimings: CustomStringConvertible {
    var cacheHit = false
    var modelInit: Duration = .zero
    var loadFunction: Duration = .zero
    var firstRun: Duration = .zero
    var description: String {
        "cacheHit=\(cacheHit) modelInit=\(modelInit) loadFunction=\(loadFunction) firstRun=\(firstRun)"
    }
}

let log = Logger(subsystem: "dev.edgefde.coreai", category: "first-token")

func timedLoad(modelURL: URL, functionName: String) async throws -> (InferenceFunction, FirstTokenTimings) {
    var t = FirstTokenTimings()
    let clock = ContinuousClock()

    t.cacheHit = (try AIModelCache.default.model(for: modelURL, options: .default)) != nil

    var start = clock.now
    let model = try await AIModel(contentsOf: modelURL, options: .default)
    t.modelInit = clock.now - start

    start = clock.now
    guard let function = try model.loadFunction(named: functionName) else {
        throw CocoaError(.fileReadCorruptFile)
    }
    t.loadFunction = clock.now - start

    log.info("\(t.description, privacy: .public)")
    return (function, t)
}

Then time the first run separately with the same clock. Log the device model and OS version with every line. Specialization output is tied to both, so an OS update can bring the slow first launch back.

For the streaming side (time to first token and time between tokens as the user sees them), see On-Device Streaming Metrics (TTFT).

Sources

• Apple: Analyzing model runtime performance with Instruments
• Apple: Monitoring model performance with the debug gauge