Build on-device chat

Why the chat screen is the real proof

Style only counts when the phone answers without a network. That is proof the phone has the model. Latency is the time from send to first useful token on the target device. Privacy means no request leaves the process. The clothing-classifier course proves the same last step for labels. This lesson proves it for writing style.

The app loads the merged artifact. It does not attach a LoRA at runtime. A second writing style is a second bake.

The steps, one box at a time

Environment setup

Use Xcode 27. Create an iOS or macOS SwiftUI App. Add Core AI. Install the Metal Toolchain. Add the .aimodel and tokenizer files to Copy Bundle Resources.

xcode-select --install
xcodebuild -version
# File > New > Project > App, Interface: SwiftUI
# Xcode Settings > Components > Metal Toolchain
# Add writing-style.aimodel and the tokenizer sidecar to the target.

Keep the Mac uv project so you can rebuild the artifact.

cd writing-style-lora
uv run python -c "import torch; print(torch.__version__); print('MPS:', torch.backends.mps.is_available())"

Worked path

Use the current Core AI language APIs for the SDK in Xcode. A CoreAILanguageModel and LanguageModelSession shape is illustrative. Confirm names in the installed docs.

import SwiftUI
import CoreAI

// Illustrative shape only. Verify names in the installed SDK.
@MainActor
final class Writer: ObservableObject {
    @Published var reply: String = ""
    @Published var status: String = "Preparing"

    private var session: AnyObject?

    func prepare() async {
        do {
            let model = try await loadDocumentedLanguageModel(
                resource: "writing-style.aimodel",
                tokenizer: "tokenizer"
            )
            session = try makeDocumentedSession(model: model)
            status = "Ready"
        } catch {
            status = "Load failed: \(error.localizedDescription)"
        }
    }

    func send(_ userText: String) async {
        do {
            reply = try await documentedRespond(session, to: userText)
            status = "Local"
        } catch {
            status = "Generate failed: \(error.localizedDescription)"
        }
    }
}

Prepare once in a .task or app setup. Keep prompts inside the context budget. Move generation off the main UI path. Make cancellation and memory pressure visible. Do not fetch an adapter.

Evaluate the real app. Short prompt. Long prompt. Two-turn rewrite. A request outside the training set. Compare voice, whether it follows the request, invented facts, latency, and memory against a base-model build. Read Apple Core AI documentation for current deployment details.

Failure modes

• Missing tokenizer sidecar. Output will look like a different model.
• Preparing on every send. The setup cost will look like generation cost.
• Copying placeholder type names into production. Verify the installed SDK.
• Testing only on the Mac merged model. Measure the target platform preset you exported.
• Building a loader that expects an adapter file. There is no documented Core AI LoRA swap.

Done when

• The target contains the .aimodel, the tokenizer, and the Core AI framework.
• Prepare succeeds once. A review prompt returns a local reply with a recorded latency.
• A note says the run used the baked resource only, on the intended macOS or iOS preset.

Keep learning

PyTorch to Core AI in Xcode · From the metal to the model · Core AI models, typed · Model architectures in plain English · Diffusion LoRA to Core AI on device