Fashion-MNIST end-to-end checklist · PyTorch to Core AI in Xcode

Environment setup

Use Python 3.11 or newer on macOS. uv manages the project and virtual environment. The device check picks Apple Silicon MPS when it is available, and CPU otherwise.

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p clothing-coreai && cd clothing-coreai
uv init --python 3.11
uv venv
uv add torch torchvision coreai-torch
uv run python -c "import torch, torchvision; print(torch.__version__, torchvision.__version__); print('MPS:', torch.backends.mps.is_available())"
uv run python train.py

Save the lesson code as train.py or the named script for this lesson. uv run uses the project environment, so you do not need a global pip install.

Train on a Mac. Download Fashion-MNIST. Train ClothingCNN with MPS if available, otherwise CPU. Do not use CUDA. Save clothing.pt from a CPU model and check one test sample.
Keep the contract. Use float32 NCHW input [1, 1, 28, 28], pixels in 0–1, and ten logits in the fixed Fashion-MNIST class order.
Export and decompose. Call torch.export.export with an example image, then call run_decompositions(get_decomp_table()).
Convert. Add the exported program to TorchConverter with input_names=["image"] and output_names=["logits"]. Call to_coreai(), optimize(), and save clothing.aimodel.
Install the build toolchain. Install the Metal Toolchain in Xcode before you build the app target.
Add and inspect. Add clothing.aimodel. Confirm main, image, logits, shapes, and element types in the model viewer.
Prepare once. Load AIModel and its InferenceFunction in .task or app startup, not on every photo pick.
Preprocess. Resize a CGImage to 28×28, convert to grayscale, scale to 0–1, and create float32 NCHW NDArray.
Run and label. Pass the tensor under image, read logits, apply softmax or argmax, and map the best index to the ten-label constant.
Profile. Use the Core AI Debugger for graph and tensor issues, the Xcode gauge for load and specialisation, and the Core AI instrument for timing.
Test on hardware. Use a bundled sample for a first run that always uses the same image. Use a real device to check latency (how long it takes), which compute unit it uses, and memory behaviour.
Evaluate labels. Print held-out accuracy and a confusion matrix, open-code mistakes, compare Mac versus Core AI JSONL, and apply a yes/no top-label-acceptable rule. Keep Instruments in lesson 06.

If something fails

Check the model viewer before you change Swift. A wrong name or shape is often the whole issue.
Check that decomposition ran before conversion.
Check the app target has the Metal Toolchain and the model file in its build phases.
Compare Swift image prep with the training transform, including grayscale, orientation, scale, crop, and inversion.
Check the first-run trace separately from later inference.
Reduce the model again if you need to isolate an unsupported operation.

Keep the Apple Core AI overview, the Core AI API reference, and the coreai-torch docs beside the project. Check exact call names against the Xcode SDK you use.

You now have a full path: Fashion-MNIST, ClothingCNN, coreai-torch, clothing.aimodel, Core AI Swift, and a top clothing label in SwiftUI.

Key concepts

The runbook covers Mac training, conversion, SwiftUI label, profile, and label eval.
The contract stays locked: ToTensor(), image [1, 1, 28, 28] float32 0-1, logits [1, 10], function main.
Both clothing.pt and clothing.aimodel must exist and match the model viewer.
Triage order: model viewer names and shapes, then decomposition, toolchain, then image prep.

Takeaways

Done when a bundled sample shows a label, the picker uses the same path, and prepare runs once.
Lesson 07 notes ship with the project: matrix, open codes, JSONL parity, yes or no rule.
You now own the full path from Fashion-MNIST to a top label in SwiftUI.