---
name: From the metal to the model
description: >-
  A step-by-step recipe for teaching how Apple silicon runs a
  Fashion-MNIST clothing classifier. Start at Metal and profiling.
  Move through Core AI runtime and SwiftUI contracts. Then explain
  conversion and training before you open the original PyTorch to
  Core AI course again.
---
# From the metal to the model

Use this runbook for an agent or teacher explaining the existing clothing-classifier project from the bottom of the stack upward. The reader already finished, or quickly looked through, [the original course](../pytorch-core-ai-xcode/index.html). The backward question is: **what has to be true so Apple silicon can run a clothing inference (one prediction) in a useful way, and what must each earlier layer give the layer below?** Start with the operation the GPU or Neural Engine can run. Move upward only after the lower limit is clear. Do not use CUDA. Do not invent SDK names. Do not use em dashes or quizzes.

## Fixed contract

- Platform is a Mac and a real Apple-silicon device. Assume Xcode 27 beta for profiling examples.
- The portable file is `clothing.aimodel`; its function is `main`.
- Input is `image`, float32 NCHW `[1, 1, 28, 28]`, grayscale pixels in `0...1`.
- Output is `logits`, float32 `[1, 10]`; the class order is the Fashion-MNIST order from the original course.
- Prepare or specialise the model once, then reuse it. A tap must not reload and prepare the model.
- Core AI may send work to the CPU, GPU, or Neural Engine. Metal is how the GPU runs work, not a requirement that every op uses the GPU.
- `TorchMetalKernel` and `register_custom_kernels` are example names from the custom Metal path. Show function names as sketches that can change by release, and point readers to the installed Apple and `coreai-torch` docs.

## Ordered backward path

1. [What the GPU actually runs](01-metal-shaders.html) · [agent brief](01-metal-shaders.llms.md)
2. [Read Instruments from the chip up](02-instruments-from-silicon.html) · [agent brief](02-instruments-from-silicon.llms.md)
3. [Treat Core AI as the program that runs the model](03-core-ai-runtime.html) · [agent brief](03-core-ai-runtime.llms.md)
4. [Keep SwiftUI thin: it only feeds the model](04-swiftui-as-thin-client.html) · [agent brief](04-swiftui-as-thin-client.llms.md)
5. [Convert so the runtime gets a graph it can run](05-conversion-exists-for-runtime.html) · [agent brief](05-conversion-exists-for-runtime.llms.md)
6. [Training must give conversion a model it can use](06-training-owes-conversion.html) · [agent brief](06-training-owes-conversion.llms.md)
7. [Meet the original pipeline again](07-back-to-the-pipeline.html) · [agent brief](07-back-to-the-pipeline.llms.md)

## Execution recipe

1. Begin with a hardware question. Describe a tensor operation, the graph that contains it, the portable `.aimodel`, and the app that feeds it. Grow the diagram one layer at a time.
2. Use Xcode 27 beta and a real device when you talk about traces. Separate Specialization, Load, Setup, and Inference. Read GPU and ANE tracks as evidence, not as labels to speed up without thinking.
3. Explain `AIModel`, `InferenceFunction`, and `NDArray` as the host contract. Open `clothing.aimodel`, its `main` function, and the `image` to `logits` interface. Prepare once.
4. Walk backward from that tensor contract to PhotosPicker and preprocessing. The SwiftUI layer exists to produce valid bytes and shape. A UI bug is a contract break when it changes the tensor.
5. Explain `torch.export`, `get_decomp_table()`, and `TorchConverter` as the path to IR the runtime can run. If custom Metal kernels are used, register them before `add_exported_program`, with version-sensitive signatures clearly marked.
6. Explain fixed shapes, class order, CPU-saved weights, and Mac MPS or CPU training as things conversion needs from training. Never add a CUDA path.
7. End by rebuilding the original map, then send the reader to [the original course's first lesson](../pytorch-core-ai-xcode/01-the-pipeline.html). Read every arrow as a requirement for the layer below.

## Key concepts

- This course walks backwards from Metal and Apple silicon up to the original pipeline.
- The contract is `clothing.aimodel`, function `main`, `image` `[1, 1, 28, 28]` float32 0-1, `logits` `[1, 10]`.
- Core AI can run on CPU, GPU, or Neural Engine. Metal is how the GPU runs work.
- Each layer above must give the layer below shapes, types, supported ops, and prepare once.

## Takeaways

- You can read the clothing classifier from the chip up, not only from training down.
- When something breaks, ask which lower-layer requirement failed before you blame the UI.
- You can reopen the original pipeline and read every forward arrow as a requirement for the layer below.

## Completion standard

A successful explanation names the hardware limit, separates preparation from inference, keeps the tensor contract, names the conversion file, and says why training choices make conversion possible. It links to every human lesson and every matching `.llms.md` brief. The final step is the original lesson with the backward reading, not a new parallel pipeline. Every human lesson HTML has **Key concepts** and **Takeaways** sections before lesson-nav. The course home has the same two headings for the whole course.

## Environment setup

Use macOS with Python 3.11+ and `uv`:

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p clothing-coreai && cd clothing-coreai
uv init --python 3.11
uv venv
uv add torch torchvision coreai-torch
uv run python -c "import torch, torchvision; print(torch.__version__, torchvision.__version__); print('MPS:', torch.backends.mps.is_available())"
uv run python train.py
```

Run lesson scripts with `uv run`. Prefer MPS on Apple silicon, and use CPU as the fallback. Do not add CUDA.

## Environment setup

Use Xcode 27 on macOS. Create an iOS or macOS SwiftUI App target, add the exported `.aimodel` to the target's model/resources build phase, add the **Core AI** framework under Frameworks, Libraries, and Embedded Content, and install the **Metal Toolchain** in Xcode Settings > Components. Confirm the installed toolchain with:

```bash
xcode-select --install
xcodebuild -version
```

Build the bundled model once before adding UI. Core AI and `NDArray` names can change with the SDK version, so check the exact names in the installed Xcode docs.