What the GPU actually runs

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p clothing-coreai && cd clothing-coreai
uv init --python 3.11
uv venv
uv add torch torchvision coreai-torch
uv run python -c "import torch, torchvision; print(torch.__version__, torchvision.__version__); print('MPS:', torch.backends.mps.is_available())"
uv run python train.py

The original course asks, “How do I get Fashion-MNIST labels into an app?” This course asks a harder question first: what has to be true so Apple silicon can run one clothing inference (one prediction) in a useful way? Start with the chip. The app, the runtime, the converter, and the training job all have to give this layer a shape, a type, and a graph it can run.

On a Mac, Core AI can send a graph to the CPU, the GPU, or the Neural Engine. Metal is how the GPU runs work. It is not CUDA. It also does not mean every step will run on the GPU. We start here because a graph is only useful when its steps can be turned into real hardware work.

Grow the map one limit at a time

A Metal operation needs buffers (chunks of data), thread layout, and a data type. It cannot take a vague “image”. The graph above it must describe the operation in a clear, exact way.

Core AI turns supported graph operations into work Apple silicon can run. If a model needs its own GPU code, the Core AI and coreai-torch custom Metal path can include Metal Shading Language through things like TorchMetalKernel and register_custom_kernels. Treat those names as the idea, not an exact API you can paste. Exact types and function names change by release, so check the Apple Core AI custom Metal docs and the installed coreai-torch docs. WWDC25 session 325 is good background for the custom-kernel path.

# Illustrative sketch only. Check the installed release for exact signatures.
custom = TorchMetalKernel(source=metal_source, name="my_op")
register_custom_kernels([custom])
converter = TorchConverter().add_exported_program(exported_program)

The order matters: register custom kernels before add_exported_program. Do not invent a registration call from this sketch. If you do not need a custom kernel, the compiler still turns ordinary operations into the hardware path that is available.

The portable file is a contract around that graph. In the original course, conversion with coreai-torch gives the model a main function with image in and logits out. Here, read that file from the bottom: it must describe operations that the runtime can turn into hardware work.

What the lower layer needs

• A known set of operations, or custom Metal code registered early enough for conversion.
• Clear tensor shapes, layouts, and element types.
• A graph that can be compiled for Apple silicon, not a CUDA-only path.

That is why we start here. Later lessons can only stay accurate if they keep these limits. Next, use a trace as proof of what the chip really did in Instruments from the chip up. For the original course, keep the original pipeline lesson nearby.