What you are building

Why this matters when the model runs on a phone

If you put the model on a phone, you want the phone to keep the personalized model. The pictures stay on the phone. The generator works without a network. Wait time is the Neural Engine and the GPU on the phone, not a trip to a server. The file is yours to version, inspect, and take back.

That only holds if personalization is baked into the model you deploy. A cloud LoRA you fetch at request time is not your model file. It needs a network. It can change without your build. It can leak the subject you trained on.

The steps, one box at a time

Start with the thing you already own. Local images of one subject or one style, plus captions you wrote.

Training adds a compact LoRA checkpoint. The base model stays frozen. The adapter is a small set of rank updates (tiny extra weights), not a second full model.

Merge bakes those updates into the UNet or the recipe's target component. After merge you have one ordinary diffusion graph (the usual model pieces, with your change already inside).

Export turns that merged pipeline into a Core AI resource folder. That folder is the thing you version and add to Xcode.

The app loads that baked resource and generates on the device. No adapter download. No runtime swap.

What LoRA is, and what it is not

A LoRA is a small trainable patch. Rank controls how much room the patch has (how many extra weights it can learn). A subject LoRA helps one person, pet, object, or prop appear in new scenes. A style LoRA changes the look while the prompt still names the content. Neither one replaces the catalog base. Neither one is a Core AI runtime feature.

The sibling course, LLM LoRA for your writing style, uses the same bake-then-export rule on a small instruct model. Read it when you want the same method for writing.

Worked path for this map lesson

This lesson has no training loop yet. Set up the Mac project so later lessons can run. Use Python 3.11 or newer. Use uv for the environment. Prefer MPS. Fall back to CPU. Never add CUDA.

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p diffusion-lora-coreai && cd diffusion-lora-coreai
uv init --python 3.11
uv venv
uv add torch torchvision diffusers peft transformers accelerate datasets safetensors coreai-torch
uv run python -c "import torch; print(torch.__version__); print('MPS:', torch.backends.mps.is_available())"
uv run coreai.diffusion.export --help

Write a one-page note in the project root. Name the subject or style, the catalog base you will pin, and the sentence that will count as success. Keep the note next to the later checkpoint list.

{
  "goal": "subject",
  "base": "sd-1.5",
  "success": "The same mug appears in three held-out kitchen scenes.",
  "runtime_lora_swap": false
}

Failure modes

• Designing for a runtime LoRA slot. There is no documented Core AI API for that. Bake first (merge the LoRA into the base, then export).
• Treating the adapter as the thing you ship. The device file is the merged export, not adapter_model.safetensors alone.
• Skipping ownership. If the images, captions, or base license are not yours to keep on the device, stop before training.
• Assuming CUDA later. This course stays on MPS or CPU. A CUDA-only script will not run on the Mac path.

Done when

• You can say the five-box chain out loud without adding a hot-swap step.
• The uv project exists, Python prints a version of 3.11 or newer, and the MPS check ran.
• A goal note records subject or style, the intended base, and runtime_lora_swap: false.

Keep learning

PyTorch to Core AI in Xcode · From the metal to the model · Core AI models, typed · Model architectures in plain English · LLM LoRA for your writing style