---
name: Diffusion LoRA to Core AI on device
description: >-
  Teach an agent the on-device personalization path for diffusion: train a
  LoRA on a Mac (MPS else CPU), merge it into a catalog base, export a baked
  Core AI .aimodel resource, and run it on device. No documented hot-swap LoRA.
---
# Diffusion LoRA to Core AI on device

This course is a practical skill for putting a personalized model on a phone. The job is local adaptation, a baked artifact (merge the LoRA into the base and export one model file), and on-device generation. You own the model, keep pictures private, get fast results, and work offline because the model file is stored on the phone. A cloud adapter you swap in at request time does not give you those things.

## The real path

Core AI has no documented hot-swap LoRA. Do not invent a runtime that attaches a new adapter on device.

1. Train the LoRA on a Mac. Use MPS when `torch.backends.mps.is_available()` is true. Use CPU otherwise. Never add CUDA.
2. Merge or fuse the adapter into the base UNet or the training recipe's target component.
3. Export the ordinary merged graph with `uv run coreai.diffusion.export <short-name-or-path> --output-dir ...`, or with `torch.export` plus `coreai-torch` `TorchConverter`.
4. Ship the multi-component `.aimodel` resource folder. Load it with current Core AI Swift diffusion utilities in Xcode 27.

## End-to-end agent recipe

1. Read [lesson 01](01-what-youre-building.llms.md) for the artifact chain and the no-hot-swap boundary.
2. Use [lesson 02](02-pick-base-and-lora-idea.llms.md) to choose `sd-1.5` or `sd-2.1` and write a testable subject or style contract.
3. Use [lesson 03](03-train-lora-on-mac.llms.md) for local PyTorch training, checkpoints, and holdout previews.
4. Use [lesson 04](04-merge-then-export.llms.md) to bake the adapter, then export.
5. Use [lesson 05](05-on-device-pipeline.llms.md) to package the resource folder and generate on device.
6. Use [lesson 07](07-evaluate-generations.llms.md) to error-analyze a golden-set grid before you score.
7. Use [lesson 08](08-checklist.llms.md) before declaring the delivery complete.

## Constraints

- Course bases are `sd-1.5` and `sd-2.1`. Cite [apple/coreai-models](https://github.com/apple/coreai-models), [coreai-torch](https://apple.github.io/coreai-torch/main/), and [Apple Core AI](https://developer.apple.com/core-ai). Confirm flags with `--help` and `--dry-run` on the installed tools.
- The export CLI takes a positional model (registry short-name, Hugging Face id, or a local merged folder). Do not invent a `--model` flag.
- No CUDA, quizzes, Mermaid, or em dashes. Use progressive HTML `.flow` diagrams that grow one box at a time.
- Cross-link [PyTorch to Core AI in Xcode](../pytorch-core-ai-xcode/), [From the metal to the model](../core-ai-from-metal-back/), [Core AI models, typed](../coreai-models-typed/), [Model architectures in plain English](../model-architectures-plain/), and [LLM LoRA for your writing style](../llm-lora-writing-style/).

## Key concepts

- Train a LoRA on a Mac, merge it into a catalog base, export one `.aimodel` folder, generate offline on device.
- Core AI has no documented hot-swap LoRA. Bake first. Do not attach an adapter at runtime.
- Training uses MPS or CPU only. Never CUDA.
- Course bases are `sd-1.5` and `sd-2.1` from `apple/coreai-models`.

## Takeaways

- You can personalize a diffusion model and ship the baked file on the phone.
- You export with `coreai.diffusion.export` or `torch.export` plus `coreai-torch`.
- You ship a multi-component resource folder, not `adapter_model.safetensors` alone.

## Acceptance checks

- Every human lesson HTML has **Key concepts** and **Takeaways** sections before lesson-nav. The course home has the same two headings for the whole course.
- Seven human lessons (01-05, 07, 08) and seven matching `.llms.md` briefs exist. Lesson number 06 is unused.
- Every lesson has a short frame about running the model on a phone, progressive `.flow` diagrams, a worked path, Failure modes or When this goes wrong, and Done when.
- Every lesson links its own brief and the five related courses.
- Merge-before-export and the absence of a documented runtime LoRA swap are explicit.

## Environment setup

Use macOS with Python 3.11+ and one `uv` project. Install the official export entry points from [apple/coreai-models](https://github.com/apple/coreai-models) so `coreai.diffusion.export` is available, plus `coreai-torch` for the `torch.export` fallback.

```bash
curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p diffusion-lora-coreai && cd diffusion-lora-coreai
uv init --python 3.11
uv venv
uv add torch torchvision diffusers peft transformers accelerate datasets safetensors coreai-torch
uv run python -c "import torch; print(torch.__version__); print('MPS:', torch.backends.mps.is_available())"
uv run coreai.diffusion.export --help
uv run python train_lora.py
```

Use `uv run` for every Python command. Pin the base revision and dataset. Keep training MPS-or-CPU only.

## Xcode setup

Use Xcode 27. Create an iOS or macOS SwiftUI App target. Add the exported resource folder to the target's resources. Add the Core AI framework. Install the Metal Toolchain in Xcode Settings > Components.

```bash
xcode-select --install
xcodebuild -version
```

Confirm current class names in the installed SDK docs. Prepare the pipeline once. The app loads the baked merged resource. It does not attach a LoRA at runtime.