← All courses

EDGE FDE

Advanced

Diffusion LoRA to Core AI On Device

A practical skill for putting a personalized model on a phone. Train a LoRA on a Mac so a catalog diffusion model learns a subject or style. Then merge that change into the base model (bake it: merge the LoRA into the base and export one model file). Export a portable .aimodel resource, and run the personalized generator on the device. You own the model, keep the pictures private, get fast results, and work offline because the model file is stored on the phone. A cloud adapter you swap in at request time does not give you those things.

For someone who can run Python on a Mac and ship a small Xcode 27 app. Pair this with LLM LoRA for your writing style. The two courses teach the same personalization skill in two places: pictures here, writing there.

The real path

Core AI has no documented hot-swap LoRA. That means you cannot load a new adapter on the phone at run time. Do not design an app that tries to attach a LoRA on device. Train the LoRA on a Mac with MPS, or CPU if MPS is missing. Merge it into the base. Export with coreai.diffusion.export, or with torch.export plus coreai-torch. Ship the on-device .aimodel resource folder.

Lessons

  1. What you are building
  2. Pick a base and LoRA idea
  3. Train a LoRA on a Mac
  4. Merge, then export
  5. Run the on-device pipeline
  6. Evaluate generations
  7. End-to-end checklist

Key concepts

Takeaways

Agent briefs

FAQ

Can I hot-swap a diffusion LoRA on device?

No. Core AI has no documented runtime LoRA slot. Bake means merge the LoRA into the base and export one model file.

Where do I train the diffusion LoRA?

Train locally on a Mac with MPS, or CPU when an op does not fit. Never add CUDA. The checkpoint stays separate from export.

Which diffusion base should I start from?

Pick sd-1.5 or sd-2.1 with a Core AI export recipe before you collect images. One narrow goal per run: subject or style, not both.

How do I export the diffusion model for the phone?

Merge with merge_and_unload() into the UNet, then export with coreai.diffusion.export or torch.export plus coreai-torch. SD 1.x needs a multi-component folder: text encoder, UNet, and VAE parts.

What do I ship in the Xcode app?

Bundle the intact baked resource folder in Xcode 27. Prepare the pipeline once at launch and generate off the main thread. The app does not download an adapter.

Related courses

Official references

apple/coreai-models · coreai-torch docs · Apple Core AI