Merge and export to Core AI

Why you bake before you export

Core AI converts an ordinary language graph. A LoRA is extra math beside frozen weights. Apple does not document a runtime adapter API for Core AI. Merge so the writing style is stored in the weights. Export that graph. Keep the tokenizer. The model file is stored on the phone.

Keep the unmerged adapter. You cannot extract it cleanly from the baked file.

The steps, one box at a time

Environment setup

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p writing-style-lora && cd writing-style-lora
uv init --python 3.11
uv venv
uv add torch torchvision transformers peft datasets accelerate safetensors coreai-torch
uv run python -c "import torch; print(torch.__version__); print('MPS:', torch.backends.mps.is_available())"
uv run coreai.llm.export --help
uv run coreai.llm.export --list-presets

Worked path: merge

Merge on CPU into a new directory. Do not overwrite the adapter.

from pathlib import Path
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

BASE = "Qwen/Qwen3-0.6B"
REVISION = "pin-this"
ADAPTER = "artifacts/writing-style-lora"
MERGED = Path("artifacts/merged-instruct")

tok = AutoTokenizer.from_pretrained(BASE, revision=REVISION)
base = AutoModelForCausalLM.from_pretrained(BASE, revision=REVISION)
model = PeftModel.from_pretrained(base, ADAPTER)
merged = model.merge_and_unload()
assert not hasattr(merged, "peft_config")
merged.save_pretrained(MERGED)
tok.save_pretrained(MERGED)
print("merged into", MERGED)

uv run python merge_lora.py
# Re-run review prompts on the merged model before export.

Worked path: official LLM export

The CLI takes a positional model. That can be a registry short-name, a Hugging Face id, or the local merged folder if the installed recipe accepts a path. Confirm with --help and --dry-run. Name the platform. macOS and iOS presets are not the same.

uv run coreai.llm.export qwen3-0.6b --platform iOS --dry-run
uv run coreai.llm.export artifacts/merged-instruct \
  --platform iOS \
  --output-dir artifacts/writing-style \
  --output-name writing-style \
  --overwrite
# Official short-name form, once the bake is represented the recipe expects:
# uv run coreai.llm.export qwen3-0.6b --platform iOS --output-dir artifacts/writing-style --overwrite

Copy the tokenizer sidecar next to the .aimodel if the recipe did not already write it. Token ids are part of the contract.

Worked path: torch.export plus coreai-torch

Use this when you are converting one exported graph the way the clothing-classifier course does. Confirm example tensors and names against the installed recipe. LLM graphs are larger than ClothingCNN. Start with a documented short-name export when you can.

import torch
from coreai_torch import TorchConverter, get_decomp_table

model = load_merged_instruct().eval()
example = (cpu_input_ids, cpu_attention_mask)
ep = torch.export.export(model, args=example)
ep = ep.run_decompositions(get_decomp_table())
program = TorchConverter().add_exported_program(
    ep, input_names=["input_ids", "attention_mask"], output_names=["logits"]
).to_coreai()
program.optimize()
program.save("artifacts/writing-style/writing-style.aimodel")

Failure modes

• Exporting the PEFT wrapper. Graph capture will see adapter modules the converter did not say it would keep.
• Overwriting the adapter directory. You lose the reversible file.
• Inventing a --model flag. The model argument is positional.
• Shipping macOS quantization to iOS, or the reverse. Run the platform you will load.
• Dropping the tokenizer. The app will tokenize a different language than the weights expect.

Done when

• The merged folder loads without PEFT and matches adapter review samples closely.
• The adapter directory still exists.
• Export used coreai.llm.export or documented torch.export plus coreai-torch.
• The bundle contains the .aimodel and the tokenizer files, with checksums recorded.

Keep learning

PyTorch to Core AI in Xcode · From the metal to the model · Core AI models, typed · Model architectures in plain English · Diffusion LoRA to Core AI on device