Pick a base LLM

Why the base model has to fit

You cannot personalize a model you cannot bake. A large 30B checkpoint that will not fit the phone is not a device model. Start from a catalog instruct preset that already has a Core AI recipe. Then adapt it locally so the device never needs your writing.

macOS and iOS presets differ. Compression, context, and embedding quantization (how numbers are stored) are not the same across platforms. Pick the platform first.

The steps, one box at a time

Start in the official list.

Check whether the context and memory budget match the chat the app needs.

Confirm the export recipe for that short-name and platform exists on the tools you installed.

Environment setup

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p writing-style-lora && cd writing-style-lora
uv init --python 3.11
uv venv
uv add torch torchvision transformers peft datasets accelerate safetensors coreai-torch
uv run python -c "import torch; print(torch.__version__); print('MPS:', torch.backends.mps.is_available())"
uv run coreai.llm.export --list-models
uv run coreai.llm.export qwen3-0.6b --platform iOS --dry-run

Worked path

Course examples are qwen3-0.6b and the small smollm2 instruct presets listed in Core AI models, typed. Treat them as starting points. Confirm the current short-name, Hugging Face id, and platform variant before you download.

{
  "base_short_name": "qwen3-0.6b",
  "base_hf_id": "Qwen/Qwen3-0.6B",
  "base_revision": "pin-this",
  "platform": "iOS",
  "max_context_length": "from-preset",
  "tokenizer": "same revision as the base",
  "instruction_format": "the model's chat template",
  "runtime_lora_swap": false
}

Run a few base samples before you train. If the untouched model cannot follow a short rewrite request, a LoRA will not fix that. Record the chat template. Training must use the same template the export and the app will use.

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

name = "Qwen/Qwen3-0.6B"
revision = "pin-this"
device = "mps" if torch.backends.mps.is_available() else "cpu"
tok = AutoTokenizer.from_pretrained(name, revision=revision)
model = AutoModelForCausalLM.from_pretrained(name, revision=revision).to(device)
messages = [{"role": "user", "content": "Rewrite clearly: circle back tomorrow on the numbers"}]
prompt = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
ids = tok(prompt, return_tensors="pt").to(device)
out = model.generate(**ids, max_new_tokens=64)
print(tok.decode(out[0], skip_special_tokens=True))

A smaller model that fits is more useful than a larger model that swaps to disk. Context length is prompt plus response. Leave room for the reply. Test the longest conversation the app needs, not the longest the card claims.

Failure modes

• Picking a base with no registry preset. Raw Hugging Face ids need --experimental and --compute-precision. That is not the first-run path.
• Ignoring platform. An iOS palettization preset (a compression recipe) will not apply on macOS, and the reverse is also true.
• Training with a different chat template than export. The baked model will look untrained in the app.
• Promising a context the device cannot hold. Measure the target, then set --max-context-length if the recipe allows it.

Done when

• The contract names a catalog short-name, revision, tokenizer, platform, and context budget.
• coreai.llm.export --dry-run resolves that short-name on the chosen platform.
• A base sample exists for a review prompt.
• The contract still forbids a runtime LoRA swap.

Keep learning

PyTorch to Core AI in Xcode · From the metal to the model · Core AI models, typed · Model architectures in plain English · Diffusion LoRA to Core AI on device