Train a LoRA on a Mac

Why you train on a Mac

Your writing is private. Train it on the Mac that already has the files. MPS on Apple silicon is the preferred device. CPU is the fallback. There is no CUDA step. Slower jobs are fine. Leaking private mail is not acceptable.

The adapter is a middle step. Core AI will not load it. Lesson 05 merges and exports.

The steps, one box at a time

Environment setup

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p writing-style-lora && cd writing-style-lora
uv init --python 3.11
uv venv
uv add torch torchvision transformers peft datasets accelerate safetensors coreai-torch
uv run python -c "import torch; print(torch.__version__); print('MPS:', torch.backends.mps.is_available())"

Worked path

Save as train_lora.py. Use the same chat template you recorded in lesson 03. Rank 8 and a short step count are enough to prove the loop. Raise them only after samples justify it.

import json
from pathlib import Path
import torch
from peft import LoraConfig, get_peft_model
from torch.utils.data import DataLoader, Dataset
from transformers import AutoModelForCausalLM, AutoTokenizer

DEVICE = torch.device("mps" if torch.backends.mps.is_available() else "cpu")
print("training on", DEVICE)
NAME = "Qwen/Qwen3-0.6B"
REVISION = "pin-this"
OUT = Path("artifacts/writing-style-lora")
OUT.mkdir(parents=True, exist_ok=True)

tok = AutoTokenizer.from_pretrained(NAME, revision=REVISION)
if tok.pad_token is None:
    tok.pad_token = tok.eos_token
model = AutoModelForCausalLM.from_pretrained(NAME, revision=REVISION)
model.to(DEVICE)
model.gradient_checkpointing_enable()
model = get_peft_model(model, LoraConfig(r=8, lora_alpha=16, lora_dropout=0.05, target_modules=["q_proj", "v_proj"]))
model.print_trainable_parameters()

class JsonlChat(Dataset):
    def __init__(self, path):
        self.rows = [json.loads(line) for line in Path(path).read_text(encoding="utf-8").splitlines() if line.strip()]

    def __len__(self):
        return len(self.rows)

    def __getitem__(self, i):
        row = self.rows[i]
        user = row["instruction"]
        if row.get("input"):
            user = f"{user}\n{row['input']}"
        text = tok.apply_chat_template(
            [{"role": "user", "content": user}, {"role": "assistant", "content": row["response"]}],
            tokenize=False,
        )
        encoded = tok(text, truncation=True, max_length=512, padding="max_length", return_tensors="pt")
        item = {k: v.squeeze(0) for k, v in encoded.items()}
        item["labels"] = item["input_ids"].clone()
        return item

loader = DataLoader(JsonlChat("data/clean/train.jsonl"), batch_size=1, shuffle=True)
optimizer = torch.optim.AdamW((p for p in model.parameters() if p.requires_grad), lr=1e-4)
model.train()
step = 0
for batch in loader:
    step += 1
    batch = {k: v.to(DEVICE) for k, v in batch.items()}
    loss = model(**batch).loss
    loss.backward()
    optimizer.step()
    optimizer.zero_grad()
    print(f"step {step} loss={loss.item():.4f}")
    if step >= 100:
        break

model.save_pretrained(OUT)
tok.save_pretrained(OUT)
(OUT / "manifest.json").write_text(json.dumps({
    "base": NAME,
    "revision": REVISION,
    "device": str(DEVICE),
    "rank": 8,
    "steps": step,
    "runtime_lora_swap": False,
}, indent=2), encoding="utf-8")
print("wrote", OUT)

uv run python train_lora.py
ls artifacts/writing-style-lora

Generate the same review prompts from the base and from the adapter. Check voice, whether it follows the request, whether the meaning stayed true, and whether a training passage was copied. A style gain that breaks helpfulness is a failed checkpoint.

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

device = "mps" if torch.backends.mps.is_available() else "cpu"
name, revision = "Qwen/Qwen3-0.6B", "pin-this"
tok = AutoTokenizer.from_pretrained(name, revision=revision)
base = AutoModelForCausalLM.from_pretrained(name, revision=revision).to(device)
adapted = PeftModel.from_pretrained(base, "artifacts/writing-style-lora")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Rewrite this Slack message so it sounds like me: circle back tomorrow on the numbers"}],
    tokenize=False,
    add_generation_prompt=True,
)
ids = tok(prompt, return_tensors="pt").to(device)
print(tok.decode(adapted.generate(**ids, max_new_tokens=80)[0], skip_special_tokens=True))

Failure modes

• CUDA in the script. Remove it.
• Wrong target modules. Confirm projection names for the chosen architecture before you assume q_proj.
• Training without the chat template. The app will not match the loss you optimized.
• Calling the adapter the shippable model. Merge first. Core AI has no documented LoRA slot.
• Memorized private sentences. Drop those rows and train again before export.

Done when

• artifacts/writing-style-lora has adapter weights, tokenizer files, and a manifest with MPS or CPU.
• Review prompts were run on base and adapter.
• You have not described the adapter as the Core AI artifact.

Keep learning

PyTorch to Core AI in Xcode · From the metal to the model · Core AI models, typed · Model architectures in plain English · Diffusion LoRA to Core AI on device