Prepare your writing data

Why cleaning your writing comes first

Privacy starts before training. If a token, an address, or another person's message sits in the JSONL, it can sit in the baked model. Read the formatted set the way you would read a production config. Your writing collection (the corpus) never needs a network.

A small, focused set beats a giant dump. You are teaching rhythm and structure. You are not putting a large public scrape onto a phone.

The steps, one box at a time

Collect writing you own or have permission to use.

Strip secrets, tokens, and other people's private detail.

Format every row the same way. Use instruction and response, or completion with a stable delimiter.

Hold out validation rows and a short list of human review prompts the trainer will never see.

Environment setup

Create the project before the first formatting command.

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir -p writing-style-lora && cd writing-style-lora
uv init --python 3.11
uv venv
uv add torch torchvision transformers peft datasets accelerate safetensors coreai-torch
uv run python -c "import torch; print(torch.__version__); print('MPS:', torch.backends.mps.is_available())"
mkdir -p data/raw data/clean artifacts

Worked path

Keep raw files in data/raw. Do not point the trainer at that folder. Write cleaned JSONL yourself or with a small script you can read.

{
  "instruction": "Rewrite this note clearly.",
  "input": "hey can you send the deck when you get a chance thx",
  "response": "Please send the deck when you have a moment. Thank you."
}

One object per line in data/clean/all.jsonl. Then split.

import json
from pathlib import Path
from random import Random

rows = [json.loads(line) for line in Path("data/clean/all.jsonl").read_text(encoding="utf-8").splitlines() if line.strip()]
rng = Random(0)
rng.shuffle(rows)
cut = max(1, int(len(rows) * 0.1))
val, train = rows[:cut], rows[cut:]
Path("data/clean/train.jsonl").write_text("\n".join(json.dumps(r, ensure_ascii=False) for r in train) + "\n", encoding="utf-8")
Path("data/clean/val.jsonl").write_text("\n".join(json.dumps(r, ensure_ascii=False) for r in val) + "\n", encoding="utf-8")
print(len(train), "train", len(val), "val")

uv run python split_corpus.py
# Inspect after formatting. JSON fields can still hold secrets.
rg -n "sk-|AKIA|password|ssn|BEGIN PRIVATE" data/clean || true

Write human review prompts that are not in the JSONL. You will reuse them against the base, the adapter, and the merged export.

{
  "review_prompts": [
    "Rewrite this Slack message so it sounds like me: 'circle back tomorrow on the numbers'",
    "Turn these bullets into a short update I would send.",
    "Say no to a meeting without sounding curt."
  ],
  "forbidden_in_corpus": ["api keys", "other people's medical notes", "unpublished credentials"]
}

Failure modes

• Training on raw mailboxes. Secrets and third-party text will travel into the bake.
• Near-duplicate train and val rows. Validation will reward memorization.
• Inconsistent fields. Mixed response and output keys silently drop text.
• No review prompts outside the set. You will only test sentences the model has already seen.

Done when

• data/clean/train.jsonl and data/clean/val.jsonl exist, with one schema.
• A secret scan of data/clean is clean enough to ship into a local artifact.
• Review prompts are written down and absent from the JSONL.
• Your writing has not left the machine.

Keep learning

PyTorch to Core AI in Xcode · From the metal to the model · Core AI models, typed · Model architectures in plain English · Diffusion LoRA to Core AI on device