Turn signals into synthetic data

A thumbs-down on "wrong fact" plus a feature name is enough to write three new prompt and target pairs. You write them. A second person reviews them. That is the seed set.

The Evaluations framework can generate more ModelSample rows, including trajectory expectations, as shown in WWDC26 session 299. Use that to grow evals and, if you choose, to draft training paraphrases. Keep the rejects. Do not train on a row nobody read.

Synthetic data will miss slang and languages you did not write. Say that out loud. It is still safer than uploading a week of chats to a public lab.

These rows feed two places: the eval harness in Test AI behavior before you ship, and the LoRA JSONL you will train on a Mac. Same facts. Two files if the formats differ. Evaluations SampleGenerator stays in Swift tests. The training file is Python because PEFT reads JSONL on the Mac.

import json
from pathlib import Path

rows = [
    {
        "instruction": "What is the refund window for order 1842?",
        "input": "",
        "response": "Order 1842 can be refunded within 30 days of delivery.",
        "reviewed": True,
        "signal": "wrong-fact",
        "feature": "support-refunds",
    }
]

out = Path("data/clean/train.jsonl")
out.parent.mkdir(parents=True, exist_ok=True)
with out.open("w", encoding="utf-8") as handle:
    for row in rows:
        if not row.get("reviewed"):
            raise SystemExit("unread row")
        handle.write(json.dumps(row, ensure_ascii=False) + "\n")
print("wrote", out, "rows", len(rows))

Next, train and bake.