← All courses ← Course home

Lesson 02 of 05

Collect signals, not raw chats

Store what failed, not what the user typed, unless they clearly agree to share text.

Agent brief (llms.md)

The default payload is small and boring: feature name, prompt version, profile name, thumbs up or down, which tool ran, whether the guardrail blocked, TTFT bucket, and a reason chip the user tapped (wrong fact, too long, rude).

UIThumbs down
UIThumbs down
MetaFeature and prompt id
UIThumbs down
MetaFeature and prompt id
StoreNo raw transcript
struct FeedbackSignal: Codable {
    var feature: String
    var promptVersion: String
    var profile: String
    var rating: String // up | down
    var reason: String?
    var toolPath: [String]
    var ttftBucket: String
    var createdAt: Date
}

Keep that struct on-device first. If you sync, sync to a server you control, with the same fields. Do not include the prompt body or the model reply in the default sync.

If you need the sentence, ask in clear language, store a yes, and keep the text on the device or on your Mac. App Review and privacy nutrition labels must match what you do. This course will not write a stealth uploader.

Log the prompt version from Version prompts like product copy. A pile of thumbs-down on one id is a rollback candidate before it is a training set.

Rule. Default signal has no user sentences.

Next, turn those signals into training rows you wrote.

Key concepts

  • A signal can hold feature, prompt version, profile, rating, reason, tool path, and TTFT bucket.
  • The default payload has no raw transcript.
  • Store on device first. Optional sync uses the same fields.
  • Text sharing needs clear consent and matching privacy labels.

Takeaways

  • The default signal has no user sentences.
  • A pile of thumbs-down on one prompt version is a rollback candidate first.
  • Do not build a stealth uploader.