# Ship an LLM on Android and iOS with LiteRT-LM: Lesson 04. Catch silent GPU errors with a parity test

Lesson id: `litert-lm-ship-android-ios/04-gpu-parity-harness`
Human lesson: https://edgefde.dev/courses/litert-lm-ship-android-ios/04-gpu-parity-harness.html

## Goal

Build a CPU vs GPU float16 vs GPU float32 parity harness and use it to make a per-SoC GPU allowlist with CPU fallback.

## Prerequisites

- Lesson 03 bundle info and limits
- Two or more real devices with different SoCs

## Inputs

- `golden.jsonl` with prompts and exact checks
- Long prompt files below 1K, near 2K, and above 3K tokens

## Outputs and artifacts

- Kotlin and Swift `runParity` functions
- `parity-.jsonl` files
- `score.py` and `gpu_allowlist.json`
- `chooseBackend` runtime logic

## Steps

1. Write golden cases with `must_contain` and `must_not_contain`, including digit copy, fact at end, and `` checks.
2. Run each case with greedy sampling in a fresh Conversation for CPU, GPU float16 (`ActivationDataType.FLOAT16`), and GPU float32.
3. Record SoC (`Build.SOC_MODEL` on Android, device model on iOS), variant, case, time, and reply as JSONL.
4. Pull the files and run `uv run python score.py parity-*.jsonl`.
5. Allow a GPU variant for a SoC only if it passes every case the CPU passes.
6. At runtime use GPU float16 only for prompts under 2048 tokens on allowlisted SoCs, float32 for longer, CPU otherwise.
7. On a GPU exception, close safely and retry once on CPU.

## Constraints

- Greedy decoding only for parity runs.
- No judge model; checks are exact strings.
- Kotlin on Android, Swift on iOS, Python only for scoring.

## Acceptance checks

- The report prints pass counts per SoC and variant.
- A known bad case (long digit copy on GPU float16) fails on at least one device or is documented as passing.
- Removing a SoC from remote config moves it to CPU without an app update.

## Key concepts

- GPU paths can give wrong text with no error (#3012, #3280, #3577, #3913).
- #3913: float16 cannot hold every position above 2048, so long prompts break on GPU float16. float32 fixes it but costs speed and memory.
- A golden set with exact checks, run with greedy decoding on CPU, GPU float16, and GPU float32, finds these errors.
- An allowlist per SoC decides where the GPU is allowed. CPU is the default.

## Takeaways

- Never turn on GPU for all devices. Earn it per SoC with the parity report.
- Send long prompts to float32 or CPU, even on allowlisted devices.
- Ship the allowlist as remote config and fall back to CPU on GPU errors.