--- name: litert-lm-ship-android-ios description: >- Ship a LiteRT-LM LLM feature on Android (Kotlin) and iOS (Swift): safe Conversation close, R8 keep rules, maxNumTokens sized to the bundle, NPU and GPU silent-error checks with a CPU golden parity harness and per-SoC allowlist, and an iOS release checklist. --- # Ship an LLM on Android and iOS with LiteRT-LM Ship a LiteRT-LM chat feature that passes release builds and does not give users wrong answers with no error. Course home: https://edgefde.dev/courses/litert-lm-ship-android-ios/ ## When to use Use when a LiteRT-LM app moves from one test phone to a release: crashes on Stop, release-only SIGABRT, crashes near the token limit, wrong answers on GPU or NPU with no error, or App Store privacy manifest warnings. ## Environment LiteRT-LM 0.18.0. Android Studio, Kotlin, `com.google.ai.edge.litertlm:litertlm-android:0.18.0`, real arm64 phones. Xcode 27, iOS 27 device, `LiteRTLM` Swift package. Python 3.11+ with uv: `uv init --python 3.11`, `uv venv`, `uv add torch litert-lm`, `uv run ...`. ## Recipe 1. **Close a Conversation safely.** Make Stop and screen close safe by cancelling, waiting for idle, and only then closing the Conversation and Engine. Brief: `01-conversation-lifecycle.llms.md`. 2. **Make release builds work.** Make a minified Android release build run LiteRT-LM generation without a JNI abort. Brief: `02-release-builds-and-proguard.llms.md`. 3. **Match maxNumTokens to the model.** Size the context to what the bundle was built for and add tests for crashes near the limit and silent prompt loss. Brief: `03-context-size-and-kv-cache.llms.md`. 4. **Catch silent GPU errors with a parity test.** Build a CPU vs GPU float16 vs GPU float32 parity harness and use it to make a per-SoC GPU allowlist with CPU fallback. Brief: `04-gpu-parity-harness.llms.md`. 5. **Pass the iOS release checklist.** Run a repeatable iOS release checklist for a LiteRT-LM app. Brief: `05-ios-release-checklist.llms.md`. ## Acceptance checks - Pressing Stop during a long prompt, then leaving the screen, never crashes (100 tries on device). - A minified release build generates a reply on device. - `maxNumTokens` is set from the bundle's own value and a test runs a prompt near that limit. - The parity report shows CPU, GPU float16, and GPU float32 results per device, and the app only uses GPU on allowlisted SoCs. - The iOS archive has a privacy manifest entry for every required-reason category the LiteRT-LM binaries use. ## Key concepts - Cancel, wait until the Conversation is idle, then close. Closing during prefill can crash in native code. - R8 can remove Kotlin members that native code looks up by name. Keep rules fix it. - `maxNumTokens` is the KV cache size. It must fit what the model file was built for. - GPU and NPU paths can give wrong text with no error. Only a comparison against CPU output catches it. - iOS release needs a privacy manifest for the library's required-reason APIs, and a check for duplicate classes and memory map failures. ## Takeaways - You can ship a Stop button that does not crash the app. - You can build a minified release that still runs local generation. - You can pick a backend per device from test results, and fall back to CPU when the GPU disagrees. - You can walk an iOS build through a checklist before App Store review. ## Sources - LiteRT-LM issues: [#3771](https://github.com/google-ai-edge/LiteRT-LM/issues/3771) (Conversation closed during prefill), [#3739](https://github.com/google-ai-edge/LiteRT-LM/issues/3739) (no consumer ProGuard rules), [#2799](https://github.com/google-ai-edge/LiteRT-LM/issues/2799) and [LiteRT #6765](https://github.com/google-ai-edge/LiteRT/issues/6765) (crashes near or above the token limit), [#3508](https://github.com/google-ai-edge/LiteRT-LM/issues/3508) (NPU prefill chunks lost) - GPU output issues: [#3012](https://github.com/google-ai-edge/LiteRT-LM/issues/3012) (Adreno 750), [#3280](https://github.com/google-ai-edge/LiteRT-LM/issues/3280) (endless ), [#3577](https://github.com/google-ai-edge/LiteRT-LM/issues/3577) (MiniCPM on Adreno), [#3913](https://github.com/google-ai-edge/LiteRT-LM/issues/3913) (iOS GPU float16 at long context), [#2421](https://github.com/google-ai-edge/LiteRT-LM/issues/2421) (`CL_INVALID_COMMAND_QUEUE` on Mali) - iOS release: [#3863](https://github.com/google-ai-edge/LiteRT-LM/issues/3863) (privacy manifest), [#3864](https://github.com/google-ai-edge/LiteRT-LM/issues/3864) (duplicate Objective-C classes), [#2545](https://github.com/google-ai-edge/LiteRT-LM/issues/2545) (contiguous memory map) - LiteRT-LM source at v0.18.0 and main: [Conversation.kt](https://github.com/google-ai-edge/LiteRT-LM/blob/main/kotlin/java/com/google/ai/edge/litertlm/Conversation.kt), [Config.kt](https://github.com/google-ai-edge/LiteRT-LM/blob/main/kotlin/java/com/google/ai/edge/litertlm/Config.kt), [Config.swift](https://github.com/google-ai-edge/LiteRT-LM/blob/main/swift/Config.swift) - Apple: [Privacy manifest files](https://developer.apple.com/documentation/bundleresources/privacy-manifest-files)