Ship an LLM on Android and iOS with LiteRT-LM

Ship a LiteRT-LM chat feature that passes release builds and does not give users wrong answers with no error.

Conversation lifecycle, R8 keep rules, maxNumTokens vs the baked KV cache, NPU prefill loss, a CPU vs GPU parity harness, and an iOS release checklist.

Close Conversations safely, keep R8 from breaking JNI, size the context to the model file, catch silent NPU and GPU errors with a golden test, and pass the iOS release checks.

Who this is for

You have a LiteRT-LM model that answers on your test phone. Now you want to ship it to many phones. This course collects the problems that show up after that point, from real LiteRT-LM GitHub issues, and turns each one into a check you can run. Every issue link was read on 15 Oct 2026. Many are still open, so check them again before you ship.

If you are new to LiteRT, start with PyTorch to Android with LiteRT. Android code here is Kotlin. iOS code is Swift.

Lessons

Sources we checked

LiteRT-LM issues: #3771 (Conversation closed during prefill), #3739 (no consumer ProGuard rules), #2799 and LiteRT #6765 (crashes near or above the token limit), #3508 (NPU prefill chunks lost)
GPU output issues: #3012 (Adreno 750), #3280 (endless <pad>), #3577 (MiniCPM on Adreno), #3913 (iOS GPU float16 at long context), #2421 (CL_INVALID_COMMAND_QUEUE on Mali)
iOS release: #3863 (privacy manifest), #3864 (duplicate Objective-C classes), #2545 (contiguous memory map)
LiteRT-LM source at v0.18.0 and main: Conversation.kt, Config.kt, Config.swift
Apple: Privacy manifest files

Key concepts

Cancel, wait until the Conversation is idle, then close. Closing during prefill can crash in native code.
R8 can remove Kotlin members that native code looks up by name. Keep rules fix it.
maxNumTokens is the KV cache size. It must fit what the model file was built for.
GPU and NPU paths can give wrong text with no error. Only a comparison against CPU output catches it.
iOS release needs a privacy manifest for the library's required-reason APIs, and a check for duplicate classes and memory map failures.

Takeaways

You can ship a Stop button that does not crash the app.
You can build a minified release that still runs local generation.
You can pick a backend per device from test results, and fall back to CPU when the GPU disagrees.
You can walk an iOS build through a checklist before App Store review.

Agent briefs

FAQ

Why does LiteRT-LM crash when I close a Conversation?

Issue 3771: closing during prefill can crash because the prefill callback uses a session that was already reset. Call cancelProcess(), wait for onDone or onError, then close the Conversation, then the Engine.

Why does my LiteRT-LM release build crash with mid == null?

Issue 3739: R8 removes Kotlin members that the native library looks up by name. Add -keep class com.google.ai.edge.litertlm.** { *; } to proguard-rules.pro and test the minified build on a device.

What should I set maxNumTokens to?

maxNumTokens is the KV cache size for input plus output. Use the cache length the bundle was exported with, not the model card context length, and roll the chat over before you reach it.

Why does LiteRT-LM give wrong answers on GPU but not on CPU?

Several open issues show GPU paths producing wrong text with no error, often on long prompts with float16 activations. Compare CPU, GPU float16, and GPU float32 on a golden set and allow GPU per SoC.

Does LiteRT-LM need a privacy manifest on iOS?

Issue 3863 found required-reason API use and no PrivacyInfo.xcprivacy in v0.17.1 binaries. A fix merged after v0.18.0. Scan your archive and declare any categories the binaries use.