# Ship an LLM on Android and iOS with LiteRT-LM: Lesson 03. Match maxNumTokens to the model

Lesson id: `litert-lm-ship-android-ios/03-context-size-and-kv-cache`
Human lesson: https://edgefde.dev/courses/litert-lm-ship-android-ios/03-context-size-and-kv-cache.html

## Goal

Size the context to what the bundle was built for and add tests for crashes near the limit and silent prompt loss.

## Prerequisites

- Know how your `.litertlm` bundle was exported
- Lesson 01 safe Conversation wrapper

## Inputs

- Bundle path and built `cache_length`
- Target backends (CPU, GPU, NPU)

## Outputs and artifacts

- `engineConfig(bundle, backend)`
- `mustRollOver` in Kotlin and Swift
- Near-limit chat test and fact-at-the-end prompt

## Steps

1. Record the export's `--cache_length` for each bundle in a `BundleInfo`.
2. Create `EngineConfig` with `maxNumTokens` equal to that value.
3. Track used tokens (`getTokenCount()` on iOS) and roll over before 90% including the reply budget.
4. Write a multi-turn test that gets within 10% of the limit on each backend.
5. Write a prompt with 300 tokens of filler then the key fact, and check the answer on CPU and NPU.
6. For Gemma 3 on a Qualcomm NPU, use a cache_length 896 build or switch to Gemma 4 or CPU/GPU.

## Constraints

- Never set `maxNumTokens` from the model card.
- Do not claim 896 is safe for every model.

## Acceptance checks

- Near-limit test passes on every shipped backend.
- Fact-at-the-end prompt is answered correctly on every shipped backend.

## Key concepts

- `maxNumTokens` is the KV cache size for input plus output.
- The model card context length is not the bundle's built cache length.
- Too high can crash or return nothing (LiteRT #6765). Close to the limit can crash (#2799).
- #3508: some Gemma 3 NPU bundles drop prompt chunks after the first, with no error. The safe cache length is per model.

## Takeaways

- Set `maxNumTokens` from the export, and roll the chat over before 90% of it.
- Prefer Gemma 4 on NPUs, or Gemma 3 on CPU or GPU.
- Add a fact-at-the-end prompt to your tests to catch silent prompt loss.