Keep On-Device LLMs Alive on Real Phones

Keep your local AI feature running after launch day, so users do not see the app vanish, slow to a crawl, or lose a half-downloaded model.

Memory budgets, one-at-a-time model loads, thermal policy, resumable downloads, process-death recovery, and MLX wired limits on a Mac.

Check memory before you load, load one model at a time, react to heat, resume downloads, recover from process death, and keep a Mac MLX box inside its GPU memory limit.

Who this is for

You already run a model on a phone or Mac, and it works in a demo. Now it crashes for some users, gets slow after a few minutes, or loses its download. This course is about those field problems. It does not teach how to convert a model. For that, see PyTorch to Core AI in Xcode or PyTorch to Android with LiteRT.

iOS code is Swift. Android code is Kotlin. The Mac lesson uses Python with MLX.

Lessons

Sources we checked

Apple: os_proc_available_memory(), Increased Memory Limit entitlement, Extended Virtual Addressing entitlement, jetsam event reports
Apple: ProcessInfo.thermalState, URLSession downloadTask(withResumeData:), didReceiveMemoryWarningNotification
Android: ADPF Thermal API
LiteRT-LM #2545: a large model section could not get one contiguous memory map inside a big iOS app, the engine still said ready, then inference failed
MLX: mx.set_wired_limit, mlx #3186 (IOGPU kernel panic during long prefill), mlx #3896 (peak memory counter under-reports the real footprint)

Key concepts

The limit that kills your app is the per-app memory limit, not the phone's RAM. Ask the OS how much is left before you load.
Load, warm up, and unload models through one owner, one at a time.
Heat lowers speed over minutes. Design a short burst mode and a slower sustained mode.
A model download must survive app kills, network loss, and reboots, and must be checked before use.
On a Mac, keep MLX inside the GPU working set and cap context length. Kernel panics in IOGPU are an OS bug you can only avoid, not fix.

Takeaways

You can refuse a model load with a clear message instead of a silent jetsam kill.
You can fall back to a smaller model or a server path when memory or heat says no.
You can resume a 2 GB download instead of starting again, and you never load a half-written file.
You can run a Mac MLX service for hours with a set wired limit, a context cap, and a real footprint check.

Agent briefs

FAQ

Why does my iOS app close with no crash log when the model loads?

That is usually a jetsam event: iOS ended the app to get memory back. Check the jetsam event report for reason per-process-limit, and gate each load on os_proc_available_memory() and the model's measured footprint.

Does the increased memory limit entitlement fix out-of-memory kills?

It can raise the limit, but Apple says it is only available on some device models and your app must still work without it. Keep a memory check and a smaller fallback model.

How do I detect thermal throttling on iOS and Android?

On iOS read ProcessInfo.thermalState and observe thermalStateDidChangeNotification. On Android 11+ read PowerManager.getThermalHeadroom no more than once every 10 seconds.

How do I resume a large model download?

On iOS use a background URLSession and downloadTask(withResumeData:). On Android use WorkManager with an HTTP Range request. Check a SHA-256 checksum before you load the file.

Does mx.set_wired_limit stop MLX kernel panics on a Mac?

No setting is a proven fix. The IOGPU panic in mlx issue 3186 is in the OS. A wired limit below the recommended working set, a prompt cap, and one MLX process lower the risk.