Fit the model in the memory budget

curl -LsSf https://astral.sh/uv/install.sh | sh
mkdir llm-survival && cd llm-survival
uv init --python 3.11
uv venv
# PyTorch: on Apple silicon the default wheel includes MPS; on other machines it runs on CPU.
uv add torch
uv add mlx mlx-lm
uv run python -c "import torch; print('mps' if torch.backends.mps.is_available() else 'cpu')"
uv run python -c "import mlx.core as mx; print(mx.device_info())"

Why the app just disappears

On iOS, when memory runs low, the system first sends a memory warning. If memory is still too low, it ends apps to get memory back. Apple calls this a jetsam event. A jetsam kill has no crash backtrace. To the user, the app just closes. Apple explains how to read these reports in Identifying high-memory use with jetsam event reports. Look for "reason" : "per-process-limit" on your process. That means your app went over its own limit.

That limit is not the device RAM. It depends on the device model and can change while the app runs. A 3 GB model can fit on one phone and get killed on another phone with the same RAM size.

Ask how much is left

os_proc_available_memory() returns the number of bytes your app can still allocate before it hits its limit. Apple says to treat the value as advice only, and not to cache it, because it changes all the time. Call it right before each load.

import os
import Foundation

struct ModelSpec {
    let name: String
    /// Measured on a real device: footprint after load and one long prompt.
    /// Do not use the file size. Runtime memory includes the KV cache and work buffers.
    let measuredFootprintBytes: UInt64
}

enum MemoryDecision: Equatable {
    case load
    case tooBig(availableBytes: UInt64, neededBytes: UInt64)
}

struct MemoryGate {
    /// Share of the *available* memory we allow one model to use.
    /// This is a starting point for this course, not an Apple number.
    /// Tune it from your own jetsam reports.
    var maxShareOfAvailable: Double = 0.6

    func decide(for spec: ModelSpec) -> MemoryDecision {
        let available = UInt64(os_proc_available_memory())
        let allowed = UInt64(Double(available) * maxShareOfAvailable)
        if spec.measuredFootprintBytes <= allowed {
            return .load
        }
        return .tooBig(availableBytes: available, neededBytes: spec.measuredFootprintBytes)
    }
}

Why 60%? You need room for your UI, images, the system, and memory that grows while the model runs. Apple does not publish a safe share. We start at 60% of what is available now, then move it up or down after we read real jetsam reports from the field. Write the number you pick in your code and in your release notes, so the team knows it is a choice.

The two memory entitlements

Apple has two entitlements that matter for large models. They do different jobs.

The second case is real. In LiteRT-LM #2545, a Gemma 4 model section of about 1.28 GB could not get one contiguous memory map inside a large host app. The same code worked in a small sample app. The engine still reported "ready", and the first inference failed. A commenter pointed to the extended virtual addressing entitlement, but the reporter had not confirmed that it fixed the problem, and the issue was closed as stale. So treat the entitlement as something to test, not a proven fix.

Add both in Xcode: target > Signing & Capabilities > + Capability > "Increased Memory Limit" and "Extended Virtual Addressing". Then call the memory gate again. The entitlement can raise the number that os_proc_available_memory() returns, so your gate picks it up with no other change.

Do something useful when the answer is no

func prepareAssistant(gate: MemoryGate, big: ModelSpec, small: ModelSpec) -> String {
    switch gate.decide(for: big) {
    case .load:
        return big.name
    case .tooBig(let available, let needed):
        print("memory: big model needs \(needed) B, available \(available) B")
        if gate.decide(for: small) == .load {
            return small.name
        }
        // No local model fits right now. Use your server path or show a clear message.
        return "server"
    }
}

Log both numbers every time. Those logs tell you which devices need the smaller model. If you already route between Apple models, see On-Device Model Router (Apple FM) for the routing side.

Sources

• Apple: os_proc_available_memory
• Apple: Identifying high-memory use with jetsam event reports
• Apple: Increased Memory Limit entitlement