# Keep On-Device LLMs Alive on Real Phones: Lesson 05. Keep a Mac MLX box stable

Lesson id: `on-device-llm-survival/05-mac-mlx-wired-memory`
Human lesson: https://edgefde.dev/courses/on-device-llm-survival/05-mac-mlx-wired-memory.html

## Goal

Run an MLX service on a Mac with a set wired limit, a cache limit, a prompt cap, one process, and real footprint logging.

## Prerequisites

- Apple silicon Mac on macOS 15+
- uv project with `mlx` and `mlx-lm`

## Inputs

- Model id
- Your chosen prompt cap

## Outputs and artifacts

- `configure_mlx_memory()`
- Prompt cap check
- Per-request memory log
- `footprint` sample log

## Steps

1. Run `uv run python -c "import mlx.core as mx; print(mx.device_info())"` and record `memory_size` and `max_recommended_working_set_size`.
2. At service start, call `mx.set_wired_limit(int(working_set * 0.85))` and `mx.set_cache_limit(...)`. Log the old and new values.
3. Before each request, count prompt tokens with the tokenizer and reject prompts over the cap.
4. Run a single MLX process. Do not call `mx.clear_cache()` from another thread during generation.
5. After each request, log `get_active_memory()`, `get_cache_memory()`, and `get_peak_memory()`.
6. Every few minutes, record `footprint ` and watch `vm_stat` for swap growth.

## Constraints

- Do not say the wired limit prevents the IOGPU panic.
- Do not raise `iogpu.wired_limit_mb` on shared machines without long tests.

## Acceptance checks

- Service log shows the wired limit at start.
- A prompt over the cap returns a clear error.
- A two-hour run shows flat active plus cache memory.

## Key concepts

- `mx.device_info()` gives `memory_size` and `max_recommended_working_set_size`.
- `mx.set_wired_limit` sets how much MLX keeps resident. It must stay below total memory and the system wired limit.
- The IOGPU kernel panic in mlx #3186 is an OS-level bug. Settings lower risk but are not a proven fix.
- `get_peak_memory()` can under-report. Use active plus cache memory, and the OS `footprint` tool.

## Takeaways

- Set a wired limit and a cache limit at service start, and log them.
- Cap prompt tokens so one request cannot start a giant prefill.
- Run one MLX process per Mac and check the real footprint from outside.