# Run One Large LLM Across Several Macs with MLX: Lesson 01. Pick pipeline or tensor, size memory

Lesson id: `mlx-distributed-mac-cluster/01-pipeline-or-tensor-and-memory`
Human lesson: https://edgefde.dev/courses/mlx-distributed-mac-cluster/01-pipeline-or-tensor-and-memory.html

## Goal

Decide pipeline or tensor parallel for a model, confirm the model supports it, and confirm every Mac can hold its share.

## Prerequisites

- Two or more Apple silicon Macs, macOS 26.2+
- uv project `~/mlx-cluster` with mlx, mlx-lm, huggingface_hub on every Mac

## Inputs

- Hugging Face model id
- Each Mac's `max_recommended_working_set_size` in hostfile order

## Outputs and artifacts

- `check_split_support.py` output
- `size_shards.py` output
- A written choice: tensor or pipeline, with the reason

## Steps

1. On each Mac run `uv run python -c "import mlx.core as mx; print(mx.device_info()['max_recommended_working_set_size'] / 2**30)"` and record the GiB value.
2. Run `uv run python check_split_support.py <model>` and record tensor and pipeline support.
3. Run `uv run python size_shards.py --model <model> --nodes N --working-set-gib <values in hostfile order>`.
4. If every rank says OK for tensor and you will have an RDMA mesh, choose tensor. Otherwise choose pipeline if the model supports it.
5. If any rank says TOO BIG, use a smaller quantization, add a Mac, or reorder hosts so the larger Mac gets more layers in pipeline mode.

## Constraints

- Do not size by combined memory.
- Treat the 25% headroom as a starting point, not an Apple number.
- Do not claim tensor is faster without a JACCL link.

## Acceptance checks

- Written choice names the split and the model support result.
- Every rank shows OK in the chosen split.

## Key concepts

- Pipeline splits a model by depth. Tensor splits every layer by width.
- Pipeline mainly makes a model fit. Tensor makes it faster, but needs a low-latency link.
- mlx-lm uses tensor by default and pipeline with `--pipeline`. Each model supports one, both, or neither.
- Pipeline splits by layer count and tensor splits evenly, so a Mac with less memory can still be overloaded.

## Takeaways

- Run `check_split_support.py` before you plan a cluster around a model.
- Run `size_shards.py` with each Mac's real working set, and keep headroom for the KV cache.
- Do not trust combined memory. Check the share on the smallest Mac.