# Core AI Fast First Token and the Right Compute Unit: Lesson 04. Catch silent Neural Engine fallback

Lesson id: `core-ai-fast-first-token/04-catch-silent-compute-fallback`
Human lesson: https://edgefde.dev/courses/core-ai-fast-first-token/04-catch-silent-compute-fallback.html

## Goal

Detect when a Core AI model meant for the Neural Engine does not run there, or runs there with wrong results.

## Prerequisites

- coreai-build with the Metal Toolchain
- A small test input set and a CPU reference

## Inputs

- `MyModel.aimodel`
- Test inputs as NDArray
- Threshold per model type

## Outputs and artifacts

- `check-ane.sh` CI script
- Instruments trace note per export
- Per-compute-unit comparison report

## Steps

1. Compile with `--preferred-compute neural-engine` and save stderr to a log.
2. Fail if the log has `ANECCompileOffline() failed` or a bundle has zero `*ANE_region*` files.
3. Record one inference in the Core AI Instruments template and confirm the compute units.
4. Run the same inputs with `SpecializationOptions(preferredComputeUnitKind: .neuralEngine)` and `.cpuOnly`.
5. Compare outputs (top-1, cosine, or greedy text) against the written threshold.
6. If they differ, open a Core AI Debugger comparison session against a reference run to find the operation.

## Constraints

- Treat the ANE_region count as a heuristic from coreai-models #205, not an Apple API.
- Swift for app code, bash for CI.

## Acceptance checks

- CI fails on a GPU-only bundle.
- The comparison report exists for every shipped model and compute unit.

## Key concepts

- #205: a Neural Engine compile can fail while coreai-build exits 0 and writes a GPU-only .aimodelc.
- #279: a model can run on the Neural Engine and give wrong numbers, with no warning.
- `preferredComputeUnitKind` is a preference. Incompatible operations fall back to other units.
- Three checks: Neural Engine regions in the build, compute units in the trace, outputs per compute unit.

## Takeaways

- Fail CI when a model meant for the Neural Engine has zero Neural Engine regions.
- Read the Instruments trace for every new export and OS version.
- Compare outputs across compute units with a written threshold before you ship.