---
name: On-Device Search with EmbeddingGemma 2
description: >-
  Build on-device embedding search with EmbeddingGemma 2 that does not fail quietly: model card prefixes, bfloat16 or float32, renormalized truncation, one setup per index, and a recall test.
---
# On-Device Search with EmbeddingGemma 2

Use this course when on-device search or RAG gives weak results with no error. The scripts run against EmbeddingGemma 2 served by llama-server on port 8094. The same rules apply with LiteRT-LM on iOS and Android.

## End-to-end agent recipe

1. Use [lesson 01](01-use-the-right-prefixes.llms.md). Serve EmbeddingGemma 2 with llama-server `--embedding`, and apply the model card prefixes for queries and documents.
2. Use [lesson 02](02-number-format-and-nan.llms.md). Run EmbeddingGemma 2 in bfloat16 or float32 and pass every vector through `check_vectors` before storing it.
3. Use [lesson 03](03-truncate-and-keep-one-model.llms.md). Truncate with renormalization and store the setup with the index using `index_guard.py`, refusing to load a mismatched index.
4. Use [lesson 04](04-recall-test-and-speed.llms.md). Run `recall_test.py` with real queries in CI and `length_timing.py` on the target hardware; use a llama.cpp build after PR #30416 or `-fa off`.
5. Keep these links exact: [Search private files on the phone](../on-device-retrieval-stack/), [Reuse answers that are almost the same](../on-device-semantic-cache/), [Test AI behavior before you ship](../on-device-eval-harness/), [Local LLM Runtime Traps (Ollama + llama.cpp)](../local-runtime-traps/).
6. Use the human HTML pages for the reader. Use the `.llms.md` briefs for agent steps and acceptance checks.

## Constraints

- Plain spoken English for a non-native reader. Literal sentences. No quizzes. No em dashes.
- Brand Edge FDE only. Do not name a course author.
- Use the model card as the source for prefixes, dimensions, and number formats.
- Report toy results honestly. Do not claim a quality gain the test did not show.
- Environment setup comes before the first runnable example.

## Key concepts

- Task prefixes come from the model card, not the LiteRT-LM guide table.
- `float16` breaks EmbeddingGemma 2 quietly.
- Truncated vectors must be normalized again.
- One model, prefix set, and dimension per index.
- A recall test is the only way to see quality.

## Takeaways

- Store prefixes and model name together.
- Run in `bfloat16` or `float32` and check vectors before storing.
- Slice then normalize.
- Store the setup with the index and rebuild on change.
- Test recall with real queries and time embeddings on your hardware.

## Sources

- EmbeddingGemma 2 model card: https://huggingface.co/google/embeddinggemma-2
- LiteRT-LM #3929: https://github.com/google-ai-edge/LiteRT-LM/issues/3929
- llama.cpp #30398: https://github.com/ggml-org/llama.cpp/issues/30398 (fixed by PR #30416)
- immich #32257: https://github.com/immich-app/immich/issues/32257