EDGE FDE
IntermediateOn-Device Search with EmbeddingGemma 2
Get on-device search results you can trust, and catch quiet embedding mistakes before users do.
Correct task prefixes, safe number formats, truncation with renormalization, a setup-checked index, and a recall test.
Serve EmbeddingGemma 2 locally, embed with the model card prefixes, store checked unit vectors with their setup, and measure recall and speed on your own data.
Lessons
Key concepts
- Task prefixes come from the model card, not the LiteRT-LM guide table.
float16breaks EmbeddingGemma 2 quietly.- Truncated vectors must be normalized again.
- One model, prefix set, and dimension per index.
- A recall test is the only way to see quality.
Takeaways
- Store prefixes and model name together.
- Run in
bfloat16orfloat32and check vectors before storing. - Slice then normalize.
- Store the setup with the index and rebuild on change.
- Test recall with real queries and time embeddings on your hardware.
Agent briefs
FAQ
Which prefixes should I use with EmbeddingGemma 2?
Use the model card prefixes. For search, task: search result | query: for queries and title: none | text: for documents. The LiteRT-LM guide table differs, see LiteRT-LM issue 3929.
Can I run EmbeddingGemma 2 in float16?
No. The model card says to use bfloat16 or float32. In float16 it returns NaN or quietly worse vectors.
Can I make EmbeddingGemma 2 vectors shorter?
Yes, keep the first 512, 256, or 128 dimensions, then L2-normalize again. Quality is close to full down to 256.
Can I mix vectors from two embedding models in one index?
No. Queries and documents must use the same model, prefixes, and dimension. Store the setup with the index and rebuild when it changes.
How do I know if my embedding search got better or worse?
Run a recall test with 50 or more real labelled queries and compare recall@1 before and after each change.
Related courses
Search private files on the phone · Reuse answers that are almost the same · Test AI behavior before you ship · Local LLM Runtime Traps (Ollama + llama.cpp)