Gemma 4 E2B on OnePlus CE5 – Structured Voice Note Parsing
Run Gemma 4 E2B locally on a 8GB phone to parse voice notes into structured JSON.
Implement the Android app pipeline and benchmark on target devices.
Summary
An Android developer runs Gemma 4 E2B (2.4GB) locally on a OnePlus CE5 with 8GB RAM, achieving JSON‑structured output from short prompts. The model is paired with Whisper Small (244MB) for transcription via Sherpa‑ONNX, and both run on the phone without cloud support. The total inference time for a 10‑15 second voice note is about 12‑15 seconds, with Whisper taking ~5s and Gemma ~8‑10s.
The app categorises voice notes into items such as reminders and purchases, resolving times and tags. At query time, the system performs query expansion, rewriting user questions into keywords and example items, then merges results from multiple FTS lanes using reciprocal rank fusion. An optional Gemma reranker pass over the top‑K results runs with a 15s timeout.
The post demonstrates that local LLMs can handle voice‑to‑text and structured data extraction on modest hardware, and invites others to share feedback on categorisation performance.
Key changes
- Gemma 4 E2B (2.4GB) on OnePlus CE5
- JSON output from structured prompts
- Whisper Small 244MB for transcription
- 12‑15s total inference time
- Query expansion and RRF for retrieval
- LiteRT‑LM runtime