Caching Gemini Responses in HiyokoLogcat to Improve UX
Patch: implement DiagnosisCache with SHA256 hash key to cache Gemini responses and avoid repeated API calls; test with Tauri command integration.
Patch: add DiagnosisCache to your HiyokoLogcat codebase and integrate it into the diagnose command to cache Gemini responses.
Summary
HiyokoLogcat’s AI diagnosis overlay calls Gemini each time it is opened, which can waste rate‑limit quota and add latency.
The author proposes a caching layer that hashes the log context with SHA‑256 to produce a deterministic cache key, storing the result in a HashMap with a configurable maximum size. The DiagnosisCache struct tracks entries with a result string and a creation timestamp, evicting the oldest entry when the capacity is exceeded. The cache is accessed from a Tauri command: on a cache hit the stored result is returned instantly; on a miss, Gemini is called, the response is stored, and then returned. The implementation is written in Rust, uses std::collections::HashMap, and includes helper methods for key generation, get, and insert. By caching identical inputs, the overlay can reopen in zero seconds without additional API calls, improving user experience and reducing costs. The code snippet demonstrates how to integrate the cache into the diagnose command, ensuring that repeated diagnostics reuse the same result. This pattern can be applied to any AI‑powered feature that receives the same prompt repeatedly.
Key changes
- Introduces DiagnosisCache struct with HashMap and configurable max_size
- Key generation uses SHA‑256 hash of context string
- Cache get returns Option<CacheEntry> with result and timestamp
- Insert evicts oldest entry when at capacity
- CacheEntry stores result string and creation timestamp
- Used in Tauri command diagnose to check cache before calling Gemini
- Cache miss triggers Gemini call and stores result
- Cache hit returns stored result instantly