LLM386: A Runtime for Managing LLM Context Windows

LLM386 is a new runtime for large‑language‑model agents that treats the model as a pure function and manages all continuity in a persistent store.

The alpha 1.0.0‑alpha release ships a Rust library, a PyO3‑based Python SDK, and a CLI, all licensed under Apache‑2.0. Internally it uses an LMDB block store that deduplicates content by hash, a pager that selects blocks from recency, BM25, ANN, and custom retrievers, and a packer that renders a deterministic prompt string or role‑tagged chat list. A tracer records byte‑level prompt hashes for replay, a reducer parses model output into committed state events, and a typed‑edge graph keeps tool results tied to the assistant message that requested them; a diff layer compares consecutive trace records. The runtime deliberately omits a UI, hidden state, distributed storage, and learned components to keep the hot path deterministic and replayable, while a ModelProfile abstraction lets you swap models without rewriting prompt assembly. A quick demo is available via Docker Compose, which runs a small chatbot with calculator and fake user‑profile tools and persists conversation across container restarts. LLM386 is designed for developers building complex agents who need to reason about what the model sees at each turn. The project is hosted on GitHub at https://github.com/fitzee/llm386.