Distill

Distill is an MIT-licensed context intelligence layer that sits between an agent's inputs — RAG chunks, tool outputs, memory, docs, conversation — and the model, running a six-stage pipeline (cache, cluster, select, compress, MMR, summarize) in roughly 12ms with no LLM calls. It is built for developers running production agentic systems who self-host, not for no-code buyers. The dedup is semantic, not hash-based: agglomerative clustering groups chunks that say the same thing in different words, keeps the best representative per cluster, then extractive compression and MMR re-ranking trim noise while preserving diversity. The vendor reports 50 chunks processed in under 2ms and 30-40% token reduction on overlapping RAG output. Because the pipeline is math rather than generation, identical input yields identical output and every clustering decision lands in an auditable log — the trade Distill makes against LLM compressors is determinism for creativity. Above the pipeline sits cross-session memory. Entries within 0.15 cosine distance merge on write, entries between 0.15 and 0.35 are flagged as conflicts the caller resolves via supersede, and pattern-based sensitivity tagging catches PII and credentials in under a millisecond. Memories decay hierarchically from full text to summary to keywords to eviction, with access resetting the clock. Sessions are token-budgeted windows (128K default, configurable) that dedupe incoming context and auto-compress aging entries. Deploy it three ways — HTTP API server, MCP server for Claude Desktop or Cursor, or an embedded Go binary — with OpenAI, Anthropic, Cohere or Ollama for embeddings; Ollama means fully local and no API key. A v0.9.1 release shipped an OpenAPI 3.1 spec with interactive docs and a Batch API for async jobs. Observability is included via Prometheus metrics, OpenTelemetry tracing with per-stage spans, JSON logging and a Grafana dashboard template. A code intelligence side adds blast-radius analysis and semantic