Cocoindex

CocoIndex reframes data engineering around a React-like mental model: you declare the desired target state (Target = F(Source)) and the engine computes the minimum work to reach it. The practical upshot is that your incremental code looks nearly identical to the one-off version—you decorate a function with @coco.fn, and the framework derives the transformation graph, memoizes by input fingerprint and code state, and skips downstream work when nothing changed. Two details stand out. First, code changes are first-class: ship new code and only affected rows re-run, with schemas auto-evolved and no index swap or downtime—addressing the migration meetings that plague batch pipelines. Second, lineage is end-to-end via CocoInsight, so every byte in a target traces back to a specific source, which matters when an agent's answer depends on stale or wrong context. The packaged CocoIndex Code starter is AST-based: it extracts call graphs, symbol tables, and hierarchies and keeps them fresh, so a coding agent can answer 'where is embed() called?' from a live index rather than a full re-scan. Breadth of connectors is genuinely wide—Amazon S3, Azure Blob Storage, BigQuery, Apache Doris, FalkorDB, Google Drive, Iggy, Kafka, LanceDB, Neo4j, OCI Object Storage, Postgres, Qdrant, Snowflake, SQLite, SurrealDB, Turbopuffer, Valkey, zvec—plus LiteLLM, Sentence Transformers, and entity resolution built in. Where it doesn't fit: it's a library, not a managed service, so you own hosting, scaling, and upgrades; the initial full-index on a very large repo can be slow; and Python 3.11–3.13 is a hard floor. It's also not built for extreme-throughput streaming at millions of events per second. Think of it as infrastructure for solving the staleness problem in long-horizon agents, where the delta engine and observability are the point—not a turnkey product.