Swiftide

Swiftide is a Rust library for building fast, streaming LLM applications. It provides indexing, querying, and agentic capabilities with a focus on performance and modularity. Designed for developers who need to ingest, transform, and index large amounts of data, then query it with retrieval-augmented generation (RAG) or build autonomous agents. The library uses an async, streaming pipeline architecture. Users can load data from various sources (e.g., files), chunk and transform text or code using built-in transformers (including tree-sitter for code), enrich with metadata, embed, and store in vector databases like Qdrant. Query pipelines transform user questions, retrieve relevant documents, and generate answers. Agent pipelines allow defining custom tools with a simple macro-based interface and hook into lifecycle events. Swiftide stands out for being written in Rust, leveraging its safety and performance. It offers minimal abstractions, making it easy to extend by implementing simple traits. Pipelines are lazy, asynchronous, and parallel, enabling efficient processing of large datasets. It also supports customizable templated prompts using Tera (Jinja-like). The target audience is Rust developers building production-level LLM applications who need speed and control. It is not a managed service but a library to be embedded in your own Rust projects.