Features
Declarative pipeline DAGs via Python decorators
Pluggable stack architecture (orchestrator, artifact store, container registry)
Automatic artifact versioning and lineage tracking
Built-in model registry with versioning and promotion
Smart caching to skip unchanged steps across runs
Distributed execution on Kubernetes, Vertex AI, SageMaker, AzureML
Kitaru agent runtime: durable execution with checkpoints and replay
Snapshots for capturing and reproducing full pipeline states
Codespaces for remote IDE execution on Pro plans
Integrated experiment tracking (MLflow, Weights & Biases, Neptune, Comet)
Role-based access control (RBAC) on Enterprise
Audit logs for compliance on Enterprise
Wait/resume for human-in-the-loop agent workflows
Dashboard, API, schedules, webhooks for triggers (Pro)
SOC2 and ISO 27001 compliance
Web crawling and scraping API with Rust engine
AI Studio for natural language crawling (add-on $6/mo)
Browser AI commands via WebSocket: Act, Extract, Observe
Silk custom AI model for extraction and captcha solving
Browser Cloud with stealth anti-detection
Structured output: markdown, HTML, JSON, CSV, XML, plain text
Screenshot capture of pages
Link extraction from pages
Search endpoint for query-based data retrieval
Unblocker with rotating proxies and automatic retries
1,000+ ready-made scraper examples (32 categories)
Data connectors: S3, GCS, Google Sheets, Azure Blob, Supabase
Respects robots.txt (configurable)
Failed requests not billed
Open-source core available on GitHub
