Rhesis

Rhesis is an open-source testing platform designed for teams building LLM and AI agent applications. It enables engineers, product managers, and domain experts to collaboratively generate test cases, simulate realistic and adversarial user conversations, and trace every failure to its root cause. By moving from ad-hoc 'hope it works' validation to systematic, repeatable testing, Rhesis helps teams catch regressions before they reach production. The platform is built for cross-functional AI teams—developers, PMs, and subject matter experts—who need a shared workflow for quality assurance. Users can create test suites from natural language descriptions, run simulations of end-user interactions, and drill down into failure traces to understand model behavior. Rhesis emphasizes traceability, allowing teams to link each test failure to the specific input, model response, and intermediate steps. What sets Rhesis apart is its open-source foundation and team-centric design. Unlike black-box testing services, Rhesis provides full visibility into test definitions and execution results. It supports adversarial testing (e.g., prompt injection attempts, edge cases) and integrates into CI/CD pipelines. The platform is self-hostable, giving teams control over their data and testing environments. Rhesis is still in early stages, with a focus on core testing workflows. While it currently lacks extensive integrations or advanced analytics, its roadmap points toward deeper support for agentic evaluation, multi-step conversation testing, and expanded model compatibility.