Promptfoo
Automated red teaming and LLM security testing for AI applications
Promptfoo is the most complete open-source red teaming tool we've seen, and the OpenAI acquisition gives it resources and credibility few competitors can match. The free Community tier offers 10k probes/month, enough for serious testing. Enterprise pricing is custom, and the acquisition may give some buyers pause about independence. If you want a quick guardrail-only fix, look at Lakera; for code-first, automated deep security testing, this is hard to beat.
Verified 8d ago · liveness 87/100 · cite: rightaichoice.com/tools/promptfoo
- Enterprise AI teams needing automated red teaming for LLM applications
- Financial services requiring FINRA-aligned security testing
- Developers wanting continuous AI security in CI/CD pipelines
- Teams building RAG and agent-based systems needing vulnerability detection
- Small teams wanting a simple prompt testing tool without automation
- Projects not using LLMs in production (overkill for prototypes)
- Teams needing a fully no-code security solution (requires some config)
We scan live Reddit threads, YouTube comments, X posts, G2 reviews and other communities — and hand you an honest verdict in under a minute.
- Honest verdict, not marketing
- Real pros & cons from real users
- Attributed quotes with receipts
3 free scans · no card needed
Skip Promptfoo if you're a small team looking for a simple prompt testing tool without automation, or if you need a no-code solution; it requires configuration and is overkill for non-production LLM projects.
The Community plan's 10k probes/month limit can be restrictive for extensive testing; additional probes require Enterprise pricing.
Promptfoo's Community plan is free and generous for individual developers, but for teams needing advanced features like SSO and continuous monitoring, Enterprise pricing is custom—potentially more expensive than fixed-tier competitors like Lakera, but offers deeper automation.
In short
Promptfoo — Automated red teaming and LLM security testing for AI applications. Best for Enterprise AI teams needing automated red teaming for LLM applications, Financial services requiring FINRA-aligned security testing, Developers wanting continuous AI security in CI/CD pipelines. Free to use.
What's new in Promptfoo
Checked 8 days agoAcross the latest 5 updates: 2 feature updates and 3 news mentions.
OpenClaw at Work: Prompt Injection Risks
Lab test shows malicious webpage can make OpenClaw enumerate tools, read files, write artifacts, and send unauthorized messages.
McKinsey's Lilli Looks More Like an API Security Failure Than a Model Jailbreak
Public reporting points to exposed API surface, unsafe SQL construction, and broken object-level authorization.
Promptfoo is joining OpenAI
Promptfoo agreed to be acquired by OpenAI. Open-source project will continue; founders start new chapter.
Open-Sourcing ModelAudit: Security Scanner for ML Model Files
ModelAudit scans 42+ ML model formats for unsafe loading behaviors, known CVEs, and suspicious artifacts.
Indirect Prompt Injection in Web-Browsing Agents
Test if AI browsing agents follow malicious instructions or leak data with the indirect-web-pwn strategy.
Viability Score
How well maintained and how widely used is Promptfoo? Built from what the vendor actually publishes (docs, changelog, tutorials, integrations, pricing), whether the site is live, and how much real users discuss it. How we calculate this
Last calculated: September 2026
How we score →Key Features
- Automated red teaming for agents & RAGs
- Context-aware attack generation (injections, jailbreaks, PII leaks)
- Real-time guardrails against jailbreaks and adversarial attacks
- CI/CD integration (GitHub, GitLab, Jenkins)
- Code scanning in IDE (VS Code, JetBrains)
- Model security testing and monitoring
- MCP Proxy for secure Model Context Protocol communications
- Evaluations for prompts, models, and RAG pipelines
- Remediation guidance in pull requests
- SaaS or on-premise deployment
- Real-time threat intel from 300k+ community
- Supports 50+ vulnerability types
- Community edition with 10k probes/month
- ModelAudit: open-source scanner for ML model files (CVEs, unsafe loading)
- Customizable attack profiles and target settings (Enterprise)
About Promptfoo
Promptfoo is an AI security testing platform that takes a developer-first approach to finding and fixing vulnerabilities in LLM applications. Built for engineering and security teams, it embeds security testing directly into the software development lifecycle, so you can catch prompt injections, jailbreaks, and other AI-specific risks before they reach production. With over 50 vulnerability types covered, it automates red teaming, evaluations, and continuous monitoring — all from a code-first workflow that developers actually use. The platform's core is automated red teaming for agents and RAG pipelines. It simulates real users and generates context-aware attacks tailored to your specific application — covering direct and indirect prompt injections, jailbreaks engineered around your guardrails, data and PII leaks, business rule violations, insecure tool use, and toxic content generation. This is not a generic test suite; it adapts to your business logic, integrations, and agent workflows. Beyond testing, Promptfoo provides guardrails for real-time protection against jailbreaks and adversarial attacks, model security testing and monitoring, an MCP proxy to secure Model Context Protocol communications, and code scanning that surfaces LLM vulnerabilities directly in VS Code and JetBrains. Evaluations let you test prompts, models, and RAG pipelines for quality and safety. Everything is designed to integrate with your existing CI/CD pipelines (GitHub, GitLab, Jenkins), with findings and remediation suggestions shown right in pull requests. Promptfoo is open source, free to start, and used by 156 of the Fortune 500. It was recently acquired by OpenAI, with the open-source project continuing under its original founders. This gives it deep backing, though some buyers may pause over long-term independence. For teams that want a code-first, automated security testing platform rather than a simple evaluation tool, Promptfoo is a leading option.
Behind the Verdict
Promptfoo stands out as a code-first, automated security testing platform that embeds directly into your development workflow. Its ability to generate context-aware attacks tailored to your specific application—covering prompt injections, jailbreaks, PII leaks, business rule violations, insecure tool use, and toxic content—is a major strength. The 300k+ community threat intelligence feeds real-time updates to attack vectors, keeping your testing relevant. The recent addition of ModelAudit, an open-source scanner for ML model files, extends its reach beyond LLM prompts to underlying model artifacts. The OpenAI acquisition is a double-edged sword: it brings resources and credibility, but may raise concerns about long-term independence and neutrality. For teams needing deep, automated testing, it's excellent; for those wanting a simple no-code guardrail, other tools may be more suitable. The Community tier's 10k probes/month limit might be restrictive for extensive testing, but it's free and sufficient for many. Overall, if you're serious about AI security and want a tool that scales with your team, Promptfoo is a compelling choice.
Researching Promptfoo? Get your full AI stack in 60 seconds.
Free, no signup — tell us your goal and get tools matched to your budget & existing stack.
Real-world workflow fit
Concrete scenarios for the personas Promptfoo actually fits — and what changes day-one when you adopt it.
Integrating Promptfoo into CI/CD
Outcome: Adds a Promptfoo run to every pull request, automatically generating attacks and providing remediation guidance before merge.
Red-teaming a new agent
Outcome: Uses the `npx promptfoo@latest redteam setup` command to simulate real users and identify vulnerabilities like prompt injections and PII leaks.
Monitoring AI security across applications
Outcome: Sets up continuous monitoring and uses the centralized dashboard to track vulnerabilities and compliance across projects.
Use Cases
- Block a prompt change in CI if semantic similarity drops below threshold on 50 test cases.
- Red-team an agent with 30 canonical jailbreak patterns before launching a public feature.
- Compare GPT-4o vs Claude Sonnet on your actual test suite and pick the winner by measurable metrics.
- Prevent regressions in a prompt by tying every merge to a passing Promptfoo run.
- Automated FINRA-aligned security testing for financial services AI.
- Ensure fair housing compliance in real estate AI agents.
- Scan for PII leaks and business rule violations in production AI applications.
Models Under the Hood
as of 2026-08-31
Limitations
- Promptfoo is an AI security testing platform that simulates real users to uncover application-specific vulnerabilities such as prompt injections, jailbreaks, and PII leaks.
- The free Community plan includes 10,000 red teaming probes per month, while Enterprise and On-Premise plans offer advanced features like continuous monitoring, SSO, and API access, with pricing available on request.
- As of March 2026, Promptfoo is being acquired by OpenAI, and the open-source project will continue under the founders.
as of 2026-08-30
Verification history
We have re-verified Promptfoo 17 times since . Each pass re-reads the vendor's own pages and re-checks every listed field against that evidence; passes where nothing had changed are marked as such.
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
- — re-verified summary, description, our verdict, our analysis, pricing model, pricing tiers, features, integrations, who it suits, who should skip it
Showing the 6 most recent of 17 verification passes.
Free to cite with attribution — this page re-verifies continuously.
12-month cost
Project the real annual outlay, including the implied monthly cost when only an annual tier is published.
Vendor list price only. Add-on usage, seat overages, and contract minimums are surfaced under Hidden costs & gotchas.
Plans compared
For each published Promptfoo tier: who it actually fits, and what it adds vs. the previous tier. Cross-reference the cost calculator above for projected annual outlay.
Community
$0/mo
Ideal for
Individual developers and small teams starting with LLM security testing, who need free, local evaluation and basic red teaming.
What this tier adds
Free entry point with all core evaluation features and 10k red teaming probes per month, but limited to community support.
Enterprise
Custom
Ideal for
Larger teams needing advanced features like SSO, continuous monitoring, and custom attack profiles for enterprise-scale operations.
What this tier adds
Adds team sharing, continuous monitoring, centralized dashboard, custom attack profiles, SSO, granular permissions, and API access over Community.
On-Premise
Custom
Ideal for
Organizations with strict data sovereignty or compliance requirements that need full control over their infrastructure and complete data isolation.
What this tier adds
Adds on-premise deployment, complete data isolation, dedicated runner, and a deployment engineer, building on Enterprise features.
Where the pricing makes sense
The company stage and team size where Promptfoo's pricing actually pencils out — and where peers do it cheaper.
Promptfoo's Community plan is free and generous for individual developers, but for teams needing advanced features like SSO and continuous monitoring, Enterprise pricing is custom—potentially more expensive than fixed-tier competitors like Lakera, but offers deeper automation.
Setup time & first value
How long it actually takes to get something useful out of Promptfoo — broken out by persona, not the marketing-page minute.
For developers, setting up Promptfoo for local evaluation is quick—within 10 minutes using npx. Integrating into CI/CD takes about 30 minutes. For enterprise deployment with continuous monitoring and SSO, expect a few days to weeks.
Switching to or from Promptfoo
How to bring data in from common predecessors and how to get it back out — written for the switcher, not the buyer.
- →From [Lakera]: Replace guardrail-only approach with comprehensive red teaming and CI/CD integration.
- →From [Custom scripts]: Transition from ad-hoc prompt testing to automated, scalable red teaming with community threat intel.
- →From [Other evaluation tools]: Consolidate evaluations and security testing in one platform with a code-first workflow.
- ↗To [Lakera]: If you only need real-time guardrails and not full-scale red teaming, consider switching to a simpler solution.
- ↗To [Homemade solutions]: For small teams, you might revert to custom scripts, but you'll lose automated attack generation and community intel.
Integrations
Resources & Guides
Tutorials & Learning
Featured Head-to-Head Comparisons
Mlflow vs Promptfoo
Choose Promptfoo if your top priority is automated red teaming and LLM vulnerability detection in production—especially for regulated industries. Choose MLflow if you need a comprehensive open-source platform for agent observability, experiment tracking, and model deployment. Both are free to start, but MLflow's open-source model has no usage caps, while Promptfoo's community edition limits probes per month.
Langfuse vs Promptfoo
Choose Promptfoo if your priority is AI security — automated red teaming, guardrails, and CI/CD scanning against 50+ attack types, backed by recent OpenClaw injection analysis and ModelAudit launch. Choose Langfuse if you need production LLM observability, prompt management, and evaluations with deep framework integration (100+), now with multi-modal datasets and monitors/alerts. Both are open-source, but Promptfoo leans security-first while Langfuse is engineering-first.
Popular in AI Governance & Guardrails
Mindgard
Automated AI red teaming platform that continuously discovers, assesses, and defends AI systems and agents.
Poolside AI
Open-weight agentic coding models for secure on-prem enterprise AI
Olas Network
Co-own and monetize AI agents on-chain with Olas.
Frequently Asked Questions
Used Promptfoo? Help shape our editorial sentiment research.