Promptfoo is a free, open-source command-line tool and library for testing and red-teaming applications built on large language models. You write test cases in a simple config file, run them against one or more models, and get pass/fail results. It can also attack your app automatically to find security problems such as prompt injection and jailbreaks. Promptfoo is MIT-licensed and runs locally. OpenAI announced its acquisition on March 9, 2026, and the project remains open source.
Most Promptfoo articles are either acquisition news or long CLI manuals. This guide covers what you need to decide whether to use it: what it does, what stays free, a first test you can copy, and where it fits next to monitoring tools.
Promptfoo at a glance
| Detail | Information |
| What it is | Open-source CLI and library for LLM evaluation and red teaming |
| Founded | 2024, by Ian Webster (CEO) and Michael D’Angelo |
| Owner | OpenAI (acquisition announced March 9, 2026) |
| License | MIT |
| Runs on | Your machine or your own infrastructure (Node.js) |
| Funding before acquisition | $5M seed (2024, led by Andreessen Horowitz); $18.4M Series A (2025, led by Insight Partners) |
| Adoption (March 2026) | 350,000+ developers had used it; 130,000 active monthly; teams at 25%+ of Fortune 500 companies |
| Model support | OpenAI, Anthropic, Azure, Google, AWS Bedrock, Ollama, Hugging Face and custom APIs |
What Promptfoo does
Promptfoo brings software-testing habits to AI apps. Instead of eyeballing a few chatbot replies after every prompt change, you define what “good” looks like and let Promptfoo check it every time.
It covers three jobs:
- Evaluations (evals). You run the same test cases across prompts and models, then compare the results side by side. Checks range from simple text matches to model-graded rubrics.
- Red teaming. Promptfoo generates adversarial inputs aimed at your app, such as prompt injection, jailbreaks, and data leaks. It covers 50+ vulnerability types and produces a risk report.
- CI/CD checks. Tests run on every code change, so a prompt edit that breaks behavior is caught before release.
Because evals run locally, your prompts don’t pass through Promptfoo’s servers. They only go to the model providers you choose.
Who owns Promptfoo now?
OpenAI announced on March 9, 2026, that it would acquire Promptfoo. It plans to build the technology into OpenAI Frontier, its enterprise platform for AI agents. The project’s README later confirmed that Promptfoo had joined OpenAI.
What matters for users:
- The license is unchanged. Promptfoo remains open source under MIT.
- Multi-provider support continues. Promptfoo said it would keep supporting a range of model providers, so you can still test Claude, Gemini or local models.
- Customers keep support. OpenAI said it will keep serving and supporting existing customers.
One caveat is worth stating honestly: a testing tool owned by one model provider may not stay fully neutral long-term. For now, nothing in the license or features favors OpenAI models. Keep an eye on release notes if neutrality matters to your team.
Promptfoo pricing: what’s free?
| Plan | Price | What you get |
| Community | Free forever | All evaluation features, all providers, vulnerability scanning, local or self-hosted use, red teaming up to 10,000 probes/month, community support |
| Enterprise | Custom quote | Team sharing, continuous monitoring, centralized security dashboard, SSO and access control, custom red-team limits, managed cloud |
| On-Premise | Custom quote | Enterprise features on your own infrastructure |
A probe is one request sent to your app during red-team testing. Normal evals don’t use probes; only red-team runs do.
Will the free 10,000 probes be enough?
That depends on how often you scan, not how big your app is. Some illustrative schedules:
| Testing schedule | Probes per run | Runs per month | Monthly probes | Fits free tier? |
| Weekly scan before releases | 1,500 | 4 | 6,000 | Yes |
| Scan on every pull request | 300 | 40 | 12,000 | No |
| Daily scan | 500 | 30 | 15,000 | No |
These are example figures, not Promptfoo defaults; real probe counts depend on which plugins and strategies you enable.
The practical takeaway: on the free tier, run plain evals on every pull request (they don’t use probes) and run red-team scans weekly or before releases. Move to Enterprise when you need scans on every change across several apps.
Your first Promptfoo test in 10 minutes
Here is a small, original example: a support bot that must follow a 30-day refund policy.
- Install Promptfoo. With a recent Node.js, run npm install -g promptfoo.
- Create a config file named promptfooconfig.yaml:
- Add your API keys as environment variables, such as OPENAI_API_KEY and ANTHROPIC_API_KEY.
- Run it with promptfoo eval, then promptfoo view to open the results grid in your browser.
- Read the grid. Each row is a test, and each column is a model. You’ll see at a glance which model holds the policy and which one caves.
Swap in the model names you actually use. The second test is a tiny taste of red teaming; promptfoo redteam automates hundreds of attacks like this.
Promptfoo vs observability tools like Langfuse
Promptfoo and observability platforms solve different halves of the same problem:
| Promptfoo | Langfuse (observability) | |
| When it runs | Before release, in development and CI | After release, on live traffic |
| Main question | “Does this change break anything?” | “What is happening in production?” |
| Core features | Evals, model comparison, red teaming | Tracing, cost tracking, prompt management |
| Owner | OpenAI | ClickHouse |
| License | MIT | MIT (core) |
Many teams use both. Langfuse shows a real failure in production; you turn it into a Promptfoo test, and that test stops it from coming back. For other monitoring options, see our LangWatch and Llm. report listings.
Who should use Promptfoo?
- Developers shipping prompts to users who want a repeatable way to compare models such as Claude, GPT or Mistral on their own data.
- Teams building AI agents with frameworks like LangGraph or CrewAI, where tool misuse and prompt injection carry real risk.
- Security and compliance teams that need evidence an app was tested before launch.
- Probably not yet: if you’re only chatting with an AI assistant in a browser and have no code calling a model, there’s nothing to test.
Red teaming is becoming standard practice even at the AI labs themselves. We covered this in Anthropic deploys AI agents to audit models for safety.
Conclusion
Promptfoo is one of the easiest ways to stop guessing whether a prompt or model change made your AI app better or worse. The free version covers most small teams, as long as you schedule red-team scans sensibly. Its new owner is worth watching, but for now it remains open, local and multi-provider.
Building or testing an AI app? Explore our full directory of AI tools for models, agent frameworks, and developer tools that pair well with Promptfoo.
Frequently asked questions
Is Promptfoo free?
Yes. The open-source Community version is free forever and includes all evaluation features plus up to 10,000 red-team probes a month. Enterprise plans are custom-priced.
Is Promptfoo owned by OpenAI?
Yes. OpenAI announced the acquisition on March 9, 2026. Promptfoo remains open source under the MIT license.
Does Promptfoo only work with OpenAI models?
No. It supports OpenAI, Anthropic, Google, Azure, AWS Bedrock, Ollama, Hugging Face and custom APIs.
Does Promptfoo send my prompts to its servers?
No. Evals run locally, so prompts go only to the model providers you configure.
What is the difference between evals and red teaming in Promptfoo?
Evals check that your app does what it should on test cases you write. Red teaming generates attacks to find what your app should not do, such as leaking data or obeying injected instructions.


