Shipping AI to production without testing is a recipe for a bad surprise. Promptfoo solves that: it is an open-source tool to test, evaluate and red-team LLM apps, agents and RAG, with declarative config and direct CI/CD integration.
What is Promptfoo?
With simple YAML configs, Promptfoo compares models side by side (GPT, Claude, Gemini, DeepSeek), catches regressions before deploy and runs AI-specific vulnerability scanning and pentesting. It is used by hundreds of thousands of developers.
Key features
- Side-by-side model comparison and regression detection
- Red-teaming and vulnerability scanning for AI apps
- Declarative YAML configs, MIT, with a native GitHub Action
- First-class CI/CD integration
- Around 22k stars and used by large companies
How Reche uses it
Automated evaluation is one of RecheOS's quality gates: no AI feature reaches production without testing and a security check. Reche implements that rigor in your product so AI is reliable, not a gamble.