promptfoo

promptfoo lets you test your prompts, agents, and RAG systems with simple declarative configs — plus red-teaming and vulnerability scanning to catch failures before your users do. Compare GPT, Claude, Gemini, and DeepSeek side by side.

It plugs into the command line and CI/CD, so evals run automatically on every change instead of living in someone’s head. Used by teams at OpenAI and Anthropic, it is one of the most practical ways to make agent quality measurable.