promptfoo
Test and red-team your prompts, agents, and RAG pipelines.
About promptfoo
promptfoo lets you test your prompts, agents, and RAG systems with simple declarative configs — plus red-teaming and vulnerability scanning to catch failures before your users do. Compare GPT, Claude, Gemini, and DeepSeek side by side.
It plugs into the command line and CI/CD, so evals run automatically on every change instead of living in someone’s head. Used by teams at OpenAI and Anthropic, it is one of the most practical ways to make agent quality measurable.
More Github
book-to-skill
book-to-skill converts a technical book PDF into a ready to load Claude Code skill, so the reference you never finished reading becomes something your agent can consult while it works. Point it at a PDF and get back a structured skill directory with the material chunked for retrieval. A neat way to turn a shelf of unread technical books into working context.
sandboxd
sandboxd is a self-hosted app builder where an agent constructs real applications inside isolated sandboxes rather than on your machine. It is the shape most hosted AI app builders should have had from the start: you keep the code, you keep the environment, and the agent gets a contained place to fail in. Run it on your own box and you are not renting your build pipeline from anyone.
Clawk
Clawk hands your coding agent a throwaway Linux VM to work in, which is the correct answer to the question every builder asks before approving a shell command. The agent gets a real machine with real tooling, your host filesystem and credentials stay out of reach, and when the task is done the VM goes away. Useful the moment you start running agents unattended.
Mantis
Mantis is a modular, stack-agnostic toolkit of security review skills for AI coding agents, open sourced by Google. Instead of hoping your agent remembers to think about injection, secrets handling or unsafe deserialization, you load the relevant skill and let it audit the code it just wrote. The skills are composable and framework independent, so they drop into Claude Code, Cursor or any harness that can load a skill directory.
Strix
Strix is an open-source AI agent that pentests your own application: it probes for vulnerabilities the way an attacker would, then reports what it found with enough context to actually fix it. For solo builders shipping AI-assisted code fast, it closes a gap that traditional linters and dependency scanners never covered. Why it earns a slot Runs real offensive-security workflows rather than pattern-matching source code, so it surfaces logic and auth flaws static tools miss. Reports come with reproduction steps and remediation guidance, not just a severity badge. Open source and self-run, which means your code and findings never leave your environment. Fits naturally after a vibe-coded sprint, where speed usually outruns security review. Use it responsibly Point it at systems you own or are explicitly authorized to test. Used that way it is a genuinely useful pre-launch gate — run it before you open signups, not after someone else does. Repo: github.com/usestrix/strix .