EvalForge

EvalForge turns AI evaluation results into release evidence. Import results from promptfoo, Ragas or DeepEval, compare a candidate against a baseline on the same dataset, apply explicit quality, security, latency and cost thresholds, and get JSON, JUnit, SARIF and Markdown reports with a non-zero exit code on regression.

Why it matters: most teams can compute eval metrics but have no shared answer to "what changed, which limits did we enforce, and where is the evidence?" EvalForge is a deliberately narrow layer that plugs into the CI you already have, with a reusable GitHub Action and no hosted account.

Python, MIT, about 190 stars at the time of listing. Version 0.6.0 is an alpha: the gate and the offline evaluator are usable today.