plugmyagent

Promptfoo

Eval & Testing Open source (MIT)

GitHub stars26.2k
Forks2.5k
LicenseMIT
Last activityOct 20, 2026
Open issues804

Repository: promptfoo/promptfoo ยท Data checked Oct 2026

About

Promptfoo is the standard open-source harness for evaluating language model apps before they embarrass you in production. You write test cases in declarative YAML, grade outputs with model-based assertions, and run red-teaming suites that probe for prompt injection and jailbreaks. Results open in a web viewer, and it gates your CI so a bad prompt never ships. It runs fully locally under MIT, no signup; the LLM-judge assertions just spend your own model tokens. Note: OpenAI announced an acquisition in March 2026, and the repo remains MIT open source.

Details

Open source. You run it yourself.

Auth: Open source (MIT)

Rate limit: None, runs locally (eval judges use your own model tokens)

Verified: 2026-09-19

Best for: developer business

Read the docs Back to catalog

Setup

  1. Run the init command above in your project folder
  2. Describe your prompt and a few test cases in the generated YAML
  3. Run npx promptfoo eval and open the results viewer
  4. Add a red-teaming run, then wire eval into your CI
Sign up