plugmyagent

Inspect AI

Eval & Testing Open source (MIT)

AuthenticationOpen source (MIT)
Rate limitNone; runs locally, free and open source

Documentation: inspect.aisi.org.uk ยท Checked Sep 21, 2026

About

Inspect AI is the evaluation framework from the UK AI Safety Institute, and it is MIT licensed with no signup. It ships with more than 200 pre-built evals covering agent tool use and reasoning, so you can benchmark your agent against standard tests instead of inventing your own. Install with pip and run evals from the command line.

Details

Open source. You run it yourself.

Auth: Open source (MIT)

Rate limit: None; runs locally, free and open source

Verified: 2026-09-21

Best for: developer

Read the docs Back to catalog

Setup

  1. Run pip install inspect-ai.
  2. Browse the 200+ built-in evals for the ones that match your agent.
  3. Run an eval against your model from the command line.
  4. Read the scores and traces to find where your agent breaks.
Sign up