ukgovernmentbeis/inspect_evals

1

code-quality-fix-all Fix code quality issues identified in a code quality review stored in agent_artefacts/code_quality/<topic>/.

ukgovernmentbeis/inspect_evals

4

2

build-repo-context Crawl repository PRs, issues, and review comments to distill institutional knowledge into a shared knowledge base. Ru…

ukgovernmentbeis/inspect_evals

3

3

check-trajectories-workflow Use Inspect Scout to analyze agent trajectories from evaluation log files.

ukgovernmentbeis/inspect_evals

3

4

ci-maintenance-workflow CI and GitHub Actions maintenance workflows — fix a failing test from a CI URL, fix a failing smoke test, add @pytest…

ukgovernmentbeis/inspect_evals

3

5

code-quality-review-all Review all evaluations in the repository against a single code quality standard.

ukgovernmentbeis/inspect_evals

3

6

create-eval Redirect to the inspect-evals-template for creating new evaluations. New evals are no longer created in this reposito…

ukgovernmentbeis/inspect_evals

3

7

ensure-test-coverage Ensure test coverage for a single evaluation - both reviewing existing tests and creating missing ones. Analyzes test…

ukgovernmentbeis/inspect_evals

3

8

eval-quality-workflow Fix or review a single evaluation against all EVALUATION_CHECKLIST.md standards. Use "fix" mode to refactor an eval i…

ukgovernmentbeis/inspect_evals

3

9

eval-report-workflow Create an evaluation report for a README by selecting models, estimating costs, running evaluations, and formatting r…

ukgovernmentbeis/inspect_evals

3

10

eval-validity-review Review a single evaluation's validity — whether its claims hold up, whether its name is accurate, whether samples can…

ukgovernmentbeis/inspect_evals

3

11

generate-asset-actions Generate asset-actions.yaml from ASSETS.yaml by classifying assets into priority tiers. Use when the user asks to reg…

ukgovernmentbeis/inspect_evals

3

12

investigate-dataset Investigate datasets from HuggingFace, CSV, or JSON files to understand their structure, fields, and data quality. Tr…

ukgovernmentbeis/inspect_evals

3

13

prepare-release Prepare a new release of inspect_evals by creating a release branch, collecting changelog fragments, and opening a PR…

ukgovernmentbeis/inspect_evals

3

14

prepare-submission-workflow Prepare an evaluation for PR submission as an entry to the register. Use when user asks to prepare an eval for submis…

ukgovernmentbeis/inspect_evals

3

15

read-eval-logs View and analyse Inspect evaluation log files using the Python API. Trigger whenever you need to look at a .eval file…

ukgovernmentbeis/inspect_evals

3

16

security-audit-eval Audit a third-party Inspect AI evaluation for security risks before running it locally. Decide whether the eval is sa…

ukgovernmentbeis/inspect_evals

3

17

write-an-adr Write an Architectural Decision Record (ADR) to document a significant design choice. Use when user asks to write/cre…

ukgovernmentbeis/inspect_evals

3