hamelsmu/evals-skills

8258

eval-audit Audit an LLM eval pipeline and surface problems: missing error analysis, unvalidated judges, vanity metrics, etc. Use…

hamelsmu/evals-skills

1.3K

2

evaluate-rag Guides evaluation of RAG pipeline retrieval and generation quality. Use when evaluating a retrieval-augmented generat…

hamelsmu/evals-skills

703

3

write-judge-prompt Design LLM-as-Judge evaluators for subjective criteria that code-based checks cannot handle. Use when a failure mode …

hamelsmu/evals-skills

680

4

error-analysis Help the user systematically identify and categorize failure modes in an LLM pipeline by reading traces. Use when sta…

hamelsmu/evals-skills

652

5

validate-evaluator Calibrate an LLM judge against human labels using data splits, TPR/TNR, and bias correction. Use after writing a judg…

hamelsmu/evals-skills

611

6

generate-synthetic-data Create diverse synthetic test inputs for LLM pipeline evaluation using dimension-based tuple generation. Use when boo…

hamelsmu/evals-skills

609

7

build-review-interface Build a custom browser-based annotation interface tailored to your data for reviewing LLM traces and collecting struc…

hamelsmu/evals-skills

555