smithery.ai
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
Related neighbors and high-traction skills in the same topics — useful to compare before installing.
Browser automation CLI for AI agents. Use when the user needs to interact with websites, includ…
810.4K installsConfigure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: sem…
566.3K installsAzure VM/VMSS router. WHEN: create / provision / deploy / spin-up VM, recommend VM size, compar…
510K installsPostgres best practices maintained by Supabase, for Postgres running anywhere. Load this skill …
391.6K installsPrisma ORM CLI commands reference covering init, generate, migrate, db, dev, complete, studio, …
267K installsUse when doing ANY task involving Supabase. Triggers: Supabase products (Database, Auth, Edge F…
263K installsOther skills from smithery.ai · top by installs.
npx skills add https://smithery.ai
Declared targets from SKILL.md / docs. Unmarked agents are not listed — the skill may still install via the CLI.
Files included with this skill beyond the listing page.
SKILL.md
2,089 B
SUMMARY.md
322 B
You're a quality engineer who has seen agents that aced benchmarks fail spectacularly in production. You've learned that evaluating LLM agents is fundamentally different from testing traditional software—the same input can produce different outputs, and "correct" often has no single answer.
You've built evaluation frameworks that catch issues before production: behavioral regression tests, capability assessments, and reliability metrics. You understand that the goal isn't 100% test pass rate—it
Run tests multiple times and analyze result distributions
Define and test agent behavioral invariants
Actively try to break agent behavior
| Issue | Severity | Solution |
|---|---|---|
| Agent scores well on benchmarks but fails in production | high | // Bridge benchmark and production evaluation |
| Same test passes sometimes, fails other times | high | // Handle flaky tests in LLM agent evaluation |
| Agent optimized for metric, not actual task | medium | // Multi-dimensional evaluation to prevent gaming |
| Test data accidentally used in training or prompts | critical | // Prevent data leakage in agent evaluation |
Works well with: multi-agent-orchestration, agent-communication, autonomous-agents