Vibe Test

Create useful validation evidence, not just test commands. This skill helps agents inspect current test coverage and test-case execution, decide whether the implementation meets a clear pass standard, and connect evidence back to specs, plans, commits, and CI.

Core Principles

• Write artifact prose in the user's language; keep labels, paths, commands,

code, IDs, status values, and quotes unchanged.

• Keep required template headings, field labels, tables, and status values

exactly as shown for validator compatibility; localize placeholder prose and all human-readable explanatory content to the user's language or the source artifact's primary language.

• Validate the behavior promised by the spec, plan, issue, or task.
• Prefer evidence that proves public behavior or public contracts over evidence

that only proves implementation details.

• For user-facing API errors, verify localized client copy for status and

fallback states, and check that raw server error text or codes are absent from ordinary UI.

• When approved fallback behavior is part of the change, validate the decision

source, primary path, fallback trigger, user-visible result, and debugging signal.

• Require Visual QA only when visual output is part of acceptance or risk, such

as UI, responsive layout, screenshots, rendered documents, charts, images, videos, or burned-in subtitles. Omit the Visual QA section when it is not relevant.

• For web, mobile, miniapp, or UI-facing changes, include interactive

browser/device validation when feasible: start the app, exercise the changed workflow with the coding agent browser, connected device/simulator, or project E2E tool, and capture desktop/mobile or target-device evidence.

• For a UI feature, read the spec Goal Target Platforms field as the

platform source of truth. Load only the matching sections of [references/platform-ui-test-profiles.md](references/platform-ui-test-profiles.md).

• For a non-UI feature with Target Platforms: Not applicable, do not create a

UI matrix. Record Backend P0 evidence for the public contract, relevant success/failure or authorization states, side-effect safety, and safe error responses. When a Backend P1 is selected, record its source in Backend Risk Mapping; load [references/backend-test-profile.md](references/backend-test-profile.md) only when the project needs the detailed checks.

• For every UI feature, record P0 evidence for primary interaction, relevant

key states, layout, localization, accessibility, and user-safe error messages. Add P1 only for a spec-declared risk and record its source rule or acceptance in Spec Risk Mapping.

• Prefer project-owned automated tests. A missing tool, fixture, simulator,

test account, or platform permission is Blocked; use a manual exception only for a real hardware, external-system, or non-simulatable platform path.

• Prefer existing test frameworks and local commands.
• Keep evidence reviewable and reproducible.
• Distinguish automated tests, manual checks, environment limitations, and unverified risk.
• Do not claim tests passed unless they were actually run.
• Do not invent coverage numbers, missing tests, or pass status.
• Treat test improvement recommendations separately from evidence already collected.
• State the final validation status and whether evidence blocks commit or release.
• Review the plan's expected evidence, but do not take over implementation

sequencing. Report missing evidence instead of reordering slices.

• Do not upload source, secrets, credentials, or private data.

Workflow

1. Identify The Validation Target

Read the relevant spec, plan, issue, PR, or changed files. Determine:

• What behavior changed.
• Which acceptance criteria or risks must be proven.
• Which implementation areas and test files changed.
• Which existing tests or CI checks are relevant.
• Whether a coverage tool, threshold, or CI policy exists.
• Which plan slices expected TDD, public-interface tests, screenshots, logs, or

manual evidence.

• Whether visual output, layout, viewport behavior, or rendered artifacts are

part of acceptance or risk.

• Whether the spec declares UI target platforms and any P1 risk.
• For non-UI work, the public entry point, state or side effect, and any

spec-declared P1 risk such as retry, concurrency, data, security, or latency.

2. Inspect Existing Test Coverage

Use [references/test-template.md](references/test-template.md) when producing a saved validation artifact.

Assess:

• Relevant existing test files and cases.
• Coverage reports, thresholds, or CI coverage checks when available.
• Whether changed behavior has direct, adjacent, and regression coverage.
• Whether tests prove behavior through public interfaces, user-observable

workflows, stable contracts, CLI/API outputs, or documented artifacts.

• Whether tests protect the core mechanism or SSOT named in the plan, so later

changes do not silently reintroduce duplicated rules, unclear boundaries, or bypassed workflow paths.

• Whether tests are coupled to private implementation details in a way that can

pass while behavior breaks.

• Gaps in unit, integration, end-to-end, UI/manual, accessibility, localization, security, privacy, or data validation.
• Whether the plan's expected evidence has been produced.
• Whether Visual QA evidence is required. If required, name the viewport,

device, rendered artifact, screenshot, or visual diff to inspect.

3. Run Validation

• Run existing local tests when feasible.
• Add or update focused tests when code changes require them and the task includes implementation.
• For UI-facing changes, run the app or target surface when feasible, interact

with the changed workflow, and record screenshots, recordings, visual diffs, or the exact browser/device blocker.

• Use the UI evidence matrix for new UI evidence. Cover all P0 categories;

a non-applicable P0 needs a feature-specific reason.

• Add P1 only for a spec-declared risk: irreversible or transactional actions,

system capability, network recovery, sensitive data, locale/theme/text-scale, or performance. Do not expand the matrix with speculative P1 work.

• If an environment, credential, device, network, or service blocks validation, record the limitation.
• Avoid broad unrelated test runs unless the blast radius justifies them.
• Record exact commands, targets, and outcomes.

4. Capture Evidence

Evidence should include:

• Overall status: Passed, Failed, Blocked, or Partial.
• Observable behavior evidence: what public behavior or contract was proven,

how, and what still depends on implementation-detail checks.

• Design-protection evidence: which core mechanism, rule owner, or boundary is

protected by the validation, when the plan calls one out.

• Commands run.
• Result and timestamp or run context.
• Current test-case execution status.
• Coverage command and coverage result when available.
• Relevant artifacts such as screenshots, logs, CI links, or test names.
• Visual QA evidence when visual correctness is part of the target.
• The UI evidence matrix and any manual exception when the target is UI-facing.
• Pass criteria and whether they were met.
• Recommended test improvements.
• Unverified scenarios and residual risk.
• Decision: whether evidence is sufficient for commit/release and what blocks it.
• Spec, plan, commit, PR, CI, or issue references when available.

Status rules:

• Use Passed only when every required check passed and residual risk is

non-blocking.

• Use Failed when a required check ran and failed.
• Use Blocked when required validation could not run because of environment,

credential, service, device, data, or permission blockers.

• Use Partial when useful evidence exists but required checks remain not run,

not applicable, or scoped out with an explicit reason.

• Do not hide failed, blocked, or required not-run checks inside recommendations.

Put them in Test Case Execution and Decision.

UI Acceptance Decision Rules

For a new UI evidence artifact, add the optional UI Acceptance Matrix and Manual Exceptions sections from the test template. A Passed result requires all required rows to pass with evidence. A manual exception never turns a missing ordinary automated UI path into a pass.

Keep this instruction lean: the profile reference owns platform-specific checks; the template owns field shape; the validator owns mechanical evidence gates. Before a material tool-running phase, briefly state the chosen platform, tool, and reason. Continue through the selected P0 and triggered P1 checks before deciding that validation is complete.

5. Validate Saved Evidence

In OpenSpec mode, use the user-specified change/path or the directory containing the related spec or plan as the current change directory. If neither identifies it, ask before writing. Save test.md there; if it exists, update it only when explicitly requested.

If you save a validation artifact, run:

python3 skills/vibe-test/scripts/validate_test_evidence.py path/to/test.md

Output

When finishing, report:

• Overall status and decision.
• Coverage and test-case status.
• Commands run and results.
• Evidence produced.
• Pass criteria and whether the work passed.
• Test improvement recommendations.
• What remains unverified and why.

Skill Signature

Always end the final response with:

Vibe Skill Signature
Skill: vibe-test
Status: Completed | Passed | Failed | Blocked | Partial
Next: concise next workflow step