AI Builder - Code Test

This skill creates and executes comprehensive test suites to verify code quality and functionality. As a QA Engineer expert, you'll design test plans, write automated tests, perform manual testing, analyze results, identify issues, and create backlogs for changes, bugs, or improvements.

When to Use This Skill

• User asks to test a backlog or feature
• User requests test creation or execution
• Code review is complete and testing is needed
• User wants to verify implementation meets requirements
• User asks to run test suite
• User wants to validate a sprint before completion

Prerequisites

This skill requires:

• Code implementation completed
• Code review completed (recommended)
• 04-prd/ - Product Requirements Document (business requirements and acceptance criteria)
• 07-tech-specs/ - Engineering standards and constraints
• features/ folder with feature design and implementation docs
• 10-sprints/ folder with backlog and test plan
• {SRC}/ folder (organized as defined in source-code-structure.md)
• Access to source code and running environment

Feature-Driven Testing Workflow

CRITICAL: This skill follows a strict feature-driven approach where feature-name is the index for the entire project:

For Each Backlog:

1. Read backlog.md from 10-sprints/SPRINT-XX-descriptive-name/[BACKLOG_TYPE]-XX-[feature-name]-<sub-feature>.md
2. Extract the feature-name from the backlog file name
3. Read features/features-index.md to find the feature file
4. Read feature documentation in this order:

- features/[feature-name].md - Feature definition (WHAT/WHY/SCOPE) - features/flows/[feature-name].md - User flows and process flows (if exists) - features/contracts/[feature-name].md - API/data contracts (if exists) - features/impl/[feature-name].md - Implementation notes (if exists)

1. Locate code and test files in {SRC}/ using features/impl/[feature-name].md
2. Write/execute tests following 07-tech-specs/testing-standards.md
3. Update backlog.md with test results and findings

This approach ensures AI testers can test large projects without reading all code at once.

Your Roles in This Skill

See dev-swarm/docs/general-dev-stage-rule.md for role selection guidance.

Role Communication

See dev-swarm/docs/general-dev-stage-rule.md for the required role announcement format.

Test Types Overview

This skill handles multiple test types:

1. Unit Tests: Test individual functions/components in isolation
2. Integration Tests: Test component interactions and data flow
3. API Tests: Test REST/GraphQL endpoints, contracts, error handling
4. CLI Tests: Test command-line interfaces and scripts
5. Web UI Tests: Test web interfaces (Playwright, Selenium, Cypress)
6. Mobile UI Tests: Test mobile apps (if applicable)
7. Log Analysis: Verify logging, monitoring, error tracking
8. Performance Tests: Load testing, stress testing, benchmarks
9. Security Tests: Vulnerability scanning, penetration testing

Instructions

Follow these steps in order:

Step 0: Verify Prerequisites and Gather Context (Feature-Driven Approach)

IMPORTANT: Follow this exact order to efficiently locate all relevant context:

1. Identify the backlog to test:

- User specifies which backlog to test - Or test latest reviewed backlog from sprint

`` 10-sprints/ └── SPRINT-XX-descriptive-name/ └── [BACKLOG_TYPE]-XX-[feature-name]-<sub-feature>.md ` - Locate the sprint README at 10-sprints/SPRINT-XX-descriptive-name/README.md` for required progress log updates

1. Read the backlog file:

- Understand requirements and acceptance criteria - Read the test plan defined in backlog - Extract the feature-name from the file name (CRITICAL) - Verify Feature Name in backlog metadata matches the file name - If they do not match, stop and ask the user to confirm the correct feature name - Note backlog type (FEATURE/CHANGE/BUG/IMPROVE) - Identify success criteria

1. Read testing standards:

- Understand test coverage requirements - Note test frameworks and conventions

1. Read PRD and tech specs:

- Read 04-prd/ (all markdown files) - Product requirements and acceptance criteria for the feature - Read 07-tech-specs/ (all markdown files) - Technical specifications and engineering standards - Understand the business context and technical constraints

1. Read feature documentation (using feature-name as index):

- Read features/features-index.md to confirm feature exists - Read features/[feature-name].md - Feature definition (expected behavior) - Read features/flows/[feature-name].md - User flows (test these flows) - Read features/contracts/[feature-name].md - API contracts (test these contracts) - Read features/impl/[feature-name].md - Implementation notes (what was built)

1. Locate code and tests:

- Use features/impl/[feature-name].md to find code locations - Navigate to {SRC}/ directory - Check existing test files in {SRC}/ (locations from features/impl/[feature-name].md) - Identify files to test

1. Read sprint test plan:

- Check 10-sprints/SPRINT-XX-descriptive-name/README.md for sprint-level test plan - Understand end-user test scenarios - Note manual vs automated test requirements

1. Determine test scope:

- What test types are needed? - Manual or automated or both? - Environment requirements?

DO NOT read the entire codebase. Use feature-name to find only relevant files.

Checklist and Status Rules (Backlog + Sprint)

Apply these rules whenever you test a backlog or sprint:

• For any checklist, acceptance criteria, or test plan, use task list items and mark results as:

- [x] = pass - [-] = no test needed

• After testing a backlog, update its status to Done.
• After finishing a backlog test, check if any backlogs remain in the sprint README.

- If none remain, test the sprint-level checklist items in the sprint README, mark each item, and set the sprint status to Completed.

Test Method Priority

Use this priority order when choosing test methods:

1. curl commands (highest priority)
2. playwright-browser-* skills for web UI (must use browser for any web UI item)
3. Write test code
4. Write unit tests

Step 1: Design Test Strategy

Before writing tests, plan the approach:

1. Identify test scenarios:

Happy Path: - Normal, expected user flows - Valid inputs and operations - Successful outcomes

Edge Cases: - Boundary values (min, max, zero, negative) - Empty inputs - Very large inputs - Special characters

Error Cases: - Invalid inputs - Missing required data - Permission denials - Network failures - System errors

Security Cases: - SQL injection attempts - XSS attempts - Authentication bypass attempts - Authorization violations - CSRF attacks

1. Select test types:

- Which test types are appropriate? - What can be automated? - What requires manual testing? - What's the priority order?

1. Define success criteria:

- What does passing mean? - What coverage is needed? - Performance benchmarks? - Security requirements?

Step 2: Write Automated Tests

Create automated test suites based on test type:

Unit Tests

Test individual functions/components in isolation:

Best Practices:

• Test one thing per test case
• Clear, descriptive test names
• Arrange-Act-Assert pattern
• Mock external dependencies
• Test both success and failure paths

Integration Tests

Test component interactions:

API Tests

Test endpoints and contracts:

CLI Tests

Test command-line interfaces:

Web UI Tests (Playwright/Cypress)

Test web interfaces:

Step 3: Execute Manual Tests

For scenarios that can't be easily automated:

1. Follow test plan from backlog:

- Execute each manual test step - Use curl for API testing - Use CLI for command testing - Use browser for UI testing

1. Document test execution:

- Record what was tested - Note any issues encountered - Capture screenshots/logs for failures - Time performance-critical operations

1. Test across environments:

- Development environment - Different browsers (Chrome, Firefox, Safari) - Different devices (mobile, tablet, desktop) - Different operating systems (if applicable)

Step 4: Analyze Logs

Review application logs for issues:

1. Check for errors:

- Unhandled exceptions - Stack traces - Error messages

1. Verify logging quality:

- Appropriate log levels (debug, info, warn, error) - No sensitive data in logs (passwords, tokens) - Sufficient context in log messages - Proper error tracking

1. Monitor performance:

- Slow queries or operations - Memory usage patterns - Resource leaks

1. Security audit:

- No secrets logged - Proper access control logging - Suspicious activity detection

Step 5: Performance Testing (When Needed)

For performance-critical features:

1. Load testing:

- Simulate multiple concurrent users - Measure response times - Identify bottlenecks

1. Stress testing:

- Push system beyond normal limits - Find breaking points - Test recovery behavior

1. Benchmark key operations:

- Database query performance - API response times - Page load times

Step 6: Analyze Results and Identify Issues

Categorize findings into three types:

1. Changes (Doesn't meet requirements)

Implementation doesn't meet original requirements:

• Missing acceptance criteria
• Incorrect behavior vs specification
• Doesn't follow test plan
• Feature doesn't work as designed

Action: Create change type backlog

2. Bugs (Defects found)

Code has defects or errors:

• Functional bugs (incorrect results)
• UI bugs (broken layouts, wrong text)
• API bugs (wrong status codes, incorrect responses)
• Performance bugs (timeouts, slowness)
• Security vulnerabilities
• Crashes or exceptions
• Data corruption

Action: Create bug type backlog

3. Improvements (Enhancement opportunities)

Non-critical enhancements:

• Better error messages
• UX improvements
• Performance optimizations
• Additional validation
• Better logging
• Test coverage gaps
• Accessibility improvements

Action: Create improve type backlog

Step 7: Create Backlogs for Issues

For each issue found, create a backlog:

1. Determine severity:

- Critical: System unusable, data loss, security breach - High: Major feature broken, significant user impact - Medium: Minor feature broken, workaround exists - Low: Cosmetic issues, minor improvements

1. Create backlog file in 10-sprints/:

Test Bug Backlog Template: ```markdown # Backlog: [Type] - [Brief Description]

## Type [change | bug | improve]

## Severity [critical | high | medium | low]

## Original Feature/Backlog Reference to original backlog that was tested

## Issue Description Clear description of the bug or issue

## Steps to Reproduce 1. Step-by-step instructions to reproduce 2. Include specific inputs/actions 3. Note environment details

## Expected Behavior What should happen

## Actual Behavior What actually happens

## Test Evidence - Screenshots - Log excerpts - Error messages - Performance metrics

## Affected Components - Files/functions involved - APIs or UI elements broken

## Reference Features Related features to consult

## Test Plan How to verify the fix works ```

1. Notify Project Management:

- Critical issues need immediate attention - High severity bugs should be prioritized - Medium/low can be batched

Step 8: Create Test Report

Document test results:

1. Test Summary:

- Total test cases executed - Passed vs Failed - Test coverage achieved - Time taken

1. Test Results by Type:

- Unit tests: X passed, Y failed - Integration tests: X passed, Y failed - API tests: X passed, Y failed - UI tests: X passed, Y failed - Manual tests: X passed, Y failed

1. Issues Found:

- Changes required: count - Bugs found: count - Improvements suggested: count - By severity breakdown

1. Test Decision:

- Passed: All tests pass, ready for production - Passed with minor issues: Non-critical improvements noted - Failed: Critical issues must be fixed before release - Blocked: Cannot test due to environment or dependency issues

Step 9: Finalize and Commit

CRITICAL: Follow this process to safely commit changes and update tracking:

1. Update Tracking Files:

- Update backlog.md: - Change status to Done after testing completes - Mark all checklist items (acceptance criteria/test plan) as [x] pass or [-] no test needed - Add "Testing Notes" section: - Test Summary: Passed/Failed counts - Issues Found: Bug backlogs created - Decision: Passed/Failed - Update feature documentation with test results - Update 10-sprints/.../README.md: - Update status in table - Add progress log entry - If all backlogs are Done, mark sprint-level checklist items as [x] pass or [-] no test needed - If sprint-level checks are complete, set sprint status to Completed

1. Request Human Review:

- Present the test results and plan to commit - Ask user: "Please review the results. If approved, I will commit and close the backlog." - Wait for approval.

1. Commit the Tests/Fixes (Content):

- Run git add . to stage all changes - Unstage the backlog file and sprint README (git reset HEAD <path-to-backlog> <path-to-sprint-readme>) - Check if there are staged changes: - If yes: - Draft conventional commit message (e.g., "test: add tests for [feature-name]") - Commit: git commit -m "test: ..." - Get Commit ID: git rev-parse --short HEAD - If no (only current backlog updated): - Skip to next step

1. Update Backlog with Commit ID:

- If a commit was made, append "Test Commit: [commit-id]" to the "Testing Notes" in backlog.md

1. Commit the Backlog (Metadata):

- Stage backlog.md and sprint README.md - Commit: git commit -m "docs([feature-name]): update backlog status to Done"

1. Notify user:

- Confirm completion - If Done: "Backlog closed successfully" - If Failed: "Backlog returned to development"

This two-step commit process ensures history is preserved before the backlog is updated with the commit reference.