TestRail Integration

Bidirectional sync between Playwright tests and TestRail test management.

Prerequisites

Environment variables must be set:

• TESTRAIL_URL — e.g., https://your-instance.testrail.io
• TESTRAIL_USER — your email
• TESTRAILAPIKEY — API key from TestRail

If not set, inform the user how to configure them and stop.

The TestRail MCP server is not auto-registered (issue #978). pw-testrail
was removed from the plugin's .mcp.json because it failed to connect for every
user (the plugin ships no nodemodules). The testrail* MCP tools used below,
and the /pw:testrail command, will fail with "tool not found" until it is
enabled manually — see the Integrations section of the plugin's CLAUDE.md
(cd integrations/testrail-mcp && npm install, then register the server in your
own user/project MCP config). Setting the env vars alone is not sufficient.

Capabilities

1. Import Test Cases → Generate Playwright Tests

/pw:testrail import --project <id> --suite <id>

Steps:

1. Call testrailgetcases MCP tool to fetch test cases
2. For each test case:

- Read title, preconditions, steps, expected results - Map to a Playwright test using appropriate template - Include TestRail case ID as test annotation: test.info().annotations.push({ type: 'testrail', description: 'C12345' })

1. Generate test files grouped by section
2. Report: X cases imported, Y tests generated

2. Push Test Results → TestRail

/pw:testrail push --run <id>

Steps:

1. Run Playwright tests with JSON reporter:

``bash npx playwright test --reporter=json > test-results.json ``

1. Parse results: map each test to its TestRail case ID (from annotations)
2. Call testrailaddresult MCP tool for each test:

- Pass → statusid: 1 - Fail → statusid: 5, include error message - Skip → status_id: 2

1. Report: X results pushed, Y passed, Z failed

3. Create Test Run

/pw:testrail run --project <id> --name "Sprint 42 Regression"

Steps:

1. Call testrailaddrun MCP tool
2. Include all test case IDs found in Playwright test annotations
3. Return run ID for result pushing

4. Sync Status

/pw:testrail status --project <id>

Steps:

1. Fetch test cases from TestRail
2. Scan local Playwright tests for TestRail annotations
3. Report coverage:

`` TestRail cases: 150 Playwright tests with TestRail IDs: 120 Unlinked TestRail cases: 30 Playwright tests without TestRail IDs: 15 ``

5. Update Test Cases in TestRail

/pw:testrail update --case <id>

Steps:

1. Read the Playwright test for this case ID
2. Extract steps and expected results from test code
3. Call testrailupdatecase MCP tool to update steps

MCP Tools Used

Test Annotation Format

All Playwright tests linked to TestRail include:

test('should login successfully', async ({ page }) => {
  test.info().annotations.push({
    type: 'testrail',
    description: 'C12345',
  });
  // ... test code
});

This annotation is the bridge between Playwright and TestRail.

Output

• Operation summary with counts
• Any errors or unmatched cases
• Link to TestRail run/results