QA Automation Specialist

Production validation specialist. Designs and executes smoke tests, SEO audits, visual regression checks, redirect verification, and analytics event validation against live deployments.

Activation Triggers

Use this skill when:

• A deployment just landed and you need to verify it works in production
• You need to validate redirect chains, broken links, or asset loading
• SEO elements need auditing (meta tags, structured data, sitemap, robots.txt)
• Visual regression testing is required (screenshot comparison between deploys)
• Analytics events (PostHog, GA4, Segment) need verification after changes
• Acceptance criteria need validation against a running environment
• You are building a smoke test suite for a deployment pipeline

Do NOT use this skill for:

• Writing unit or integration tests (use test-automation-expert)
• Performance/load testing (use site-reliability-engineer)
• Security scanning (use security-auditor)
• Building the feature itself (use the appropriate feature skill)

Core Capabilities

1. Production Smoke Tests

Design smoke tests that validate critical user journeys after deployment:

// smoke-test.spec.ts — Playwright-based production smoke suite
import { test, expect } from '@playwright/test';

const BASE_URL = process.env.SMOKE_URL || 'https://yoursite.com';

test.describe('Production Smoke Tests', () => {
  test('homepage loads with correct status and critical elements', async ({ page }) => {
    const response = await page.goto(BASE_URL);
    expect(response?.status()).toBe(200);

    // Core content renders
    await expect(page.locator('h1')).toBeVisible();
    await expect(page.locator('nav')).toBeVisible();
    await expect(page.locator('footer')).toBeVisible();

    // No console errors
    const errors: string[] = [];
    page.on('console', msg => {
      if (msg.type() === 'error') errors.push(msg.text());
    });
    await page.reload();
    await page.waitForLoadState('networkidle');
    expect(errors.filter(e => !e.includes('favicon'))).toHaveLength(0);
  });

  test('critical API endpoints respond', async ({ request }) => {
    const endpoints = ['/api/health', '/api/status'];
    for (const endpoint of endpoints) {
      const resp = await request.get(`${BASE_URL}${endpoint}`);
      expect(resp.ok(), `${endpoint} returned ${resp.status()}`).toBeTruthy();
    }
  });

  test('authentication flow works end-to-end', async ({ page }) => {
    await page.goto(`${BASE_URL}/login`);
    // Use a dedicated smoke-test account, never production credentials
    await page.getByLabel('Email').fill(process.env.SMOKE_USER!);
    await page.getByLabel('Password').fill(process.env.SMOKE_PASS!);
    await page.getByRole('button', { name: /sign in/i }).click();
    await expect(page).toHaveURL(/dashboard/);
  });
});

Smoke test design principles:

• Test user journeys, not implementation details
• Keep the suite under 2 minutes total runtime
• Use dedicated smoke-test accounts with minimal permissions
• Every smoke test must have a clear pass/fail signal — no ambiguity
• Run on every deployment, gate rollback on failure

2. Redirect Chain & Link Validation

#!/bin/bash
# validate-redirects.sh — Check redirect chains and broken links
set -euo pipefail

SITE_URL="${1:?Usage: validate-redirects.sh <base-url>}"
ERRORS=0

# Check redirect chains (max 2 hops allowed)
check_redirect() {
  local url="$1"
  local expected_final="$2"
  local hops
  hops=$(curl -sL -o /dev/null -w '%{num_redirects}' "$url")
  local final
  final=$(curl -sL -o /dev/null -w '%{url_effective}' "$url")

  if [ "$hops" -gt 2 ]; then
    echo "WARN: $url has $hops redirects (max 2)"
    ((ERRORS++))
  fi
  if [ "$final" != "$expected_final" ]; then
    echo "FAIL: $url -> $final (expected $expected_final)"
    ((ERRORS++))
  fi
}

# Check for broken internal links
check_links() {
  local url="$1"
  # Extract href values, filter to same-origin, check each
  curl -s "$url" | grep -oP 'href="\K[^"]+' | grep "^/" | sort -u | while read -r path; do
    local status
    status=$(curl -s -o /dev/null -w '%{http_code}' "${SITE_URL}${path}")
    if [ "$status" -ge 400 ]; then
      echo "BROKEN: ${path} -> HTTP ${status}"
      ((ERRORS++))
    fi
  done
}

# Check asset loading (CSS, JS, images)
check_assets() {
  local url="$1"
  curl -s "$url" | grep -oP '(src|href)="\K[^"]+' | grep -E '\.(css|js|png|jpg|svg|woff2?)' | while read -r asset; do
    local full_url
    [[ "$asset" == http* ]] && full_url="$asset" || full_url="${SITE_URL}${asset}"
    local status
    status=$(curl -s -o /dev/null -w '%{http_code}' "$full_url")
    if [ "$status" -ge 400 ]; then
      echo "MISSING ASSET: ${asset} -> HTTP ${status}"
      ((ERRORS++))
    fi
  done
}

echo "=== Redirect Validation ==="
check_redirect "${SITE_URL}/old-path" "${SITE_URL}/new-path"

echo "=== Link Validation ==="
check_links "${SITE_URL}"

echo "=== Asset Validation ==="
check_assets "${SITE_URL}"

exit $ERRORS

3. SEO Element Validation

// seo-audit.spec.ts
import { test, expect, Page } from '@playwright/test';

const BASE_URL = process.env.SMOKE_URL || 'https://yoursite.com';

async function auditSEO(page: Page, url: string) {
  await page.goto(url);

  // Title tag
  const title = await page.title();
  expect(title.length, 'Title should be 30-60 chars').toBeGreaterThanOrEqual(30);
  expect(title.length, 'Title should be 30-60 chars').toBeLessThanOrEqual(60);

  // Meta description
  const metaDesc = await page.getAttribute('meta[name="description"]', 'content');
  expect(metaDesc, 'Meta description must exist').toBeTruthy();
  expect(metaDesc!.length, 'Meta desc should be 120-160 chars').toBeGreaterThanOrEqual(120);
  expect(metaDesc!.length, 'Meta desc should be 120-160 chars').toBeLessThanOrEqual(160);

  // Open Graph
  const ogTitle = await page.getAttribute('meta[property="og:title"]', 'content');
  const ogDesc = await page.getAttribute('meta[property="og:description"]', 'content');
  const ogImage = await page.getAttribute('meta[property="og:image"]', 'content');
  expect(ogTitle, 'og:title must exist').toBeTruthy();
  expect(ogDesc, 'og:description must exist').toBeTruthy();
  expect(ogImage, 'og:image must exist').toBeTruthy();

  // Canonical URL
  const canonical = await page.getAttribute('link[rel="canonical"]', 'href');
  expect(canonical, 'Canonical URL must exist').toBeTruthy();
  expect(canonical, 'Canonical must be absolute').toMatch(/^https?:\/\//);

  // Heading hierarchy
  const h1Count = await page.locator('h1').count();
  expect(h1Count, 'Exactly one H1 per page').toBe(1);

  // Image alt text
  const imagesWithoutAlt = await page.locator('img:not([alt])').count();
  expect(imagesWithoutAlt, 'All images must have alt text').toBe(0);

  return { title, metaDesc, ogTitle, canonical };
}

test.describe('SEO Audit', () => {
  test('homepage SEO elements', async ({ page }) => {
    await auditSEO(page, BASE_URL);
  });

  test('sitemap.xml is valid', async ({ request }) => {
    const resp = await request.get(`${BASE_URL}/sitemap.xml`);
    expect(resp.ok()).toBeTruthy();
    const body = await resp.text();
    expect(body).toContain('<urlset');
    expect(body).toContain('<loc>');
    // Every loc should be reachable
    const locs = body.match(/<loc>([^<]+)<\/loc>/g) || [];
    expect(locs.length, 'Sitemap should have entries').toBeGreaterThan(0);
  });

  test('robots.txt is valid', async ({ request }) => {
    const resp = await request.get(`${BASE_URL}/robots.txt`);
    expect(resp.ok()).toBeTruthy();
    const body = await resp.text();
    expect(body).toContain('User-agent:');
    expect(body).toContain('Sitemap:');
    // Should not disallow critical paths
    expect(body).not.toMatch(/Disallow: \/$/m);
  });

  test('structured data is valid JSON-LD', async ({ page }) => {
    await page.goto(BASE_URL);
    const jsonLd = await page.locator('script[type="application/ld+json"]').allTextContents();
    expect(jsonLd.length, 'At least one JSON-LD block').toBeGreaterThan(0);
    for (const block of jsonLd) {
      const parsed = JSON.parse(block);
      expect(parsed['@context']).toContain('schema.org');
      expect(parsed['@type']).toBeTruthy();
    }
  });
});

4. Visual Regression Testing

// visual-regression.spec.ts
import { test, expect } from '@playwright/test';

const BASE_URL = process.env.SMOKE_URL || 'https://yoursite.com';

// Pages to capture for visual regression
const PAGES = [
  { name: 'homepage', path: '/' },
  { name: 'pricing', path: '/pricing' },
  { name: 'blog-index', path: '/blog' },
  { name: 'docs', path: '/docs' },
];

const VIEWPORTS = [
  { name: 'desktop', width: 1280, height: 800 },
  { name: 'tablet', width: 768, height: 1024 },
  { name: 'mobile', width: 375, height: 812 },
];

for (const pageConfig of PAGES) {
  for (const viewport of VIEWPORTS) {
    test(`visual: ${pageConfig.name} @ ${viewport.name}`, async ({ page }) => {
      await page.setViewportSize({ width: viewport.width, height: viewport.height });
      await page.goto(`${BASE_URL}${pageConfig.path}`);
      await page.waitForLoadState('networkidle');

      // Hide dynamic content that changes between runs
      await page.evaluate(() => {
        document.querySelectorAll('[data-testid="timestamp"], .live-counter')
          .forEach(el => (el as HTMLElement).style.visibility = 'hidden');
      });

      await expect(page).toHaveScreenshot(
        `${pageConfig.name}-${viewport.name}.png`,
        { maxDiffPixelRatio: 0.01, fullPage: true }
      );
    });
  }
}

Visual regression workflow:

1. First run generates baseline screenshots (commit to repo or artifact store)
2. Subsequent runs compare against baselines
3. Threshold: 1% pixel difference tolerance for anti-aliasing/rendering variance
4. Mask dynamic content (timestamps, counters, avatars) before capture
5. Review failures manually -- not all pixel changes are bugs

5. Analytics Event Validation

// analytics-validation.spec.ts
import { test, expect, Page } from '@playwright/test';

const BASE_URL = process.env.SMOKE_URL || 'https://yoursite.com';

interface AnalyticsEvent {
  event: string;
  properties: Record<string, unknown>;
}

async function captureAnalyticsEvents(page: Page): Promise<AnalyticsEvent[]> {
  const events: AnalyticsEvent[] = [];

  // Intercept PostHog capture calls
  await page.route('**/e/?ip=1', async (route) => {
    const postData = route.request().postDataJSON();
    if (postData?.batch) {
      for (const item of postData.batch) {
        events.push({ event: item.event, properties: item.properties });
      }
    }
    await route.fulfill({ status: 200, body: '1' });
  });

  return events;
}

test.describe('Analytics Event Validation', () => {
  test('page view fires on navigation', async ({ page }) => {
    const events = await captureAnalyticsEvents(page);
    await page.goto(BASE_URL);
    await page.waitForTimeout(2000); // Allow batch to flush

    const pageViews = events.filter(e => e.event === '$pageview');
    expect(pageViews.length).toBeGreaterThanOrEqual(1);
    expect(pageViews[0].properties).toHaveProperty('$current_url');
  });

  test('CTA click fires custom event', async ({ page }) => {
    const events = await captureAnalyticsEvents(page);
    await page.goto(BASE_URL);
    await page.getByRole('link', { name: /get started/i }).click();
    await page.waitForTimeout(2000);

    const ctaEvents = events.filter(e => e.event === 'cta_clicked');
    expect(ctaEvents.length).toBeGreaterThanOrEqual(1);
    expect(ctaEvents[0].properties).toHaveProperty('cta_location');
  });

  test('no duplicate event IDs in a single session', async ({ page }) => {
    const events = await captureAnalyticsEvents(page);
    await page.goto(BASE_URL);
    await page.goto(`${BASE_URL}/pricing`);
    await page.goto(`${BASE_URL}/blog`);
    await page.waitForTimeout(2000);

    const ids = events.map(e => e.properties?.['$insert_id']).filter(Boolean);
    const uniqueIds = new Set(ids);
    expect(uniqueIds.size, 'No duplicate event IDs').toBe(ids.length);
  });
});

Decision Points

When to run smoke tests vs. full regression

When to block deployment vs. alert

• Block: Broken critical paths (login, checkout, core API), missing assets, 5xx errors
• Alert: SEO regressions, minor visual diffs, missing analytics events, slow load times
• Log only: Cosmetic differences under threshold, deprecation warnings

Anti-Patterns

1. Testing against mocked production

Symptom: Smoke tests hit a staging environment and call it "production validation" Why wrong: Staging hides DNS issues, CDN caching, environment variable differences, and database state Fix: Run smoke tests against the actual production URL, with a dedicated smoke-test account

2. Flaky selectors in smoke tests

Symptom: Tests break on every deploy because they target CSS classes or DOM structure Fix: Use accessible selectors (getByRole, getByLabel, data-testid) that survive refactors

3. Screenshot comparison without masking

Symptom: Visual regression tests fail on every run due to timestamps, avatars, or ad content Fix: Mask dynamic regions before screenshot capture. Use data-testid to identify volatile elements.

4. Validating analytics by checking the dashboard

Symptom: "I'll check PostHog tomorrow to see if events came through" Fix: Intercept network requests in tests and validate event shape and presence in real time

5. One enormous smoke test file

Symptom: A single 500-line test file that tests everything and takes 10 minutes Fix: Separate by concern (smoke, SEO, visual, analytics). Run them in parallel. Gate on smoke, alert on the rest.

6. Hardcoded URLs and credentials

Symptom: const url = 'https://mysite.com'; scattered through test files Fix: Environment variables for everything. Never commit credentials. Use a .env.smoke file.

7. Ignoring HTTP cache headers

Symptom: Tests pass but users see stale content because cache headers are wrong Fix: Validate Cache-Control, ETag, and Last-Modified headers on critical assets

Quality Checklist

• Smoke test suite covers all critical user journeys (login, core feature, payment if applicable)
• Smoke suite completes in under 2 minutes
• Redirect chain validation covers all known legacy URLs
• No broken internal links on key pages
• All critical assets load (CSS, JS, fonts, images)
• SEO elements present and within length guidelines on all indexed pages
• Structured data (JSON-LD) is valid and contains required fields
• sitemap.xml is reachable and contains current URLs
• robots.txt does not block critical paths
• Visual regression baselines are up to date
• Analytics events fire correctly for tracked interactions
• No duplicate analytics event IDs within a session
• Smoke test accounts use minimal permissions (not admin)
• All test configuration is via environment variables (no hardcoded secrets)
• Tests run in CI on every deploy and gate rollback on critical failures