nirholas/three.ws · Archived

gcp-triage

Monitor three.ws production on Google Cloud Run and fix what the sweep finds.

First seen Aug 15, 2026

Installation

$ npx skills add nirholas/three.ws --skill gcp-triage

Summary

  • Monitor three.ws production on Google Cloud Run and fix what the sweep finds.
  • Use when the user asks to check production, diagnose an outage or error, "what's wrong with three.ws", read production logs, or run the monitoring loop.
  • Broad questions get the deep sweep (healthz + all-service logs + version, TLS, fleet readiness, live pages, crons, DB migrations, wallets), then the fixes each class allows.

Stronger alternatives

This repository is archived — consider an actively maintained alternative.

Similar popular skills

Related neighbors and high-traction skills in the same topics — useful to compare before installing.

Also in this package

Other skills from nirholas/three.ws · top by installs.

npx skills add nirholas/three.ws

Browse all from nirholas/three.ws

More details

Agent compatibility

Declared targets from SKILL.md / docs. Unmarked agents are not listed — the skill may still install via the CLI.

Claude Code Declared
Cursor Not declared
Codex Not declared
GitHub Copilot Not declared
Windsurf Not declared
Gemini CLI Not declared
Cline Not declared
OpenCode Not declared

Repository health

Stars 95
License LICENSE
Default branch main
Open issues 7
Status Archived

Skill metadata

Parsed from SKILL.md frontmatter.

LicenseMIT
Declared agents claude-code
More metadata
category
ops/production
cross-platform-safe
pack
three-ws-ops

Package contents

Files included with this skill beyond the listing page.

  • skill md SKILL.md 6,933 B
  • docs SUMMARY.md 422 B

History

  1. First seen on skills.sh
  2. First recorded snapshot · 1 installs

SKILL.md

GCP production triage: monitor, classify, fix

The vercel logs era is over; production is 20 Cloud Run services in project aerial-vehicle-466722-p5 (region us-central1). This skill is the loop an agent runs to answer "is production healthy?" and to act on what it finds. gcloud is already authenticated in this workspace.

Step 1: run the monitor

When the user asks "what's wrong with three.ws?" (or anything equally broad), run the DEEP sweep; it checks everything, not just what happened to log:

npm run triage:gcp -- --json --deep --since 6h   # agents: always --json

For a quick pulse check mid-incident, the fast form is still fine:

npm run triage:gcp -- --json --since 1h

The base monitor merges three signals and classifies every distinct problem signature:

  1. https://three.ws/api/healthz: the platform's own subsystem roll-up

(database, cache, Helius, x402 ring, world, sniper).

  1. A WARNING+ log sweep across every Cloud Run service, fingerprinted so

repeats group into one finding.

  1. HTTP request logs: 5xx groups per route, 429s.

--deep adds nine concurrent read-only probes, each wrapping an existing standalone audit, normalized into the same findings stream:

probe what it proves wraps
version /api/version answers; deployed commit is in local history; deploy lag counted built-in
tls certs on three.ws + world.three.ws have >21 days left built-in
fleet every Cloud Run service's latest revision is Ready gcloud run services list
pages every page advertised in data/pages.json serves on the live site scripts/check-pages.mjs --base
cron-drift vercel.json crons match live Cloud Scheduler jobs scripts/check-cron-drift.mjs
cron-liveness every cron routes, resolves a handler, and loads scripts/audit-cron-liveness.mjs --static
db-migrations no migration is pending against the database scripts/apply-migrations.mjs --check
service-wallets signer wallets above SOL floors; advertised x402 keys match secrets scripts/audit-service-wallets.mjs
custodial-keys no funded custodial wallet sits behind an undecryptable key scripts/audit-custodial-key-health.mjs

A probe that cannot run becomes an investigate finding (a blind spot is not "healthy"); one skipped for missing local secrets is reported as skipped. --skip pages,custodial-keys drops probes when you must (e.g. re-running in a tight loop). The deep sweep takes a few minutes; that is the point.

Exit 0 = healthy or self-healing noise only. Exit 1 = findings[] contains actionable items. Each finding carries class, count, services, sample, and a concrete action. Deep findings carry signature: "deep-<probe>".

Surfaces even the deep sweep does not cover, worth running when the complaint points at them: npm run smoke:mcp (remote MCP endpoints), npm run smoke:x402-facilitator (facilitator verifies a real signed payment without broadcasting it, so it is safe to run any time), npm run audit:web or audit:web:login (real-browser page audit with the QA account).

Step 2: act per class, in this order

class what it means what you do
owner money / billing / security credential Do NOT act. Put the exact command from [docs/ops/production-log-triage.md](../../../docs/ops/production-log-triage.md) in your report.
env-action fix is a config-only Cloud Run env/resource change Apply it now. Config-only gcloud run services update is pre-approved (CLAUDE.md). ALWAYS --update-env-vars (merges); NEVER --set-env-vars (replaces the whole set).
investigate unknown signature or 5xx group Root-cause it (Step 3). Fix the code, add tests, commit locally. Deploys need owner approval, so prepare the one-command ship and say so in the report.
self-healing documented graceful degradation No action. Only escalate if the same finding persists across several runs (compare firstSeen, it should not span many hours).

Step 3: root-causing an investigate finding

  • Read the exact logs with the reader CLI (vercel logs equivalent):

``sh npm run logs -- -s <service> --errors --since 6h # errors, one service npm run logs -- -s <service> --app --warnings # app logs only npm run logs -- --all --grep "<term>" --since 1d # search the fleet npm run logs:tail # live tail three-ws-api ``

  • 5xx with no matching app-level error means the handler itself returned

5xx (upstream dependency reply), not a crash. Read the handler in api/ for what makes it return that status, and check which upstream it wraps.

  • Generation failures: the forge_creations table carries per-generation

backend/status/error/prompt (DATABASE_URL from the Cloud Run service env).

  • Crash signatures (Uncaught signal, OOM, no available instance):

check the revision, memory, and instance ceilings via gcloud run services describe <service> --region us-central1.

  • Rollback and LB/DNS/TLS runbook: [docs/ops/gcp-production.md](../../../docs/ops/gcp-production.md).

Step 4: close the loop

  • Fixed something in code? Commit locally with explicit paths (never

git add -A). Do not push or deploy without owner approval; leave the ship as one command in the report.

  • Found a NEW signature that is expected degradation (a fallback firing

correctly)? Add it to KNOWN_SIGNATURES in [scripts/gcp-triage.mjs](../../../scripts/gcp-triage.mjs) AND document it in [docs/ops/production-log-triage.md](../../../docs/ops/production-log-triage.md), so the monitor never flags it as investigate again. That is how this system learns.

  • Healthz unreachable? That is an outage: check three-ws-api revisions

(gcloud run revisions list --service three-ws-api --region us-central1) and the load balancer per the production runbook before anything else.

Report format

Lead with the verdict (healthy / degraded / outage). Then: what you fixed, what you committed, what needs the owner (exact commands), and what is self-healing noise. Solana-facing findings first.