run-e2e

Run a single new-e2e target with dda inv -- new-e2e-tests.run. Most targets provision real infrastructure in a cloud account — usually AWS, sometimes GCP or Azure — though the framework also has local provisioners that cost nothing but time. Run duration is a property of the target: minutes for one VM, considerably longer for a Kubernetes cluster. Either way, aim for one correct run rather than a fast iteration loop.

Step 1 — Get the target, or ask

A target is a package path relative to /test/new-e2e/, like ./tests/agent-subcommands/flare; the task resolves against that module, so repeating the prefix is wrong.

Requests usually name a test, not a package — "run the flare e2e test" is the normal shape. Use a target if given, otherwise ask, offering candidates you already know. Never search the tree: a guessed target provisions the wrong thing and you find out after paying for it.

Anchor a supplied test name (--run '^TestFlareSuite$'), or TestFlare also selects TestFlareOpts.

Step 2 — Decide where it runs

test -f /.started && echo IN_DEVENV || echo ON_HOST

The dev env entrypoint creates /.started. ONHOST → step 3A, INDEVENV → step 3B, --host → 3C.

Step 3A — On the host (the usual case)

python .agents/skills/run-e2e/scripts/devenv_e2e.py up --json

Starts a dev env at id e2e-run if needed, gives it the host's E2E config and keypair, establishes Pulumi's backend, checks AWS access, and prints the run_prefix for step 5. Idempotent, so a reused env pays the setup cost once. Add --no-aws-check for a locally-provisioned target to skip the SSO acceptance; the host still needs AWS config, because the run task requires it whatever the target is.

The AWS check needs the user present — authorizing a new container means completing an SSO flow whose browser tab opens on their desktop. Warn them, and if it gives up, relay the aws-vault login it prints and rerun up. That is normal on a new env.

Every failure prints an actionable message; relay it. The table picks the reference file and says which machine the remedy belongs on.

Azure and GCP targets are not handled — only AWS credentials reach the container. Use --host.

Step 3B — Already inside a dev env

test -f ~/.test_infra_config.yaml && echo CONFIG_OK || echo CONFIG_MISSING
pulumi whoami >/dev/null 2>&1 && echo BACKEND_OK || echo BACKEND_MISSING

BACKENDMISSING → PULUMISKIPUPDATECHECK=true dda inv -- e2e.setup --no-interactive. Do not install Pulumi; the image ships it, only its plugins and backend are missing. CONFIGMISSING → stop; this env has no E2E identity, so have them recreate it with devenve2e.py up from the host. Never the interactive dda inv -- e2e.setup here — see references/setup.md.

This container needs its own AWS authorization, which nothing on the host provides. Have them run aws-vault login sso-agent-sandbox-account-admin-8h first rather than discovering it ten minutes in.

Then step 4 with a suffix identifying them, because stack names take the container user name dd and would otherwise collide. git config user.email is set from the host.

E2E_STACK_NAME_SUFFIX=<you> dda inv -- new-e2e-tests.run --targets=<target> [--run <regex>] [flags]

Step 3C — --host escape hatch

Check ~/.testinfraconfig.yaml, pulumi whoami, and a live AWS session — here the host's own counts — then run dda inv -- new-e2e-tests.run directly. Faster on a configured Linux or macOS machine, and Pulumi state survives there. Not the default because an unconfigured or Windows host fails in ways the dev env does not.

Step 4 — Confirm before provisioning

Get an explicit yes for: the exact command, target and --run, where it runs, what it provisions and in which account, roughly how long, and whether the stack is destroyed afterwards. If you cannot tell what it provisions, say so — that is worth confirming before paying for it.

Step 5 — Run it

Use the run_prefix from step 3A verbatim and append the test command. Do not add container paths of your own: on Windows, Git Bash rewrites them before they reach the container, and everything the run needs is already in the config the bootstrap installed.

dda env dev run -t linux-container --id e2e-run -- env <env_args...> \
  dda inv -- new-e2e-tests.run --targets=<target> [--run <regex>] [flags]

Start it with runinbackground: true; these outlast a foreground Bash call. The first run in a fresh env is much the slowest — the test binary compiles from a cold cache before any infrastructure is touched, so several minutes of silence is normal.

Step 6 — Report

### E2E run — <target> [--run <regex>]
- Where:       dev env `e2e-run` | host
- Command:     <exact command as executed>
- Result:      PASS | FAIL | SETUP FAILURE (failed before any test ran)
- Duration:    <mm:ss>
- Stack:       <name> — destroyed | kept (--keep-stack)
- Failures:
  - <TestSuite/TestName> — <one-line reason>
- Diagnostics: <path>   (+ the `docker cp` to retrieve it, if it ran in a dev env)
- Next step:   <the single most useful action>

For a SETUP FAILURE, take the symptom to references/troubleshooting.md rather than reporting raw stderr.

Step 7 — Tear down

The env is reusable and costs nothing idle, so leave it unless asked. When asked:

python .agents/skills/run-e2e/scripts/devenv_e2e.py down

It exits 6 rather than removing an env whose Pulumi stacks are live or uncheckable — that state exists nowhere else, so an orphaned cluster is the cost of getting this wrong. It prints the destroy command. --force overrides it; only reach for that once you have confirmed nothing is running. --keep-stack implies keeping the env too.

Examples

"run TestVMSuite in ./examples" — the default path, from a host that may not be configured

python .agents/skills/run-e2e/scripts/devenv_e2e.py up --json
dda env dev run -t linux-container --id e2e-run -- env E2E_STACK_NAME_SUFFIX=alice \
  dda inv -- new-e2e-tests.run --targets=./examples --run='^TestVMSuite$'

"just run it here, my machine is already set up" — --host, skipping the container

dda inv -- new-e2e-tests.run --targets=./examples --run='^TestVMSuite$'