ae-experiment

AE CLI (ae-cli) exposes Atlas AB Experiment capabilities through the experiment domain.

Global Rules

• Prefer ae-cli experiment <command> for Atlas AB Experiment work.
• Use --project-id / -p for project-scoped commands.
• Use --req JSON for complex save, status, and delete DTOs.
• Do not invent experiment IDs, traffic layer IDs, bucket IDs, Feature keys, metric IDs, or payload field names.
• Bind only metric IDs returned by experiment metric list; create and verify a missing metric before saving the experiment.
• Read commands can run directly after IDs are verified.
• Write commands require explicit user intent and normally keep the confirmation prompt. Use --dry-run before write calls when composing JSON.

Naming and response boundary:

• CLI command segments and flags use kebab-case.
• Outer Capability input and all response keys use snake_case.
• Nested business DTOs passed through --req keep their native camelCase fields.
• CRITICAL: save build-guide / save validate responses recursively snake_case

exampleargs.req. Never copy those keys into --req. Use camelCase (expName, metricId, …). Authoritative names: ae-cli capability inspect experiment.experiment.save (or the matching final save id) → inputschema.properties.req. save validate valid: true is not a final-save schema pass — snake_case req can still fail on experiment … save.

• Audience QP is semantic at the CLI boundary: write targeting.definitionRequest; read

targeting.definition_request. Never generate or submit targetConfig.

• Metric QP is semantic at the CLI boundary: write metricDefinition; read

metric_definition. Never generate or submit metricConfig, calcType, or Axxx codes.

• Event-property comparisons belong in metricDefinition.filters, using

{relation,items:[{field,operator,values}]}. Never put comparison fields directly on metricDefinition, and never submit raw filts, calcuSymbol, or ftv fields.

• For metric aggregations sum / avg / max / distinct_count, metricDefinition.property

is required and must be a concrete available event property from Analysis metadata. Never create property aggregations without property (for example metricavgnoproperty*).

• Preset event aggregations totalcount / usercount / active_days omit property.
• Experiment metric creation does not support avgperuser / A105.
• Resolve event and property names with Analysis metadata before saving semantic definitions.
• Lists return data.items and data.total; detail commands return data.item.
• Readiness returns data.readiness; reports return data.report; save guides return data.guide; save dry-run validation returns data.validation; writes return data.result.
• Query cancellation returns data.success.

Typical Workflow

1. Discover reusable assets:

- experiment bucket list - experiment traffic-layer list - experiment feature list - experiment metric list

1. Create missing assets if needed:

- experiment save build-guide --operation-mode savemetric when save validation fails or req shape is unclear - experiment save validate --operation-mode savemetric --req '{...}' before retrying a failed save - experiment traffic-layer save - experiment feature save - experiment metric save

1. Create or patch the experiment draft with experiment experiment save.

Use experiment experiment update-metrics when replacing metric bindings or assigning guardrail roles.

1. Check readiness with experiment experiment ready-check.
2. For a non-mutex traffic layer, run experiment experiment conflict-check before submit (needs featurekeylist from context or experiment get).
3. Move status with experiment experiment manage.
4. Query reports with experiment report summary, experiment report sample-size, and experiment report metric-trend.

If an experiment save returns errorcode: METRICNOTFOUND, list metrics for the same project. Create and verify the metric before retrying; never retry with another invented ID. Metric deletion returns errorcode: METRICINUSE while an active experiment binding exists.

Parameter Conventions

• Experiment save payloads distinguish two allocation fields: experiment-level req.allocation (integer only; no decimals) and group-level req.groups[].allocation (integer only; sum must equal 100 exactly).
• Public experiment metric roles are primary, secondary, and guardrail. The internal observation role is currently unavailable for saves. Guardrail is a binding role; updating metrics replaces the full binding list.

ae-cli experiment experiment get --project-id 1 --exp-id exp_123
ae-cli experiment experiment save --project-id 1 --req '{"expName":"Demo"}' --dry-run
ae-cli experiment metric save --project-id 1 --req '{"metricId":"payment_gold_sum","metricName":"Payment gold sum","createType":"event","goalDirection":"up","metricDesc":"Gold sum for payments above 10","metricDefinition":{"type":"event","event":"payment","aggregation":"sum","property":"gold","filters":{"relation":"and","items":[{"field":"gold","operator":"gt","values":[10]}]}}}' --dry-run
ae-cli experiment report metric-trend --project-id 1 --exp-id exp_123 --metric-id metric_1 --start-time 2026-07-01 --end-time 2026-07-07

Optional global parameters work as in other domains: --host, --mcp-url, --format, --jq, --dry-run, and --yes.

References

Open the matching file in references/ before using a command, especially for write operations and JSON payloads.

Save Helpers

experiment save build-guide, experiment save validate

When a save command returns next_tool: experiment.save.build-guide, call the guide first, then experiment save validate, then retry the final save capability.

Read [savebuildguide.md](references/savebuildguide.md) and [savevalidate.md](references/savevalidate.md) before using these helpers. Rebuild --req in camelCase from inspect / skill references; do not paste example_args.req.

Experiment

experiment experiment save, experiment experiment update-metrics, capability run experiment.experiment.save-submit, experiment experiment list, experiment experiment list-archived, experiment experiment get, experiment experiment ready-check, experiment experiment conflict-check, experiment experiment manage, experiment experiment update-group, experiment experiment batch-delete, experiment operation-log query

Read [manageguardrailmetrics.md](references/manageguardrailmetrics.md) before assigning or replacing metric roles.

Traffic Layer and Buckets

experiment traffic-layer save, experiment traffic-layer get, experiment traffic-layer list, experiment traffic-layer batch-delete, experiment bucket list

Reports

experiment report summary, experiment report sample-size, experiment report metric-trend, capability run experiment.query.cancel

Metric and Feature

experiment metric save, experiment metric get, experiment metric list, experiment metric delete, experiment feature save, experiment feature update-status, experiment feature get, experiment feature list, experiment feature version-list, experiment feature operation-log query, experiment feature batch-delete, experiment feature whitelist list, experiment feature whitelist save, experiment feature whitelist update-status, experiment feature whitelist batch-delete

Read [managefeaturewhitelist.md](references/managefeaturewhitelist.md) before querying or changing Feature whitelist rules.