Deploy App Workflow

End-to-end orchestration for deploying applications to the Kubernetes homelab with full monitoring integration.

Workflow Overview

┌─────────────────────────────────────────────────────────────────────┐
│                      /deploy-app Workflow                           │
├─────────────────────────────────────────────────────────────────────┤
│  1. RESEARCH                                                        │
│     ├─ Invoke kubesearch skill for real-world patterns              │
│     ├─ Check if native Helm chart exists (helm search hub)          │
│     ├─ Determine: native chart vs app-template                      │
│     └─ AskUserQuestion: Present findings, confirm approach          │
│                                                                     │
│  2. SETUP                                                           │
│     └─ task wt:new -- deploy-<app-name>                             │
│                                                                     │
│  3. CONFIGURE (in worktree)                                         │
│     ├─ kubernetes/platform/versions.env (add version)               │
│     ├─ kubernetes/platform/namespaces.yaml (add namespace)          │
│     ├─ kubernetes/platform/helm-charts.yaml (add input)             │
│     ├─ kubernetes/platform/charts/<app>.yaml (create values)        │
│     ├─ kubernetes/platform/kustomization.yaml (register)            │
│     └─ kubernetes/platform/config/<app>/ (optional extras)          │
│        ├─ route.yaml, canary.yaml, prometheus-rules.yaml            │
│                                                                     │
│  4. VALIDATE: task k8s:validate && task renovate:validate           │
│                                                                     │
│  5. TEST ON DEV (direct helm install, bypass Flux)                  │
│     ├─ Wait for pods ready, verify network + monitoring             │
│     └─ AskUserQuestion: Report status, confirm proceed              │
│                                                                     │
│  6. CLEANUP & PR                                                    │
│     ├─ helm uninstall, Flux reconcile-validate, commit, PR          │
│     └─ Report PR URL to user                                        │
└─────────────────────────────────────────────────────────────────────┘

Phase 1: Research

Invoke the kubesearch skill to find real-world chart configurations: /kubesearch <chart-name>

Check for a native chart: helm search hub <app-name> --max-col-width=100

Use AskUserQuestion to confirm: chart selection, exposure type (internal/external/none), namespace, persistence requirements.

Phase 2: Setup

Create an isolated worktree: task wt:new -- deploy-<app-name>

This creates branch deploy-<app-name> and worktree ../homelab-deploy-<app-name>/. Work exclusively in the worktree.

Phase 3: Configure

3.1 Add Version to versions.env

Add a version entry with a Renovate annotation. For annotation syntax and datasource selection, see the [versions-renovate skill](../versions-renovate/SKILL.md).

3.2 Add Namespace to namespaces.yaml

Add to kubernetes/platform/namespaces.yaml inputs array:

- name: <namespace>
  dataplane: ambient
  security: baseline       # restricted | baseline | privileged
  networkPolicy: false     # or object with profile/enforcement

PodSecurity level:

Network policy profile:

Access labels (add as needed):

    access.network-policy.homelab/postgres: "true"
    access.network-policy.homelab/garage-s3: "true"
    access.network-policy.homelab/kube-api: "true"

For PostgreSQL provisioning, see the [cnpg-database skill](../cnpg-database/SKILL.md).

For Garage S3 storage, see Section 3.8 below — the access.network-policy.homelab/garage-s3 label alone is not sufficient.

3.3 Add to helm-charts.yaml

Add to kubernetes/platform/helm-charts.yaml inputs array:

- name: "<app-name>"
  namespace: "<namespace>"
  chart:
    name: "<chart-name>"
    version: "${<APP>_VERSION}"
    url: "https://charts.example.com"  # or oci://registry.io/path
  dependsOn: [cilium]

3.4 Create Values File

Create kubernetes/platform/charts/<app-name>.yaml. See [references/file-templates.md](references/file-templates.md) for complete templates.

Security context for restricted namespaces (cert-manager, external-secrets, system, database, kromgo): add full restricted context to all containers. task k8s:validate does NOT catch PodSecurity violations — only admission time reveals them.

# Pod-level
podSecurityContext:       # key varies by chart
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

# Container-level (every container and init container)
securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    drop: ["ALL"]
  readOnlyRootFilesystem: true
  runAsNonRoot: true
  seccompProfile:
    type: RuntimeDefault

Check the image's default user — if it runs as root, add runAsUser: 65534.

3.5 Register in kustomization.yaml

Add to kubernetes/platform/kustomization.yaml configMapGenerator files list: - charts/<app-name>.yaml

3.6 Configure Renovate Tracking

Renovate tracks versions.env entries automatically via inline # renovate: annotations added in step 3.1. No changes to .github/renovate.json5 are needed unless adding grouping or automerge overrides. See the [versions-renovate skill](../versions-renovate/SKILL.md).

3.7 Optional: Additional Configuration

Create kubernetes/platform/config/<app-name>/ for extra resources. See [references/file-templates.md](references/file-templates.md) for HTTPRoute, secret, and ExternalSecret templates. See [references/monitoring-patterns.md](references/monitoring-patterns.md) for Canary, PrometheusRule, and Grafana dashboard examples.

For gateway routing and TLS, see the [gateway-routing skill](../gateway-routing/SKILL.md).

3.8 Garage S3 Object Storage

Garage S3 requires three resources across two locations. Missing any one causes an admission webhook denial.

Step 1: GarageBucket — in the cluster config, app namespace, under garage namespace:

# kubernetes/clusters/live/config/<app>/garage-bucket.yaml
apiVersion: garage.rajsingh.info/v1beta1
kind: GarageBucket
metadata:
  name: <app>
  namespace: garage
spec:
  clusterRef:
    name: garage

Step 2: GarageKey — in the cluster config, app namespace:

# kubernetes/clusters/live/config/<app>/garage-key.yaml
apiVersion: garage.rajsingh.info/v1beta1
kind: GarageKey
metadata:
  name: <app>
  namespace: <app>
spec:
  clusterRef:
    name: garage
    namespace: garage
  neverExpires: true
  bucketPermissions:
    - bucketRef:
        name: <app>
        namespace: garage
      read: true
      write: true
  secretTemplate:
    name: <app>-s3-credentials

Step 3: GarageReferenceGrant — PLATFORM-level, must go in kubernetes/platform/config/garage/:

# kubernetes/platform/config/garage/reference-grant-<app>.yaml
apiVersion: garage.rajsingh.info/v1beta1
kind: GarageReferenceGrant
metadata:
  name: allow-<app>
spec:
  from:
    - kind: GarageKey
      namespace: <app>

Also register it in kubernetes/platform/config/garage/kustomization.yaml. Without this, the Garage admission webhook (vgaragekey.kb.io) will deny the GarageKey with cross-namespace reference not permitted.

Namespace label (also required for network access):

access.network-policy.homelab/garage-s3: "true"

S3 env vars in the app (GarageKey generates the <app>-s3-credentials secret automatically):

S3_ACCESS_KEY_ID:
  valueFrom:
    secretKeyRef:
      name: <app>-s3-credentials
      key: access-key-id
S3_SECRET_ACCESS_KEY:
  valueFrom:
    secretKeyRef:
      name: <app>-s3-credentials
      key: secret-access-key
S3_BUCKET: <app>
S3_REGION: garage
S3_ENDPOINT: "http://${garage_s3_endpoint}"
S3_FORCE_PATH_STYLE: "true"

Phase 4: Validate

task k8s:validate → task renovate:validate. Fix all errors before proceeding.

Phase 5: Test on Dev

The dev cluster is a sandbox — iterate freely.

Deploy directly (suspend Flux first if needed: task k8s:flux-suspend -- <kustomization-name>):

helm install <app-name> <repo>/<chart> \
  -n <namespace> --create-namespace \
  -f kubernetes/platform/charts/<app-name>.yaml \
  --version <version>

kubectl -n <namespace> \
  wait --for=condition=Ready pod -l app.kubernetes.io/name=<app-name> --timeout=300s

For OCI charts, use oci://registry/<path>/<chart>. For iteration, use helm upgrade instead of helm install.

Verify network connectivity (CRITICAL — network policies are enforced):

kubectl port-forward -n kube-system svc/hubble-relay 4245:80 &
hubble observe --verdict DROPPED --namespace <namespace> --since 5m
hubble observe --from-namespace istio-gateway --to-namespace <namespace> --since 2m
hubble observe --from-namespace <namespace> --to-namespace database --since 2m

Common causes: missing profile label (gateway blocked), missing access label (database/S3 blocked), wrong profile (external API calls blocked).

Verify monitoring using the helper scripts:

.claude/skills/deploy-app/scripts/check-deployment-health.sh <namespace> <app-name>
.claude/skills/deploy-app/scripts/check-servicemonitor.sh <app-name>
.claude/skills/deploy-app/scripts/check-alerts.sh
.claude/skills/deploy-app/scripts/check-canary.sh <app-name>  # if canary created

Iterate until all checks pass. AskUserQuestion to report status and confirm proceeding.

Phase 6: Validate GitOps & PR

Uninstall direct helm install → reconcile via Flux → validate clean convergence:

helm uninstall <app-name> -n <namespace>
task k8s:reconcile-validate

If reconciliation fails, fix manifests and retry.

Commit using conventional commits format, push, and create the PR:

git push -u origin deploy-<app-name>
gh pr create --title "feat(k8s): deploy <app-name>" --body "..."

The worktree is kept until PR is merged. User cleans up with task wt:remove -- deploy-<app-name>.

Secrets Handling

For detailed workflows, see the [secrets skill](../secrets/SKILL.md).

App needs a secret?
├─ Random/generated → secret-generator annotation
│     secret-generator.v1.mittwald.de/autogenerate: "key"
├─ External service (OAuth, third-party API) → ExternalSecret → AWS SSM
└─ Unclear → AskUserQuestion: "Can this be randomly generated?"

Error Handling

References

• [File Templates](references/file-templates.md) - Copy-paste templates for all config files
• [Monitoring Patterns](references/monitoring-patterns.md) - ServiceMonitor, PrometheusRule, Canary examples
• [flux-gitops skill](../flux-gitops/SKILL.md) - ResourceSet patterns
• [app-template skill](../app-template/SKILL.md) - For apps without native charts
• [kubesearch skill](../kubesearch/SKILL.md) - Research workflow