CI/CD Master

Advisory guidance for robust and secure CI/CD pipelines in the SAPPHIRE project.

Role: Review, advise, and suggest improvements. Does not execute server commands directly or indirectly.

Project CI/CD Architecture

Developer → GitHub Actions → Docker Hub → AWS Server → Local Hydromet Servers
              │
              ├── build_test.yml (on push/PR)
              │   └── Test builds, unit tests
              │
              └── deploy_main.yml (on merge to main)
                  └── Build, sign, push images with attestations

GitHub Actions Security Checklist

When reviewing or editing workflow files, verify:

• Pinned action versions (@v4 not @main)
• Minimal permissions: scope (prefer read over write)
• Secrets via ${{ secrets.* }} never hardcoded
• No command injection via ${{ github.event.* }} in run:
• Timeout set for long-running jobs (timeout-minutes:)
• Concurrency controls for expensive jobs

Current Security Features

Workflow Best Practices

Job Dependencies

jobs:
  test:
    runs-on: ubuntu-latest
  build:
    needs: test  # Only runs if test passes

Disk Space for Large Builds

ML image builds (~4GB) need disk cleanup:

- name: Free disk space
  run: |
    sudo rm -rf /usr/share/dotnet
    sudo rm -rf /opt/ghc
    sudo rm -rf /usr/local/share/boost
    sudo rm -rf /usr/local/lib/android

Supply Chain Security

- uses: docker/build-push-action@v6
  with:
    provenance: true
    sbom: true

Deployment Script Review (bin/)

When reviewing scripts in bin/, check for:

• Proper error handling (set -e or equivalent)
• No hardcoded credentials (use environment variables)
• Logging for debugging
• Idempotency (safe to run multiple times)
• Clear documentation of prerequisites

Task Scheduling Guidance

Cron Syntax Reference

┌───────────── minute (0-59)
│ ┌───────────── hour (0-23)
│ │ ┌───────────── day of month (1-31)
│ │ │ ┌───────────── month (1-12)
│ │ │ │ ┌───────────── day of week (0-6, Sun=0)
│ │ │ │ │
* * * * *

# Examples
0 6 * * *    # Daily at 6:00 AM
0 */6 * * *  # Every 6 hours
0 6 1 * *    # First day of month at 6:00 AM

Systemd Timers (recommended over cron)

• More reliable with Persistent=true (runs missed jobs)
• Better logging via journalctl
• Dependency management with other services

Common Issues & Solutions