From 6f8d349f7f9e8a4f0b5201ac89a2fb199206a332 Mon Sep 17 00:00:00 2001 From: Paul O'Reilly Date: Mon, 23 Mar 2026 17:35:40 +1300 Subject: [PATCH] Update README with full architecture diagrams and API walkthrough Expand from a brief summary to a comprehensive guide that an existing Octopus Deploy user can follow to adopt the dual-image channel pattern. Includes CI-to-deployment flow diagrams, step-by-step CI workflow explanation, API payload breakdown, and a portable curl example. Co-Authored-By: Claude Opus 4.6 (1M context) --- README.md | 423 ++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 413 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 37b3346..ccc9b26 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,419 @@ -# Demo App — Dual Image Channel Pattern +# Dual-Image Channel Pattern -Demonstrates Octopus Deploy's channel pattern for routing non-prod and prod container images through separate lifecycles. +A reference implementation for routing **two separate container registries** (non-prod and prod) through a single Octopus Deploy project using **channels**, **CI-driven release creation**, and **auto-promoting lifecycles**. -## How it works +## The Problem -- **Push to `main`** → builds non-prod image → pushes to `acme-nonprod/demo-app` → creates Octopus release in Non-Prod channel -- **Push a `v*` tag** → builds prod image → pushes to `acme-prod/demo-app` → creates Octopus release in Prod channel +Your CI pipeline builds two Docker images per commit — one for non-prod environments, one for prod — and pushes them to separate container registries. You need Octopus Deploy to: -Each channel has its own lifecycle: -- Non-Prod: Development → Staging (cannot reach Production) -- Prod: Staging → Production +- Create releases from **either** image independently (not require both) +- **Prevent** non-prod images from reaching Production +- Work with **SHA-based tags** (not SemVer) -## Octopus Space +## Architecture Overview -https://taniwha.octopus.app/app#/Spaces-102 +``` +┌─────────────────────────────────────────────────────────────────────┐ +│ Git Repository │ +│ │ +│ Push to main ─────────────┐ Push v* tag ─────────────┐ │ +└─────────────────────────────┼────────────────────────────┼─────────┘ + │ │ + ▼ ▼ +┌─────────────────────────────────────┐ ┌─────────────────────────────────┐ +│ CI: build-nonprod.yml │ │ CI: build-prod.yml │ +│ │ │ │ +│ 1. Build image │ │ 1. Build image │ +│ 2. Tag: sha-<8chars> │ │ 2. Tag: sha-<8chars> │ +│ 3. Push → Non-Prod Registry │ │ 3. Push → Prod Registry │ +│ 4. POST /api/releases │ │ 4. POST /api/releases │ +│ Channel: Non-Prod │ │ Channel: Prod │ +│ Version: sha-<8chars> │ │ Version: prod-sha-<8chars> │ +└──────────────┬──────────────────────┘ └──────────────┬──────────────────┘ + │ │ + ▼ ▼ +┌──────────────────────────────────────────────────────────────────────────┐ +│ Octopus Deploy │ +│ │ +│ ┌─────────────────────────┐ ┌─────────────────────────┐ │ +│ │ Non-Prod Channel │ │ Prod Channel │ │ +│ │ │ │ │ │ +│ │ Step: Deploy Non-Prod │ │ Step: Deploy Prod │ │ +│ │ Package: acme-nonprod │ │ Package: acme-prod │ │ +│ │ /demo-app │ │ /demo-app │ │ +│ └────────┬────────────────┘ └────────┬─────────────────┘ │ +│ │ │ │ +│ ▼ ▼ │ +│ ┌─────────────────────────┐ ┌─────────────────────────┐ │ +│ │ Non-Prod Lifecycle │ │ Prod Lifecycle │ │ +│ │ │ │ │ │ +│ │ Development ──auto──► │ │ Staging ──auto──► │ │ +│ │ Staging │ │ Production │ │ +│ │ │ │ │ │ +│ │ (no Production phase) │ │ │ │ +│ └─────────────────────────┘ └─────────────────────────┘ │ +│ │ +└──────────────────────────────────────────────────────────────────────────┘ +``` + +## How Each Piece Fits Together + +### 1. CI Workflows (the trigger) + +Two workflows run independently based on different git events: + +| Workflow | Trigger | Registry | Octopus Channel | +|----------|---------|----------|-----------------| +| `build-nonprod.yml` | Push to `main` | Non-Prod (`acme-nonprod/demo-app`) | Non-Prod | +| `build-prod.yml` | Push a `v*` tag | Prod (`acme-prod/demo-app`) | Prod | + +Each workflow: checks out code, builds a Docker image, tags it with the git SHA (`sha-<8chars>`), pushes to its registry, then calls the Octopus API to create a release. + +**Why CI calls the API directly (not feed triggers):** Octopus feed triggers poll for new packages every ~2-3 minutes, but they only detect SemVer-compatible tags. SHA-based tags like `sha-3e09802d` are invisible to the trigger mechanism. Calling `POST /api/{spaceId}/releases` from CI is instant and works with any version format. + +### 2. Release Creation (the API call) + +The CI workflow creates a release by POSTing to the Octopus API: + +``` +POST /api/{spaceId}/releases +``` + +```json +{ + "ProjectId": "Projects-101", + "ChannelId": "Channels-124", + "Version": "sha-3e09802d", + "SelectedPackages": [ + { + "ActionName": "Deploy Non-Prod Image", + "PackageReferenceName": "demo-app", + "Version": "sha-3e09802d" + } + ] +} +``` + +Key fields: +- **`ChannelId`** — routes the release to the correct channel (and therefore the correct lifecycle) +- **`Version`** — the release version in Octopus (can be any string — doesn't need to be SemVer) +- **`SelectedPackages`** — tells Octopus which package version to use for each step. `ActionName` must match the deployment step name exactly + +### 3. Channels (the router) + +Channels control two things: **which lifecycle** the release follows, and **which steps** run during deployment. + +``` +┌──────────────────────────────────────────────────────────┐ +│ Octopus Project │ +│ │ +│ ┌──────────────────┐ ┌──────────────────┐ │ +│ │ Non-Prod Channel │ │ Prod Channel │ │ +│ │ │ │ │ │ +│ │ Lifecycle: │ │ Lifecycle: │ │ +│ │ Non-Prod │ │ Prod │ │ +│ │ │ │ │ │ +│ │ Runs step: │ │ Runs step: │ │ +│ │ Deploy Non-Prod │ │ Deploy Prod │ │ +│ │ Image │ │ Image │ │ +│ │ │ │ │ │ +│ │ Skips step: │ │ Skips step: │ │ +│ │ Deploy Prod │ │ Deploy Non-Prod │ │ +│ │ Image │ │ Image │ │ +│ └──────────────────┘ └──────────────────┘ │ +│ │ +└──────────────────────────────────────────────────────────┘ +``` + +Each deployment step is **scoped to a single channel**. When a release is created in the Non-Prod channel, only the "Deploy Non-Prod Image" step runs — the prod step is skipped (and vice versa). This means each release only needs one package, not both. + +### 4. Lifecycles (the guardrail) + +Each channel has its own lifecycle that defines which environments the release can reach and whether deployment is automatic: + +``` +Non-Prod Lifecycle: Development ──auto──► Staging ──(stops here) +Prod Lifecycle: Staging ──auto──► Production +``` + +The Non-Prod lifecycle has **no Production phase** — this is what prevents non-prod images from ever reaching Production. It's not a permission or a rule that can be overridden; the lifecycle simply doesn't include Production as a valid target. + +Auto-deploy means releases flow through environments automatically once created — no manual approval needed. (You can add manual intervention steps or approvals if your process requires them.) + +### 5. Deployment Process (the execution) + +The project has two steps. Each step references a different package from a different feed (registry), and is scoped to its channel: + +| Step | Package | Feed (Registry) | Channel Scope | +|------|---------|-----------------|---------------| +| Deploy Non-Prod Image | `acme-nonprod/demo-app` | Non-Prod Registry | Non-Prod only | +| Deploy Prod Image | `acme-prod/demo-app` | Prod Registry | Prod only | + +In this demo, each step runs a script that logs the deployment details. In a real implementation, you'd replace these with your actual deployment steps (Kubernetes, Helm, cloud targets, etc.). + +## End-to-End Flow + +### Non-prod: developer pushes to main + +``` +git push origin main + │ + ▼ +CI: build-nonprod.yml + ├── Builds image + ├── Pushes acme-nonprod/demo-app:sha-a1b2c3d4 + └── POST /api/releases → Non-Prod channel, version "sha-a1b2c3d4" + │ + ▼ + Octopus creates release "sha-a1b2c3d4" in Non-Prod channel + │ + ▼ + Auto-deploys to Development (runs "Deploy Non-Prod Image" step only) + │ + ▼ + Auto-promotes to Staging + │ + ▼ + Stops. Production is not in this lifecycle. +``` + +### Prod: developer tags a release + +``` +git tag v1.2.0 && git push origin v1.2.0 + │ + ▼ +CI: build-prod.yml + ├── Builds image + ├── Pushes acme-prod/demo-app:sha-a1b2c3d4 + └── POST /api/releases → Prod channel, version "prod-sha-a1b2c3d4" + │ + ▼ + Octopus creates release "prod-sha-a1b2c3d4" in Prod channel + │ + ▼ + Auto-deploys to Staging (runs "Deploy Prod Image" step only) + │ + ▼ + Auto-promotes to Production +``` + +## Creating a Release via the API + +Octopus feed triggers poll registries every ~2-3 minutes looking for new packages, but they only recognise **SemVer-compatible tags**. SHA-based tags like `sha-a1b2c3d4` are invisible to the polling mechanism. The solution is to have CI call the Octopus REST API directly after pushing the image — it's instant, reliable, and works with any version format. + +### What the CI workflow does (step by step) + +Here's the non-prod workflow broken down. The prod workflow is identical except it targets the prod registry and prod channel. + +#### Step 1: Check out the code + +```yaml +- uses: actions/checkout@v4 +``` + +Standard git checkout — nothing special here. + +#### Step 2: Generate a version from the git SHA + +```yaml +- name: Generate version + id: version + run: echo "sha=sha-${GITHUB_SHA::8}" >> "$GITHUB_OUTPUT" +``` + +Takes the first 8 characters of the commit SHA and prefixes with `sha-`. This becomes both the **Docker image tag** and the **Octopus release version**. Using the commit SHA means every build produces a unique, traceable version — you can always map a running container back to the exact commit that produced it. + +The output is stored in `steps.version.outputs.sha` for use in later steps. + +#### Step 3: Build and push the image + +```yaml +- name: Login to registry + uses: docker/login-action@v3 + with: + registry: ${{ env.REGISTRY }} + username: ${{ secrets.REGISTRY_USERNAME }} + password: ${{ secrets.REGISTRY_PASSWORD }} + +- name: Build and push + run: | + docker build -t ${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ steps.version.outputs.sha }} . + docker push ${{ env.REGISTRY }}/${{ env.IMAGE }}:${{ steps.version.outputs.sha }} +``` + +Logs into the container registry, builds the image, and pushes it. The image tag is the SHA version from step 2 (e.g., `gitea.oreillyit.nz/acme-nonprod/demo-app:sha-a1b2c3d4`). + +The non-prod workflow pushes to the non-prod registry (`acme-nonprod/demo-app`), and the prod workflow pushes to the prod registry (`acme-prod/demo-app`). + +#### Step 4: Create the Octopus release + +This is the key step — instead of waiting for Octopus to poll and discover the new image, CI tells Octopus directly: + +```yaml +- name: Create Octopus Release + run: | + RESPONSE=$(curl -s -w "\n%{http_code}" -X POST \ + -H "X-Octopus-ApiKey: ${{ secrets.OCTOPUS_API_KEY }}" \ + -H "Content-Type: application/json" \ + "${{ env.OCTOPUS_SERVER }}/api/${{ env.OCTOPUS_SPACE_ID }}/releases" \ + -d '{ + "ProjectId": "${{ env.OCTOPUS_PROJECT_ID }}", + "ChannelId": "${{ env.OCTOPUS_CHANNEL_ID }}", + "Version": "${{ steps.version.outputs.sha }}", + "SelectedPackages": [ + { + "ActionName": "Deploy Non-Prod Image", + "PackageReferenceName": "demo-app", + "Version": "${{ steps.version.outputs.sha }}" + } + ] + }') + HTTP_CODE=$(echo "$RESPONSE" | tail -1) + BODY=$(echo "$RESPONSE" | sed '$d') + echo "Response: $BODY" + if [ "$HTTP_CODE" -ge 400 ]; then + echo "::error::Failed to create release (HTTP $HTTP_CODE)" + exit 1 + fi + echo "Release created successfully" +``` + +Let's break down the API payload: + +```json +{ + "ProjectId": "Projects-101", + "ChannelId": "Channels-124", + "Version": "sha-a1b2c3d4", + "SelectedPackages": [ + { + "ActionName": "Deploy Non-Prod Image", + "PackageReferenceName": "demo-app", + "Version": "sha-a1b2c3d4" + } + ] +} +``` + +| Field | Purpose | +|-------|---------| +| `ProjectId` | Which Octopus project to create the release in | +| `ChannelId` | Which channel to route through — this determines the lifecycle (and therefore which environments the release can reach) | +| `Version` | The release version shown in Octopus. Can be any string. For prod releases, we prefix with `prod-` (e.g., `prod-sha-a1b2c3d4`) to distinguish them in the release list | +| `SelectedPackages` | Tells Octopus which package version to use for each step in the deployment process | +| `ActionName` | Must **exactly match** the deployment step name in Octopus. If you rename the step in the UI, this must change too | +| `PackageReferenceName` | The package reference name within the step (set when you add the package to the step) | +| `Version` (in SelectedPackages) | The Docker image tag. Can differ from the release version — e.g., the prod release version is `prod-sha-a1b2c3d4` but the image tag is just `sha-a1b2c3d4` | + +The `curl` wrapper captures the HTTP status code separately so the workflow can fail the CI build if the API call returns an error (e.g., duplicate version, invalid channel, package not found in feed). + +### What happens after the API call + +Once the release is created, Octopus takes over automatically: + +1. **Release created** in the specified channel +2. **Lifecycle kicks in** — the channel's lifecycle determines where the release goes. If the first phase has auto-deploy enabled, deployment starts immediately +3. **Channel-scoped steps** — only the step scoped to this channel runs. The other step is skipped +4. **Auto-promotion** — after each phase completes successfully, the release automatically advances to the next phase in the lifecycle + +No further CI involvement needed. The release flows through environments on its own. + +### Adapting for your CI platform + +The API call is a simple `curl` POST. Here's a minimal example you can drop into any CI system: + +```bash +# Required variables — set these as CI secrets/env vars +OCTOPUS_SERVER="https://your-instance.octopus.app" +OCTOPUS_API_KEY="API-XXXXXXXXXXXX" +SPACE_ID="Spaces-1" +PROJECT_ID="Projects-123" +CHANNEL_ID="Channels-456" +VERSION="sha-$(git rev-parse --short=8 HEAD)" +STEP_NAME="Deploy My App" +PACKAGE_REF="my-app" +IMAGE_TAG="$VERSION" + +# Create the release +curl -sf -X POST \ + -H "X-Octopus-ApiKey: $OCTOPUS_API_KEY" \ + -H "Content-Type: application/json" \ + "$OCTOPUS_SERVER/api/$SPACE_ID/releases" \ + -d "{ + \"ProjectId\": \"$PROJECT_ID\", + \"ChannelId\": \"$CHANNEL_ID\", + \"Version\": \"$VERSION\", + \"SelectedPackages\": [{ + \"ActionName\": \"$STEP_NAME\", + \"PackageReferenceName\": \"$PACKAGE_REF\", + \"Version\": \"$IMAGE_TAG\" + }] + }" +``` + +You can find your resource IDs in the Octopus web UI — they appear in the URL when you navigate to each resource (e.g., `https://your-instance.octopus.app/app#/Spaces-1/projects/Projects-123`), or via the API at `/api/{spaceId}/projects`, `/api/{spaceId}/channels`, etc. + +## Adopting This Pattern + +To implement this in your own Octopus instance: + +### Octopus Setup + +1. **Create two Docker feeds** — one pointing to your non-prod registry, one to your prod registry. Both can be the same registry host if your images are in different paths/orgs. + +2. **Create two lifecycles:** + - Non-Prod: includes your lower environments only (e.g., Dev → Staging). **Do not include Production.** + - Prod: includes environments from your promotion point through Production (e.g., Staging → Production). + +3. **Create two channels** in your project: + - Non-Prod channel → assigned the Non-Prod lifecycle + - Prod channel → assigned the Prod lifecycle + - Remove any version rules from both channels (they require SemVer ranges and don't work with SHA tags) + +4. **Create two deployment steps**, each scoped to one channel: + - Step 1: references the non-prod package/feed, scoped to Non-Prod channel + - Step 2: references the prod package/feed, scoped to Prod channel + +### CI Setup + +5. **Store an Octopus API key** as a CI secret (`OCTOPUS_API_KEY`). + +6. **Add a release-creation step** to each CI workflow. After pushing the image, call: + + ```bash + curl -X POST \ + -H "X-Octopus-ApiKey: $OCTOPUS_API_KEY" \ + -H "Content-Type: application/json" \ + "https://your-octopus/api/{spaceId}/releases" \ + -d '{ + "ProjectId": "{projectId}", + "ChannelId": "{channelId}", + "Version": "{your-version-string}", + "SelectedPackages": [{ + "ActionName": "{step-name}", + "PackageReferenceName": "{package-ref}", + "Version": "{image-tag}" + }] + }' + ``` + + **Tip:** `Version` (release version) and `SelectedPackages[].Version` (image tag) can be different. This demo uses `prod-sha-xxx` for the prod release version to distinguish it from the non-prod `sha-xxx`, even though both use the same `sha-xxx` image tag. + +### Gotchas + +- **Feed triggers won't work with SHA tags.** Octopus's built-in feed trigger polling only detects SemVer-compatible versions. Use the API approach shown here instead. +- **Channel version rules require SemVer ranges.** If you're using SHA tags, remove all version rules and rely on channel-scoped steps for enforcement. +- **The Default channel can't be deleted.** Every Octopus project has one. Just leave it unused, or make one of your channels the default. +- **`ActionName` must match exactly.** The step name in `SelectedPackages` must match the deployment step name in your process. If you rename a step, update your CI workflow too. + +## Repository Structure + +``` +duel-image-demo/ + .gitea/workflows/ + build-nonprod.yml # CI: build + push non-prod image + create Octopus release + build-prod.yml # CI: build + push prod image + create Octopus release + Dockerfile # Minimal demo image (alpine + echo) + README.md # This file +```