Migrate best practices to skynet/best-practices repo
Remove all best-practices topic files and .distill-state.json from claude-foundations. BESTPRACTICES.md now redirects to the standalone repo at ~/dev/claude/projects/best-practices/. Remove the now-unnecessary sync-best-practices.sh script. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Paul O'Reilly
@@ -1,31 +1,9 @@
|
||||
> **Note:** Best practices are now maintained in [skynet/best-practices](https://gitea.oreillyit.nz/skynet/best-practices).
|
||||
> AI agents with /best-practices mounted should read /best-practices/INDEX.md instead.
|
||||
> The content below is kept as a local copy for context-load compatibility.
|
||||
# Best Practices
|
||||
|
||||
# Best Practices Index
|
||||
Best practices have moved to their own repository: **[skynet/best-practices](https://gitea.oreillyit.nz/skynet/best-practices)**
|
||||
|
||||
Generalised best practices extracted from real project work. Each topic file is self-contained — read only the files relevant to the current project.
|
||||
- **Local clone:** `~/dev/claude/projects/best-practices/`
|
||||
- **Index:** `~/dev/claude/BESTPRACTICES.md` (symlink to the clone)
|
||||
- **Container agents:** cloned to `/best-practices` by the entrypoint
|
||||
|
||||
## Topics
|
||||
|
||||
- [Validation & Deployment](best-practices/validation.md) — Validate locally, deploy once; full-chain testing; pre-flight checks
|
||||
- [Security Architecture](best-practices/security-architecture.md) — Server boundary rule: no credential crosses to the client; proxy + identity mapping pattern; defense in depth; anti-patterns
|
||||
- [Secrets Management](best-practices/secrets-management.md) — SOPS + age, credential handling, file naming, encryption gotchas
|
||||
- [Git & Source Control](best-practices/git-source-control.md) — Commit practices, GitOps workflows, remote conventions
|
||||
- [Kubernetes Patterns](best-practices/kubernetes.md) — Volume mounts, deployment strategies, naming, bootstrap ordering
|
||||
- [Helm Charts](best-practices/helm.md) — Schema validation, version verification, values structure
|
||||
- [Ansible](best-practices/ansible.md) — Inventory, templates, idempotency, credential safety
|
||||
- [Scripting](best-practices/scripting.md) — Shell conventions, verification scripts, idempotency, colour output
|
||||
- [Documentation Standards](best-practices/documentation.md) — CLAUDE.md, MEMORY.md, FUTURE.md, README.md structure and tiered memory
|
||||
- [Milestones & Reflections](best-practices/milestones.md) — Milestone workflow, verification, reflection process
|
||||
- [Debugging Methodology](best-practices/debugging.md) — Systematic diagnosis, full-chain testing, common pitfalls
|
||||
- [Claude Code Skills](best-practices/skills-development.md) — Skill authoring, context injection, tool restrictions
|
||||
- [Linting & Formatting](best-practices/linting.md) — Tool choices per language, PostToolUse hook, pre-commit integration, formatter contract
|
||||
- [Spec-Driven Development](best-practices/spec-driven-development.md) — Spec structure, requirement numbering, test-first workflow, context tiers, anti-patterns
|
||||
- [Test-Driven Development](best-practices/test-driven-development.md) — Edge case discovery, property-based testing, mutation testing, AI agent testing patterns, test architecture
|
||||
- [Networking & Infrastructure](best-practices/networking.md) — nftables safety, systemd socket activation, Docker forwarding, TLS SNI vs Host header, wildcard certs
|
||||
- [Docker UID Matching](best-practices/docker-uid-matching.md) — UID wrapper entrypoint for mounted volumes, gosu pattern, when to use vs K8s securityContext
|
||||
- [Database Selection](best-practices/database-selection.md) — SQLite is not a production database; always use PostgreSQL for services with FQDNs, multiple consumers, or concurrent access
|
||||
- [Docker](best-practices/docker.md) — gosu PID 1, GIT_SSH_COMMAND scope, slim image health checks, buildx local images, default users, TTY flags, UID resolution
|
||||
- [Octopus Process Templates](best-practices/octopus-process-templates.md) — OCL syntax, step template references, channel scoping, parameters, versioning, Platform Hub patterns
|
||||
- [Platform Hub Knowledge Base](PLATFORMHUB.md) — Comprehensive guide: architecture, OCL syntax, template patterns, gotchas, API reference
|
||||
Read `BESTPRACTICES.md` in the best-practices repo for the full topic index.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# claude-foundations
|
||||
|
||||
Shared infrastructure for Claude Code sessions: hooks, formatters, best practices, and the knowledge distillation pipeline.
|
||||
Shared infrastructure for Claude Code sessions: hooks, formatters, and the knowledge distillation pipeline.
|
||||
|
||||
## Repository Structure
|
||||
|
||||
@@ -25,11 +25,10 @@ claude-foundations/
|
||||
setup-formatters.sh # Set up formatters for a project
|
||||
statusline.sh # Status line renderer (symlinked from ~/.claude/status/)
|
||||
set-topic.sh # Set per-session topic for the status line
|
||||
best-practices/ # Generalised best practices (one file per topic) — also maintained in skynet/best-practices
|
||||
context/ # Active work focus detail files
|
||||
memory/ # Session logs and reflections
|
||||
settings.yaml # Knowledge pipeline configuration
|
||||
BESTPRACTICES.md # Best practices index (loaded by context-load)
|
||||
BESTPRACTICES.md # Redirect to skynet/best-practices repo
|
||||
CLAUDE.md # Global project guidelines (symlinked to ~/dev/claude/)
|
||||
CONTEXT.md # Active work focus index (loaded by context-load)
|
||||
HOOKS.md # Hook documentation
|
||||
|
||||
@@ -1,41 +0,0 @@
|
||||
{
|
||||
"version": 1,
|
||||
"last_run": "2026-03-25T12:00:00Z",
|
||||
"projects": {
|
||||
"agent-runtimes": {
|
||||
"path": "/home/paul/dev/claude/projects/agent-runtimes",
|
||||
"last_sha": "d40ea0b39140077a482ce9ae304fad1cf51c832e",
|
||||
"last_run": "2026-03-25T12:00:00Z"
|
||||
},
|
||||
"claude-foundations": {
|
||||
"path": "/home/paul/dev/claude/projects/claude-foundations",
|
||||
"last_sha": "6c0f2db169b588afa9c92587eef3b64311acc025",
|
||||
"last_run": "2026-03-25T12:00:00Z"
|
||||
},
|
||||
"cluster-apps/octopus-deploy": {
|
||||
"path": "/home/paul/dev/claude/projects/cluster-apps/octopus-deploy",
|
||||
"last_sha": "d6aee319f33853ad58ea304d9ea1deedff8444d0",
|
||||
"last_run": "2026-03-25T12:00:00Z"
|
||||
},
|
||||
"cluster-bootstrap": {
|
||||
"path": "/home/paul/dev/claude/projects/cluster-bootstrap",
|
||||
"last_sha": "edc48d7a3113c049786cf6de74b2f33f101f39ef",
|
||||
"last_run": "2026-03-25T12:00:00Z"
|
||||
},
|
||||
"custom-claude-skills": {
|
||||
"path": "/home/paul/dev/claude/projects/custom-claude-skills",
|
||||
"last_sha": "e8e572e0afdcf3895f4a500ec73ec6d95d1cc19c",
|
||||
"last_run": "2026-03-25T12:00:00Z"
|
||||
},
|
||||
"hugo-accelerator": {
|
||||
"path": "/home/paul/dev/claude/projects/hugo-accelerator",
|
||||
"last_sha": "bf0b1a8bc7ed7964a724a2b8ee6c15b6dc1c926c",
|
||||
"last_run": "2026-03-25T12:00:00Z"
|
||||
},
|
||||
"small-scripts": {
|
||||
"path": "/home/paul/dev/claude/small-scripts",
|
||||
"last_sha": "161c633b42c5535b1c649487a330b23a9fa11626",
|
||||
"last_run": "2026-03-25T12:00:00Z"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,36 +0,0 @@
|
||||
# Ansible
|
||||
|
||||
## Inventory and Execution
|
||||
|
||||
- Always pass `-i inventory.yml` explicitly or run from the directory containing `ansible.cfg`
|
||||
- Playbooks that can't find inventory skip silently with no error — a common source of "it ran but nothing happened" confusion
|
||||
- Variables that need customisation go in `inventory.yml` files, not scattered across role defaults
|
||||
|
||||
## Role Structure
|
||||
|
||||
- Roles follow standard structure: `tasks/main.yml`, `templates/*.j2`, `handlers/main.yml`
|
||||
- Jinja2 templates have `.j2` extension and include a "managed by Ansible" header comment
|
||||
|
||||
## Template Safety
|
||||
|
||||
- **Never use placeholder values with `-e` for vars that template config files.** Using `-e "var=dummy"` will overwrite live configs with garbage. Either read real values, use `--skip-tags` to skip templating tasks, or restructure roles so sensitive templates are in a separate tag.
|
||||
|
||||
## Credential Safety
|
||||
|
||||
- Pass secrets via `@file` not `-e` on the command line — `-e "key=value"` exposes secrets in `ps` output
|
||||
- Use temp files with `trap rm` cleanup: `-e "@${tmpfile}"`
|
||||
|
||||
## Module Gotchas
|
||||
|
||||
- `docker_compose_v2` doesn't support `state: restarted` — use `recreate: always` instead
|
||||
- `ansible.builtin.unarchive` with `remote_src` and `extra_opts: --strip-components` is unreliable — use `get_url` + `command: tar` separately
|
||||
- `get_url` won't re-download when the URL changes but the destination filename stays the same — use a version marker file to detect changes
|
||||
|
||||
## Service Restarts
|
||||
|
||||
- Some services (dnsmasq, etc.) need container restarts for config changes to take effect
|
||||
- Ansible handlers handle this, but always verify the change took effect (e.g., `dig @<ip> <record> +short`)
|
||||
|
||||
## Docker Compose
|
||||
|
||||
- `network_mode: host` ignores `ports:` mappings — remove `ports:` to avoid warnings
|
||||
@@ -1,100 +0,0 @@
|
||||
# Database Selection
|
||||
|
||||
## The Rule: SQLite Is Not a Production Database
|
||||
|
||||
**Any service that meets ANY of the following criteria MUST use PostgreSQL (or equivalent server-grade database) from day one:**
|
||||
|
||||
- Attached to a FQDN (has a real domain name, even internal)
|
||||
- Serves traffic from more than one process (API consumers, CI runners, webhooks, polling)
|
||||
- Backs infrastructure that other systems depend on (Git hosting, container registries, auth providers)
|
||||
- Will be accessed concurrently by automated systems (ArgoCD, CI runners, cron jobs)
|
||||
|
||||
**Do not use SQLite for these workloads. Not temporarily. Not "to start with." Not "we'll migrate later."**
|
||||
|
||||
SQLite uses file-level locking — only one writer at a time, and writes block reads. Under concurrent access, requests queue up waiting for the write lock, causing cascading timeouts. The failure mode is insidious: the service appears to work fine under light load but becomes intermittently unresponsive under real workloads. By the time you notice, everything that depends on it is also failing.
|
||||
|
||||
## The Cost of "We'll Migrate Later"
|
||||
|
||||
The Gitea SQLite→PostgreSQL migration (2026-03-28) cost nearly a full day of productivity:
|
||||
|
||||
- **Hours of accumulated unresponsiveness** across multiple projects before root cause was identified
|
||||
- **Planning and implementation** of the migration itself
|
||||
- **Migration complexity** that didn't need to exist: Gitea 1.23 has no `restore` command, `doctor convert` only handles charset conversion, `docker cp` corrupted PostgreSQL directory permissions, SSH authorized_keys weren't regenerated
|
||||
- **Downstream impact** on ArgoCD (20 apps polling a locked database), CI runners (continuous 500 errors), container registry pulls (timeouts)
|
||||
|
||||
The PostgreSQL container takes 5 minutes to add to a Docker Compose stack at initial setup time. The migration took a day. Always pay the 5 minutes upfront.
|
||||
|
||||
## When SQLite Is Acceptable
|
||||
|
||||
SQLite is fine for:
|
||||
- Local development databases (single developer, single process)
|
||||
- Embedded application data stores (mobile apps, desktop apps, CLI tools)
|
||||
- Read-heavy workloads with rare writes and a single writer process
|
||||
- Test fixtures and throwaway data
|
||||
- Configuration stores read at startup (not at request time)
|
||||
|
||||
## Implementation Pattern
|
||||
|
||||
For Docker Compose services that need a database:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:17-alpine
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: myapp
|
||||
POSTGRES_USER: myapp
|
||||
POSTGRES_PASSWORD: {{ db_password }}
|
||||
volumes:
|
||||
- /opt/postgres-myapp:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U myapp -d myapp"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
start_period: 30s
|
||||
networks:
|
||||
- app_internal
|
||||
deploy:
|
||||
resources:
|
||||
limits:
|
||||
memory: 1G
|
||||
|
||||
myapp:
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- app_internal
|
||||
- external_network
|
||||
|
||||
networks:
|
||||
app_internal:
|
||||
driver: bridge
|
||||
internal: true
|
||||
```
|
||||
|
||||
Key points:
|
||||
- PostgreSQL on an **internal bridge network** (no external access needed)
|
||||
- Application **depends on PostgreSQL health** before starting
|
||||
- **Resource limits** to prevent runaway memory usage
|
||||
- **Separate data directory** per application (`/opt/postgres-myapp`, not shared)
|
||||
- PostgreSQL container UID is **999** (not 1000) — set directory ownership accordingly
|
||||
|
||||
## For Kubernetes Deployments
|
||||
|
||||
Use the application's Helm chart PostgreSQL subchart, or deploy a standalone PostgreSQL instance:
|
||||
- Bitnami PostgreSQL Helm chart for simple deployments
|
||||
- CloudNativePG operator for production-grade PostgreSQL with HA, backups, and failover
|
||||
- Never use SQLite with `emptyDir` or even PVC-backed volumes in multi-replica deployments
|
||||
|
||||
## Checklist for New Service Deployment
|
||||
|
||||
Before deploying any new service, check:
|
||||
|
||||
1. What database does the default configuration use?
|
||||
2. If SQLite: does the service support PostgreSQL? (Almost all do — Gitea, Authelia, Headscale, Zulip, etc.)
|
||||
3. Switch to PostgreSQL **before the first deployment**, not after problems appear
|
||||
4. Add the database password to SOPS-encrypted secrets
|
||||
5. Verify the database connection works before adding consumers
|
||||
@@ -1,80 +0,0 @@
|
||||
# Debugging Methodology
|
||||
|
||||
## Check Before You Act
|
||||
|
||||
- Before writing firewall/network rules, check actual routing (`ip route get <dest>`)
|
||||
- Before running config management with variables, ensure values are real, not placeholders
|
||||
- Before assuming a container has a shell, `docker inspect` it
|
||||
- Before creating API tokens, research all required scopes upfront — iterating one scope at a time costs a push-debug cycle each
|
||||
|
||||
## Routing and Networking
|
||||
|
||||
- Always run `ip route get <dest>` on the forwarding host first
|
||||
- macvlan, Docker bridge, and other virtual interfaces mean the "obvious" physical interface is often wrong
|
||||
- Test from both in-cluster and external perspectives
|
||||
|
||||
## Full-Chain Testing
|
||||
|
||||
After wiring up any new service:
|
||||
1. Test direct to backend (bypass all proxies)
|
||||
2. Test through reverse proxy (bypass DNS)
|
||||
3. Test end-to-end as a user would
|
||||
|
||||
Use `curl --resolve` to test specific paths without depending on DNS propagation.
|
||||
|
||||
## When Something Doesn't Sync/Apply
|
||||
|
||||
- Check resource exclusions in the GitOps controller immediately
|
||||
- Check if the resource type requires special permissions or labels
|
||||
- Check if ServerSideApply conflicts are preventing field changes
|
||||
- Don't try workarounds before understanding the root cause
|
||||
|
||||
## OIDC Integration Checklist
|
||||
|
||||
Before starting any OIDC integration, research:
|
||||
1. What format is the `sub` claim (UUID? username?)
|
||||
2. Which claims are in the ID token vs userinfo endpoint
|
||||
3. How the consumer matches RBAC identities (groups? email? username?)
|
||||
|
||||
## Log-First Diagnosis
|
||||
|
||||
- **CrashLoopBackOff: check logs first.** Error messages in pod logs usually point directly to the fix. Don't tweak configuration or security contexts blindly — `kubectl logs <pod>` first.
|
||||
- **Discriminate transient from persistent errors.** CSI lock contention, etcd timeouts during first install, and brief connectivity blips are self-healing. Don't spend time debugging errors that resolve on retry. If you see retry/backoff patterns in logs, wait before intervening.
|
||||
- **Trust controller retry logic.** CSI controllers, operators, and reconciliation loops have built-in retry. Transient failures during rapid provisioning are expected, not bugs.
|
||||
|
||||
## Reproduce Before Fixing
|
||||
|
||||
When a bug is discovered or reported, **do not start by trying to fix it.** The first step is always to write a test that reproduces the failure:
|
||||
|
||||
1. **Write a failing test.** Capture the bug as a test case that demonstrates the broken behaviour. This forces you to understand the bug precisely — what input triggers it, what the wrong output is, and what the correct output should be.
|
||||
2. **Fix the bug in isolation.** Use a subagent or a separate session to write the fix. The fixing agent gets the failing test as its success criterion — it's done when the test passes. This separation prevents the fixer from unconsciously weakening the test to match a broken implementation.
|
||||
3. **The test stays forever.** The reproduction test becomes a permanent regression test. It proves the fix works and prevents the bug from returning.
|
||||
|
||||
This workflow has several advantages:
|
||||
- **Forces precise understanding.** Writing a test means you know exactly what's broken, not just "it doesn't work."
|
||||
- **Prevents partial fixes.** The test defines "done" objectively — the fix either passes or it doesn't.
|
||||
- **Parallelises work.** While one agent fixes the bug, you can continue other work.
|
||||
- **Catches regressions.** The test remains in the suite, guarding against the same class of failure.
|
||||
|
||||
```python
|
||||
# Step 1: Write the failing test FIRST
|
||||
def test_regression_issue_427_empty_payload_crashes():
|
||||
"""Bug #427: Empty payload causes unhandled TypeError in dispatcher.
|
||||
Should return a 400 validation error, not crash."""
|
||||
response = client.post("/dispatch", json={})
|
||||
assert response.status_code == 400 # Currently crashes with 500
|
||||
|
||||
# Step 2: Hand to a subagent/session: "Make this test pass without breaking others"
|
||||
```
|
||||
|
||||
## `GIT_SSH_COMMAND` Only Affects Git-Invoked SSH
|
||||
|
||||
`GIT_SSH_COMMAND` (e.g., `ssh -o StrictHostKeyChecking=no`) only applies when `git` invokes SSH internally (clone, push, fetch). Direct `ssh` calls — such as `ssh -T git@host` for connectivity testing — ignore it entirely. When working in containers or CI environments where host keys aren't pre-trusted, direct SSH commands need explicit flags: `ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null`.
|
||||
|
||||
## Pattern Mining Before Authoring
|
||||
|
||||
Before building a new service, component, or script, read existing patterns in the codebase first. This matches conventions on the first attempt and avoids rework on naming, structure, and integration points. Applies to K8s manifests, CI pipelines, skill authoring, and script structure.
|
||||
|
||||
## Grep Your Own Docs
|
||||
|
||||
Known issues documented in CLAUDE.md or MEMORY.md but not applied to new scripts/configs waste debugging time. Search your own documentation before writing automation that touches areas with known gotchas.
|
||||
@@ -1,113 +0,0 @@
|
||||
# Docker UID Matching for Mounted Volumes
|
||||
|
||||
When a host directory is mounted into a Docker container, files created by the container process are owned by the container user's UID/GID. If this doesn't match the host user, you get one of two problems:
|
||||
|
||||
1. **Permission denied** — the container can't write to the mounted directory
|
||||
2. **Wrong ownership** — files created inside the container are owned by a different user on the host (e.g., `root` or UID `1001`)
|
||||
|
||||
Both are common sources of friction in developer-facing container tools and CI/CD pipelines.
|
||||
|
||||
## The Pattern: UID Wrapper Entrypoint
|
||||
|
||||
The solution is a small entrypoint wrapper script that:
|
||||
|
||||
1. Starts as root
|
||||
2. Detects the UID/GID of the mounted directory via `stat`
|
||||
3. Adjusts the container user's UID/GID to match using `usermod`/`groupmod`
|
||||
4. Drops privileges via `gosu` and execs the real command
|
||||
|
||||
This is transparent to the user — they don't need to pass `--user` flags or know their UID.
|
||||
|
||||
### Implementation
|
||||
|
||||
**Dockerfile:**
|
||||
|
||||
```dockerfile
|
||||
FROM ubuntu:24.04
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends gosu \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Create non-root user
|
||||
RUN useradd -m -s /bin/bash agent
|
||||
RUN mkdir -p /project && chown agent:agent /project
|
||||
|
||||
COPY uid-wrapper.sh /usr/local/bin/uid-wrapper.sh
|
||||
|
||||
# Start as root — the wrapper drops privileges after UID adjustment
|
||||
ENTRYPOINT ["uid-wrapper.sh", "your-actual-command"]
|
||||
```
|
||||
|
||||
**uid-wrapper.sh:**
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
TARGET_DIR="/project" # The mount point to match
|
||||
AGENT_USER="agent" # The non-root user to adjust
|
||||
|
||||
# If not root, skip adjustment (e.g., K8s securityContext sets UID)
|
||||
if [ "$(id -u)" != "0" ]; then
|
||||
exec "$@"
|
||||
fi
|
||||
|
||||
if [ -d "$TARGET_DIR" ]; then
|
||||
HOST_UID=$(stat -c '%u' "$TARGET_DIR")
|
||||
HOST_GID=$(stat -c '%g' "$TARGET_DIR")
|
||||
|
||||
if [ "$HOST_UID" != "0" ]; then
|
||||
# Adjust GID if different
|
||||
if [ "$HOST_GID" != "$(id -g $AGENT_USER)" ]; then
|
||||
groupmod -g "$HOST_GID" "$AGENT_USER" 2>/dev/null || true
|
||||
fi
|
||||
# Adjust UID if different
|
||||
if [ "$HOST_UID" != "$(id -u $AGENT_USER)" ]; then
|
||||
usermod -u "$HOST_UID" "$AGENT_USER" 2>/dev/null || true
|
||||
fi
|
||||
# Fix home directory ownership
|
||||
chown -R "$AGENT_USER:$(id -g $AGENT_USER)" /home/"$AGENT_USER" 2>/dev/null || true
|
||||
fi
|
||||
fi
|
||||
|
||||
exec gosu "$AGENT_USER" "$@"
|
||||
```
|
||||
|
||||
### Key Details
|
||||
|
||||
- **`gosu` over `su`/`sudo`.** `gosu` execs directly (PID 1 becomes the real process), while `su` creates a child process that breaks signal handling. `gosu` is the standard tool for this pattern.
|
||||
|
||||
- **Handle existing UIDs/GIDs.** The host UID may already be taken by another user in the container. Ubuntu 24.04 images ship with a `ubuntu` user at UID 1000 — the most common host UID. The wrapper must evict the conflicting user to a high unused UID (e.g., 59999 and search downward) before assigning the target UID to the agent user. `usermod -u <new> <conflicting_user>` followed by `usermod -u <target> agent`. Same applies to GIDs — use `getent group` to check before `groupmod`. **Simpler alternative:** Delete the conflicting user at build time (`RUN userdel ubuntu` in the Dockerfile). This avoids runtime conflict handling entirely and is preferred when you control the image.
|
||||
|
||||
- **SSH agent socket forwarding requires UID match.** When mounting `$SSH_AUTH_SOCK` into a container, the socket is mode 0600 owned by the host UID. The container process must run as the same UID to use it — which the UID wrapper handles naturally. Set `SSH_AUTH_SOCK` in the container env to the mounted path. Note: private keys cannot be extracted via the agent protocol — it only supports signing operations.
|
||||
|
||||
- **Skip when not root.** In Kubernetes, `securityContext.runAsUser` sets the UID before the container starts. The wrapper detects this (`id -u != 0`) and skips adjustment — the platform is handling it.
|
||||
|
||||
- **Mount point owned by root.** If the mounted directory is owned by root (UID 0), don't adjust — running as root defeats the purpose. The wrapper only adjusts for non-root UIDs.
|
||||
|
||||
- **Fix home directory.** After `usermod`, the user's home directory still has the old UID. `chown -R` fixes this. Skip other directories — only fix what the user needs.
|
||||
|
||||
## When to Use
|
||||
|
||||
- **Developer-facing container tools** where users mount local project directories (e.g., running a CLI tool inside a container)
|
||||
- **CI/CD containers** that write build artifacts back to a mounted workspace
|
||||
- **Any container that writes to a host-mounted volume** and the output needs correct ownership
|
||||
|
||||
## When NOT to Use
|
||||
|
||||
- **Kubernetes with securityContext** — the platform handles UID assignment; the wrapper correctly skips adjustment in this case
|
||||
- **Containers that don't write to mounted volumes** — unnecessary overhead
|
||||
- **Images that must run as root** — the wrapper's purpose is to run as non-root; if you need root, you don't need the wrapper
|
||||
|
||||
## Alternatives
|
||||
|
||||
| Approach | Pros | Cons |
|
||||
|----------|------|------|
|
||||
| `docker run --user $(id -u):$(id -g)` | Simple, no image changes | No home directory, no `/etc/passwd` entry, breaks tools that need a user identity |
|
||||
| `fixuid` | Purpose-built tool | Another binary to install and maintain |
|
||||
| UID wrapper (this pattern) | Transparent, handles edge cases, works in Docker and K8s | Requires `gosu` in image, starts as root |
|
||||
| Named volumes only | Docker manages ownership | Can't mount host directories |
|
||||
|
||||
## Source
|
||||
|
||||
Learned from the agent-runtimes project (M1) where Claude Code refuses `--dangerously-skip-permissions` as root for security reasons. The container needed to run as non-root, but files created in mounted project directories needed to be owned by the host user. The UID wrapper solved both problems transparently.
|
||||
@@ -1,54 +0,0 @@
|
||||
# Documentation Standards
|
||||
|
||||
Every project maintains four core markdown files.
|
||||
|
||||
## CLAUDE.md
|
||||
|
||||
The primary reference for Claude sessions. Should contain:
|
||||
- Project overview and architecture
|
||||
- Repository structure (keep updated as the project evolves)
|
||||
- Key design decisions with rationale
|
||||
- Conventions and coding standards
|
||||
- Environment details (IPs, URLs, credential references — never actual values)
|
||||
- Common operations / how-to recipes
|
||||
- Put critical rules at the top — Claude reads sequentially and earlier content has more influence
|
||||
|
||||
## MEMORY.md (Tiered Memory System)
|
||||
|
||||
Long-running projects accumulate significant context. Use a **tiered memory** structure:
|
||||
|
||||
**MEMORY.md** is a **thin index only** — one-line descriptions with links to topic files in `memory/`. No content lives in MEMORY.md itself. Keep it under ~50 lines.
|
||||
|
||||
**memory/** contains the actual content, split by topic:
|
||||
- `memory/project-status.md` — Current milestone, what's next, blockers
|
||||
- `memory/gotchas-<topic>.md` — Gotchas grouped by technology
|
||||
- `memory/process-lessons.md` — How-to-work-with-this-repo lessons
|
||||
- `memory/m<N>-reflection.md` — One file per milestone reflection
|
||||
- `memory/decisions.md` — Architecture and design decisions
|
||||
|
||||
**Principles:**
|
||||
- Split by topic, not by time
|
||||
- Index descriptions matter — they're used to decide what to read
|
||||
- Prune aggressively — stale memory is worse than no memory
|
||||
- Each file should be self-contained and greppable
|
||||
- Deduplicate with CLAUDE.md — stable conventions go in CLAUDE.md, learnings and gotchas go in memory
|
||||
|
||||
## FUTURE.md
|
||||
|
||||
Backlog of improvement ideas, each with:
|
||||
- **Problem:** What's painful or manual today
|
||||
- **Idea:** What the improvement looks like
|
||||
- **Open questions:** Unknowns to research before starting
|
||||
- **Depends on:** Other items or milestones that should come first
|
||||
|
||||
## README.md
|
||||
|
||||
Human-readable project documentation:
|
||||
- Architecture summary
|
||||
- Quick start / setup instructions
|
||||
- Milestone table with status
|
||||
- Scripts section listing every script with purpose and usage
|
||||
|
||||
## Extract Reusable Patterns Early
|
||||
|
||||
When a reusable pattern emerges during project work (a gotcha that applies to any project using the same tool, a process lesson that generalises), extract it into a best-practices guide immediately rather than waiting for a dedicated distillation pass. The guide pays for itself when used to audit and improve the current project in the same session, and benefits all future projects.
|
||||
@@ -1,39 +0,0 @@
|
||||
# Git & Source Control
|
||||
|
||||
## Commit Practices
|
||||
|
||||
- Use meaningful commit messages; prefer small, focused commits over large batches
|
||||
- Never commit secrets in plaintext — use SOPS + age or equivalent encryption
|
||||
- Enable pre-commit hooks where appropriate (secret detection, linting, formatting)
|
||||
|
||||
## Pre-Commit Hooks
|
||||
|
||||
- Block `local_secrets/` and similar directories from being committed
|
||||
- Auto-encrypt files matching `.sops.yaml` rules that aren't yet encrypted
|
||||
- Enable with `git config core.hooksPath .githooks`
|
||||
- Consider secret detection, linting, and formatting hooks
|
||||
|
||||
## GitOps Workflow
|
||||
|
||||
- All infrastructure changes should be tracked in Git
|
||||
- No manual changes without corresponding GitOps manifests — anything applied manually (e.g., `kubectl apply`, `helm install`) should immediately get a corresponding tracked manifest
|
||||
- For ArgoCD-managed clusters: edit in Git, push, sync — never edit live resources directly
|
||||
|
||||
## Remote Conventions
|
||||
|
||||
- SSH workflows preferred over HTTPS for Git remotes
|
||||
- Use SSH config host aliases for multi-user setups (e.g., `gitea.example.com-<user>`)
|
||||
- Remote URL format: `git@<host-alias>:<org>/<repo>.git`
|
||||
- Optionally push-mirror to GitHub for public visibility
|
||||
|
||||
## Access and Clone Gotchas
|
||||
|
||||
- **Org repos require explicit collaborator grants.** Don't assume organizational membership implies write access — verify permissions before setting up automation or CI/CD.
|
||||
- **Shallow clones break push operations.** `git clone --depth 1` is fine for read-only CI jobs, but pipelines that push artifacts, tags, or mirror to other remotes need full clones.
|
||||
|
||||
## Version Management
|
||||
|
||||
- Use the latest stable version of dependencies unless pinned for a reason
|
||||
- Verify versions from live sources (`helm search repo`, upstream docs, package registries) — don't rely on memory
|
||||
- Document the reason in a comment if a version is intentionally pinned below latest
|
||||
- Check compatibility matrices before upgrading (e.g., Talos ↔ Kubernetes, framework ↔ runtime)
|
||||
@@ -1,25 +0,0 @@
|
||||
# Helm Charts
|
||||
|
||||
## Schema Validation
|
||||
|
||||
- **Always validate values against the chart schema before committing.** Run `helm show values <repo>/<chart> --version <ver>` to check the actual structure.
|
||||
- Helm chart schemas change between versions — field names and nesting can differ from documentation or online examples.
|
||||
- A quick `helm template` test locally catches schema errors before deployment.
|
||||
- `additionalProperties: false` in chart schemas means any extra keys at the wrong nesting level cause a hard failure.
|
||||
|
||||
## Version Verification
|
||||
|
||||
- Run `helm search repo` or check upstream docs to confirm latest stable versions
|
||||
- Don't rely on memory for chart versions — they go stale quickly
|
||||
- Check compatibility matrices between chart version, app version, and other cluster components
|
||||
|
||||
## Multi-Source Applications
|
||||
|
||||
- ArgoCD multi-source Applications use `$ref` syntax to combine external Helm charts with Git-stored values files
|
||||
- Keep values files in Git alongside the ArgoCD Application manifest
|
||||
|
||||
## Timeout Handling
|
||||
|
||||
- Under cluster pressure (many events, etcd busy), default Helm timeouts may not be enough
|
||||
- Increase timeout for initial installs (e.g., 10m instead of 5m)
|
||||
- `helm upgrade --install` is idempotent — retries are safe
|
||||
@@ -1,68 +0,0 @@
|
||||
# Kubernetes Patterns
|
||||
|
||||
## Volume Mounts
|
||||
|
||||
- **Avoid `subPath` volume mounts** for Secrets and ConfigMaps. The kubelet does not auto-update `subPath` mounts when the source changes — the pod must be restarted. Use directory mounts instead and adjust the application's config path.
|
||||
- **Secret volume propagation is async.** After updating a Secret, the kubelet takes seconds to sync mounted volumes. A `rollout restart` issued immediately after may start pods with stale data. Add a short delay (5s) before restarting.
|
||||
|
||||
## Deployment Strategies
|
||||
|
||||
- **RWO PVC + RollingUpdate = Deadlock.** New pod can't attach the volume while the old pod holds it. Use `strategy: Recreate` for single-replica deployments with RWO PVCs.
|
||||
- **SSA + strategy change conflict.** Switching from RollingUpdate to Recreate via ServerSideApply fails because SSA won't remove the old `rollingUpdate` field. Must patch the live resource first.
|
||||
|
||||
## Naming
|
||||
|
||||
- `metadata.name` must be DNS-1035 compliant — no dots allowed. Replace dots with dashes (e.g., `oreillyit-nz` not `oreillyit.nz`). Label values CAN contain dots.
|
||||
|
||||
## Bootstrap Ordering
|
||||
|
||||
Some components have chicken-and-egg dependencies:
|
||||
1. CNI (e.g., Cilium) must be installed before anything else — nodes are NotReady without it
|
||||
2. GitOps controller (e.g., ArgoCD) installed second
|
||||
3. Root app applied last — the GitOps controller then "adopts" CLI-installed releases
|
||||
|
||||
Manual bootstrap secrets (encryption keys, OIDC client secrets) must be documented as explicit steps.
|
||||
|
||||
## Network Policies
|
||||
|
||||
- DNS egress for `toFQDNs` rules must use `toEndpoints` targeting kube-dns pods with `rules.dns` — this triggers the DNS proxy. Using `toCIDRSet` for DNS bypasses the proxy and FQDN rules never populate.
|
||||
- Cross-namespace policies need explicit namespace matching (e.g., `matchExpressions` on namespace label).
|
||||
- Always test from the actual consumer namespace, not same-namespace test pods.
|
||||
|
||||
## Probe Strategy
|
||||
|
||||
- **Liveness vs readiness probes serve different purposes.** TCP checks confirm the process is listening (liveness). Exec/command checks confirm the application is ready to serve (readiness). Don't conflate them.
|
||||
- **Probes must match application host validation.** Applications that validate Host headers (e.g., Next.js `ALLOWED_HOSTS`) will reject probes sent to the pod IP. Set `httpGet.httpHeaders` with the expected Host value.
|
||||
- **Don't load credentials into liveness probes.** If readiness requires an authenticated check (e.g., `sqlcmd`), use a simple TCP check for liveness and reserve the authenticated check for readiness only.
|
||||
|
||||
## Init Container Patterns
|
||||
|
||||
- **Writable config via init container + emptyDir.** When apps require writable directories but ConfigMaps are read-only, use an init container to copy config into an emptyDir volume that the main container mounts read-write.
|
||||
- **Privilege separation.** Init containers can run as root to create directories or set ownership, while the main container runs as a non-root UID. Prefer this over running the entire workload as root.
|
||||
- **Non-root images have hidden filesystem requirements.** Many modern images (e.g., MSSQL 2022, UID 10001) need writable directories beyond the obvious ones. Always check image documentation or `docker inspect` before writing manifests.
|
||||
|
||||
## StatefulSet Edge Cases
|
||||
|
||||
- **CrashLoopBackOff pods won't auto-replace on spec update.** The StatefulSet controller won't delete and recreate a crashing pod when you update the spec — manual `kubectl delete pod` is required to force recreation.
|
||||
- **Immutable field diffs can deadlock auto-sync.** StatefulSet fields like `volumeClaimTemplates` are immutable after creation. GitOps controllers (ArgoCD) will show permanent OutOfSync if the desired state differs from the live immutable fields. Force sync or recreate the StatefulSet.
|
||||
|
||||
## GitOps: Imperative vs Declarative
|
||||
|
||||
- **Never use imperative operations on GitOps-managed resources.** `kubectl rollout restart` adds annotations that conflict with the GitOps controller's desired state, causing permanent OutOfSync. Use declarative paths instead — update a configmap hash annotation in Git, or change a pod template label.
|
||||
- **ArgoCD reconciliation has latency.** New Application manifests don't appear immediately due to polling intervals. Use manual refresh annotations when automation needs immediate reconciliation.
|
||||
|
||||
## PodSecurity Alignment
|
||||
|
||||
- **Namespace PodSecurity labels must match container security contexts.** DinD, CSI drivers, and other privileged workloads need `pod-security.kubernetes.io/enforce: privileged` on their namespace. A `baseline` or `restricted` namespace silently blocks privileged pods.
|
||||
- **Document privileged namespace requirements.** When a workload needs elevated privileges, document the specific requirement (e.g., "Docker-in-Docker for CI builds") alongside the namespace label.
|
||||
|
||||
## ArgoCD Source Type Detection
|
||||
|
||||
- **ArgoCD auto-detects Kustomize.** When a source directory contains `kustomization.yaml`, ArgoCD runs Kustomize automatically. Adding an explicit `directory:` source type overrides this detection and causes ArgoCD to try applying `kustomization.yaml` as a raw K8s resource, which fails with schema errors. Remove explicit directory source types from Kustomize sources.
|
||||
- **Credential template URL-prefix must match exactly.** ArgoCD repo-creds secrets use URL prefix matching. When migrating Git server URLs (hostname, protocol, or port changes), update the credential template to match the new prefix. Stale credentials cause "authentication required" errors on all apps using that prefix.
|
||||
|
||||
## Miscellaneous
|
||||
|
||||
- `enableServiceLinks: false` may be needed when K8s-injected service env vars conflict with app config (e.g., Authelia interprets `AUTHELIA_*` service vars as configuration).
|
||||
- Proxmox VM names must match K8s node hostnames for cloud controller manager integration.
|
||||
- Metrics-server on Talos needs `--kubelet-insecure-tls` (self-signed kubelet certs).
|
||||
@@ -1,94 +0,0 @@
|
||||
# Linting & Formatting
|
||||
|
||||
Automated code formatting and linting integrated into the Claude Code workflow via PostToolUse hooks and git pre-commit hooks.
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
claude-foundations/formatters/ ← canonical formatter scripts (one per extension)
|
||||
project/formatters/ ← symlinks to the formatters this project uses
|
||||
~/.claude/hooks/post-edit-lint.sh ← dispatches to project formatters on Edit/Write
|
||||
.git/hooks/pre-commit ← symlink to pre-commit-lint.sh
|
||||
```
|
||||
|
||||
Projects opt in by symlinking only the formatters they need. No formatter = no-op.
|
||||
|
||||
## Tool Choices by Language
|
||||
|
||||
| Language | Formatter | Linter | Config File |
|
||||
|----------|-----------|--------|-------------|
|
||||
| Python | `ruff format` | `ruff check` | `pyproject.toml` |
|
||||
| Shell | `shfmt` | `shellcheck` | `.editorconfig` |
|
||||
| TypeScript/JavaScript | `biome format` | `biome check` | `biome.json` |
|
||||
| SQL | `sqlfluff fix` | `sqlfluff lint` | `.sqlfluff` |
|
||||
| JSON/YAML/Markdown | `prettier` | — | `.prettierrc` |
|
||||
|
||||
### Why these tools?
|
||||
- **ruff**: Rust-based, extremely fast, replaces black + isort + flake8 + pyflakes in one tool
|
||||
- **biome**: Rust-based, replaces prettier + eslint for JS/TS in one tool
|
||||
- **shfmt + shellcheck**: The standard combo for shell scripts; shfmt formats, shellcheck catches bugs
|
||||
- **prettier**: Handles JSON/YAML/MD well; biome doesn't cover these yet
|
||||
|
||||
## Formatter Script Contract
|
||||
|
||||
Every script in `formatters/` follows the same interface:
|
||||
|
||||
- **Input:** `$1` = absolute file path
|
||||
- **Behaviour:** format the file in place, then lint it
|
||||
- **Stdout:** suppressed
|
||||
- **Stderr:** lint warnings/errors that couldn't be auto-fixed
|
||||
- **Exit code:** `0` if clean, `1` if lint errors remain
|
||||
- **Missing tools:** exit `0` silently (`command -v` check)
|
||||
- **No `set -e`:** individual commands may fail; execution must continue
|
||||
|
||||
## PostToolUse Integration
|
||||
|
||||
The `post-edit-lint.sh` hook fires on Edit/Write/MultiEdit:
|
||||
|
||||
1. Extracts `file_path` and `cwd` from stdin JSON
|
||||
2. Walks up from `cwd` to find `formatters/` directory
|
||||
3. Creates a checkpoint using `git hash-object` (fast, no commits)
|
||||
4. Runs the matching formatter
|
||||
5. On clean pass: removes checkpoint, exits 0 (silent)
|
||||
6. On lint errors: keeps checkpoint, exits 2 (feeds errors to Claude)
|
||||
|
||||
Exit code 2 is special for PostToolUse — it feeds stderr back to Claude as feedback without blocking the edit.
|
||||
|
||||
## Checkpoint Mechanism
|
||||
|
||||
Uses `git hash-object -w` to store pre-format content as an orphan blob (~1ms, no commits, no stash). The `.pre-lint` file stores only a 40-char SHA. Falls back to `cp` outside git repos.
|
||||
|
||||
```bash
|
||||
# Revert after lint errors:
|
||||
git cat-file blob "$(cat file.py.pre-lint)" > file.py
|
||||
rm file.py.pre-lint
|
||||
```
|
||||
|
||||
## Subagent Fix Pattern
|
||||
|
||||
When lint errors occur, Claude sees the errors via stderr. The recommended workflow:
|
||||
|
||||
1. Claude reports the errors to the user
|
||||
2. If the user says "fix", Claude spawns a subagent via the Agent tool
|
||||
3. The subagent reads the file and errors, makes Edit calls to fix them
|
||||
4. Each Edit triggers the hook again (re-format, re-lint) in the subagent
|
||||
5. Fix iterations stay in the subagent's context, not the main conversation
|
||||
|
||||
## Pre-commit Integration
|
||||
|
||||
`pre-commit-lint.sh` reuses the same formatter scripts:
|
||||
|
||||
1. Iterates staged files (Added/Modified only)
|
||||
2. Runs matching formatters
|
||||
3. Re-stages formatted files
|
||||
4. Exits non-zero if lint errors remain (blocks commit)
|
||||
|
||||
Install: symlink `.git/hooks/pre-commit` → `pre-commit-lint.sh`, or use `setup-formatters.sh` which does this automatically.
|
||||
|
||||
## Setup Checklist
|
||||
|
||||
1. Run `scripts/setup-formatters.sh <project> <ext> [<ext> ...]` to symlink formatters
|
||||
2. Install the required tools (`ruff`, `shfmt`, `shellcheck`, `biome`, `prettier`, `sqlfluff`)
|
||||
3. Create per-project config files as needed (`pyproject.toml`, `.editorconfig`, etc.)
|
||||
4. Run `scripts/install-hooks.sh` to install the PostToolUse hook (one-time global setup)
|
||||
5. Use `/linter scan` to verify everything is connected
|
||||
@@ -1,42 +0,0 @@
|
||||
# Milestones & Reflections
|
||||
|
||||
## Milestone Structure
|
||||
|
||||
Break projects into numbered milestones (M1, M2, ...). This provides clear checkpoints, measurable progress, and natural reflection points.
|
||||
|
||||
## Milestone Completion Checklist
|
||||
|
||||
Every milestone MUST include:
|
||||
|
||||
### 1. Verification Script
|
||||
|
||||
`scripts/verify-m<N>.sh` — automated checks confirming all milestone outcomes.
|
||||
- Idempotent, non-destructive, returns non-zero on failure
|
||||
- Colour output (green/red) for pass/fail
|
||||
- Environment-resilient (no sudo, test from accessible side)
|
||||
- Check for default/insecure credentials
|
||||
|
||||
### 2. Milestone Reflection
|
||||
|
||||
Write `memory/m<N>-reflection.md` by reviewing the **entire conversation** from milestone start. Cover:
|
||||
|
||||
- **Process improvements:** What slowed us down? Wrong assumptions? Where did we go in circles? What would make this faster if redone from scratch?
|
||||
- **Key knowledge for reproduction:** Critical facts, gotchas, non-obvious config details, version-specific quirks, debugging detours
|
||||
- **Scripts and automation:** Existing tools that proved valuable, new scripts to build, patterns to extract into reusable automation
|
||||
- **Future improvements:** Ideas that surfaced but don't belong in current scope — add to FUTURE.md
|
||||
|
||||
### 3. Updated README.md
|
||||
|
||||
Ensure the scripts section, milestone table, and setup steps are current.
|
||||
|
||||
### 4. Updated CLAUDE.md
|
||||
|
||||
Reflect new repo structure, conventions, and patterns discovered during the milestone.
|
||||
|
||||
## Reflection Quality
|
||||
|
||||
Good reflections capture:
|
||||
- Commit stats (total commits, fix percentage) to measure validation discipline
|
||||
- Longest detour and root cause
|
||||
- Most avoidable waste and what would have prevented it
|
||||
- Concrete checklist items for future similar work
|
||||
@@ -1,33 +0,0 @@
|
||||
# Networking & Infrastructure
|
||||
|
||||
## nftables Flush Ruleset on Remote Hosts
|
||||
|
||||
On remote hosts, `nftables flush ruleset` followed by a failed rule load leaves the host with NO firewall. SSH survives only on existing connections — new connections are blocked or allowed depending on the default policy.
|
||||
|
||||
**Always validate rules before applying:** `nft -c -f <rulefile>` does a dry-run parse. For extra safety, deploy a cron-based auto-rollback timer that reverts rules unless explicitly confirmed (similar to `shutdown -c` pattern).
|
||||
|
||||
## systemd Socket Activation Overrides Config File Ports
|
||||
|
||||
On modern Linux systems (Ubuntu 24.04+), systemd socket activation controls the listening port for services like SSH. Editing the service config file alone (e.g., `sshd_config Port 2222`) has no effect — the socket unit still binds the original port.
|
||||
|
||||
**Check socket activation first:** `systemctl cat <service>.socket` shows whether socket activation is in play. If so, override the socket unit's `ListenStream` directive, not the service config.
|
||||
|
||||
## Docker Sets iptables FORWARD Policy to DROP
|
||||
|
||||
Docker sets the iptables FORWARD chain default policy to DROP. This affects ALL forwarding on the host, not just Docker traffic. Non-Docker forwarding (VPN, VM bridges, custom NAT) silently breaks.
|
||||
|
||||
**Fix:** Add explicit ACCEPT rules in the `DOCKER-USER` chain for non-Docker forwarding needs. This chain is processed before Docker's own rules and persists across Docker restarts.
|
||||
|
||||
## HTTP Host Header vs TLS SNI Are Different Layers
|
||||
|
||||
When proxying to a backend over HTTPS, two independent identifiers must be set correctly:
|
||||
- **TLS SNI** — sent during the TLS handshake, used for certificate selection. Missing SNI causes `x509: cannot validate certificate for <IP>`.
|
||||
- **HTTP Host header** — sent after TLS is established, used for virtual host routing. Missing or wrong Host header causes 404 from the backend.
|
||||
|
||||
A reverse proxy must set both. They often need to be the same value, but they're configured independently.
|
||||
|
||||
## Wildcard Certs in Auto-Renewing Proxies
|
||||
|
||||
Auto-renewing proxies (Caddy, Traefik with Let's Encrypt, etc.) that also support file-loaded certificates treat file-loaded certs as globally available. A wildcard cert loaded for one site block will match ALL matching subdomains, silently preventing automatic certificate issuance for other sites.
|
||||
|
||||
**Rule:** Use automatic certificate management for all sites. Don't mix file-loaded and automatic certs unless you understand the matching priority.
|
||||
@@ -1,221 +0,0 @@
|
||||
# Octopus Deploy Process Templates
|
||||
|
||||
Best practices for creating and managing Octopus Deploy process templates using OCL (Octopus Configuration Language) in Platform Hub.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
- **Step templates** (action templates) are reusable individual steps, created via API or UI, stored in a space's library
|
||||
- **Process templates** are reusable multi-step deployment processes, stored as OCL files in Platform Hub's Git repo
|
||||
- **Project templates** compose process templates into full project configurations (feature coming soon)
|
||||
- Process templates live in `.octopus/process-templates/<slug>.ocl` in the Platform Hub Git repo
|
||||
- Projects consume process templates via `process_template` blocks in their `deployment_process.ocl`
|
||||
|
||||
## OCL File Structure
|
||||
|
||||
### Process Template File
|
||||
|
||||
```hcl
|
||||
name = "Deploy to Kubernetes - Helm"
|
||||
description = "Standard Helm-based Kubernetes deployment with pre-validation and smoke tests"
|
||||
|
||||
# Parameters — values supplied by consuming projects
|
||||
parameter "target_tags" {
|
||||
display_settings = {
|
||||
Octopus.ControlType = "TargetTags"
|
||||
}
|
||||
help_text = "Kubernetes target tags"
|
||||
label = "Target Tags"
|
||||
}
|
||||
|
||||
parameter "cloud_target" {
|
||||
display_settings = {
|
||||
Octopus.ControlType = "SingleLineText"
|
||||
}
|
||||
help_text = "Cloud provider (gcp, aws, azure)"
|
||||
label = "Cloud Target"
|
||||
value "gcp" {} # default value
|
||||
}
|
||||
|
||||
# Steps — ordered deployment steps
|
||||
step "deploy-helm" {
|
||||
name = "Deploy via Helm"
|
||||
properties = {
|
||||
Octopus.Action.TargetRoles = "#{target_tags}"
|
||||
}
|
||||
|
||||
action {
|
||||
action_type = "Octopus.Script"
|
||||
properties = {
|
||||
Octopus.Action.Script.ScriptSource = "Inline"
|
||||
Octopus.Action.Script.Syntax = "PowerShell"
|
||||
Octopus.Action.Script.ScriptBody = "Write-Host 'Deploying...'"
|
||||
}
|
||||
worker_pool_variable = ""
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### How Projects Consume Process Templates
|
||||
|
||||
In a project's `deployment_process.ocl`:
|
||||
|
||||
```hcl
|
||||
process_template "deploy-app" {
|
||||
name = "Deploy Application"
|
||||
process_template_slug = "deploy-to-kubernetes-helm"
|
||||
version_mask = "1.X" # auto-update minor/patch
|
||||
|
||||
parameter "target_tags" {
|
||||
value = "kubernetes,production"
|
||||
}
|
||||
|
||||
parameter "cloud_target" {
|
||||
value = "aws"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## OCL Syntax Rules
|
||||
|
||||
From the EBNF grammar (https://github.com/OctopusDeploy/Ocl):
|
||||
|
||||
- **Name, `=`, and value** must be on the same line
|
||||
- **Block name, labels, and `{`** must be on the same line
|
||||
- **Closing `}`** must be on its own line (except empty blocks like `value "default" {}`)
|
||||
- **Strings** use double quotes, cannot contain unescaped `"`
|
||||
- **Multi-line strings** use heredoc: `<<-EOF` / `EOF` (indented variant)
|
||||
- **Arrays** use `["item1", "item2"]`
|
||||
- **Dictionaries** use `{ key = value }` (one entry per line inside braces)
|
||||
|
||||
## Referencing Step Templates
|
||||
|
||||
Step templates are referenced by **ID and version** in the action's properties, NOT by name:
|
||||
|
||||
```hcl
|
||||
step "run-tests" {
|
||||
name = "Run Unit Tests"
|
||||
|
||||
action {
|
||||
# Reference a step template instead of action_type
|
||||
properties = {
|
||||
Octopus.Action.Template.Id = "ActionTemplates-62"
|
||||
Octopus.Action.Template.Version = "1"
|
||||
# Step template parameter values
|
||||
Language = "go"
|
||||
CloudTarget = "gcp"
|
||||
}
|
||||
worker_pool_variable = ""
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
When NOT using a step template, define `action_type` directly:
|
||||
|
||||
```hcl
|
||||
action {
|
||||
action_type = "Octopus.Script"
|
||||
properties = { ... }
|
||||
}
|
||||
```
|
||||
|
||||
## Channel Scoping
|
||||
|
||||
Scope steps to specific channels using the `channels` attribute on the action block. Uses **channel slugs** (auto-generated from names):
|
||||
|
||||
```hcl
|
||||
action {
|
||||
action_type = "Octopus.Script"
|
||||
channels = ["non-prod"] # only runs in non-prod channel
|
||||
properties = { ... }
|
||||
}
|
||||
```
|
||||
|
||||
## Package References
|
||||
|
||||
```hcl
|
||||
# Container image for worker execution
|
||||
container {
|
||||
feed = "registered-dockerhub" # feed slug, not ID
|
||||
image = "octopusdeploy/worker-tools:ubuntu.22.04"
|
||||
}
|
||||
|
||||
# Package reference in a step
|
||||
packages "MyPackage" {
|
||||
acquisition_location = "NotAcquired" # Server | ExecutionTarget | NotAcquired
|
||||
feed = "platformhub-non-prod" # feed slug
|
||||
package_id = "paul-oreilly-octopus/nonprod/listing-service"
|
||||
properties = {
|
||||
SelectionMode = "immediate"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Parameter Types
|
||||
|
||||
| Control Type | `Octopus.ControlType` Value | Can Have Default |
|
||||
|---|---|---|
|
||||
| Single-line text | `SingleLineText` | Yes |
|
||||
| Multi-line text | `MultiLineText` | Yes |
|
||||
| Sensitive/password | `Sensitive` | Yes (encrypted) |
|
||||
| Checkbox | `Checkbox` | Yes |
|
||||
| Dropdown | `Select` | Yes |
|
||||
| AWS/Azure/GCP Account | `AWSAccount` etc. | Yes |
|
||||
| Worker Pool | (worker pool) | No |
|
||||
| Package | (package) | No |
|
||||
| Target Tags | `TargetTags` | No |
|
||||
| Environments | (environments) | No |
|
||||
| Channels | (channels) | No |
|
||||
|
||||
## Step Properties Reference
|
||||
|
||||
| Property | Type | Values | Default |
|
||||
|---|---|---|---|
|
||||
| `step.condition` | enum | `Success`, `Failure`, `Always`, `Variable` | `Success` |
|
||||
| `step.start_trigger` | enum | `StartAfterPrevious`, `StartWithPrevious` | `StartAfterPrevious` |
|
||||
| `step.package_requirement` | enum | `LetOctopusDecide`, `BeforePackageAcquisition`, `AfterPackageAcquisition` | `LetOctopusDecide` |
|
||||
| `action.channels` | string[] | channel slugs | all |
|
||||
| `action.environments` | string[] | environment slugs | all |
|
||||
| `action.excluded_environments` | string[] | environment slugs | none |
|
||||
| `action.is_disabled` | bool | | `False` |
|
||||
| `action.is_required` | bool | | `False` |
|
||||
| `action.notes` | string | step description | |
|
||||
| `action.worker_pool` | string | worker pool slug | |
|
||||
| `action.worker_pool_variable` | string | variable name | |
|
||||
|
||||
## Versioning
|
||||
|
||||
- Process templates use **semantic versioning** (major.minor.patch)
|
||||
- `version_mask = "1.X"` in consuming projects auto-updates on minor/patch changes
|
||||
- **Major version bumps** require explicit upgrade by consuming projects
|
||||
- Keep major bumps for breaking changes (parameter renames, step removals)
|
||||
- Minor/patch for new optional parameters, script improvements, bug fixes
|
||||
|
||||
## Gotchas
|
||||
|
||||
1. **Step templates are space-scoped.** Process templates in Platform Hub cannot reference `ActionTemplates-*` IDs from other spaces. If you need reusable steps, use inline `action_type = "Octopus.Script"` in the process template OCL. Step templates are useful within a single space's projects, but not for cross-space process templates.
|
||||
2. **Process template names** cannot contain parentheses, slashes, or ampersands — only letters, numbers, periods, commas, dashes, underscores, and hashes.
|
||||
3. **Heredoc for multi-line scripts** — use `<<-EOT` / `EOT` for PowerShell scripts that contain double quotes. The `-` prefix allows indented closing tags.
|
||||
4. **Every step needs a worker pool.** Process templates must have a `worker_pool` parameter (type `WorkerPool`), and every action must set `worker_pool_variable = "worker_pool"` referencing it. Without this, the template will fail to parse with "A step must specify a worker pool parameter".
|
||||
5. **Publishing and sharing is UI-only.** Process template sharing (which spaces can see/use a template) is stored in the Octopus database, not in Git/OCL. You must publish and share each template through the UI. There is no API or CLI for this currently.
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **One template per deployment pattern**, not per cloud. Use parameters to vary cloud-specific behaviour.
|
||||
2. **Use step templates for reusable individual steps**, process templates for reusable multi-step workflows.
|
||||
3. **Parameters should have sensible defaults** where possible — reduces friction for consuming projects.
|
||||
4. **Use `TargetTags` parameter type** for Kubernetes target selection rather than hardcoding roles.
|
||||
5. **Name templates with the action, not the technology**: "Deploy to Kubernetes" not "Helm Chart Deployer".
|
||||
6. **Keep descriptions updated** — they appear in the UI when browsing templates.
|
||||
7. **Process templates cannot reference the project's own Git repo** for scripts — use inline scripts or external URLs.
|
||||
8. **Test templates** by creating a test project that consumes them before sharing widely.
|
||||
|
||||
## References
|
||||
|
||||
- OCL Syntax: https://octopus.com/docs/projects/version-control/ocl-file-format
|
||||
- Config as Code Reference: https://octopus.com/docs/projects/version-control/config-as-code-reference
|
||||
- Process Templates: https://octopus.com/docs/platform-hub/templates/process-templates
|
||||
- Template Parameters: https://octopus.com/docs/platform-hub/templates/parameters
|
||||
- Publishing & Sharing: https://octopus.com/docs/platform-hub/templates/publishing-and-sharing
|
||||
- Best Practices: https://octopus.com/docs/platform-hub/templates/process-templates/best-practices
|
||||
- Troubleshooting: https://octopus.com/docs/platform-hub/templates/process-templates/troubleshooting
|
||||
- OCL Grammar (EBNF): https://github.com/OctopusDeploy/Ocl
|
||||
@@ -1,48 +0,0 @@
|
||||
# Scripting Conventions
|
||||
|
||||
## Structure
|
||||
|
||||
- All scripts live in `scripts/` and run from the repository root
|
||||
- Scripts should be idempotent and safe to re-run
|
||||
- Exit non-zero on failure so `&&` chains work naturally
|
||||
|
||||
## Verification Scripts
|
||||
|
||||
- Automated checks confirming milestone or feature outcomes
|
||||
- Use colour output (green/red) for pass/fail indicators
|
||||
- Should be non-destructive and environment-resilient
|
||||
- Avoid needing sudo — test from the accessible side of a connection instead
|
||||
- Check for default/insecure credentials and print remediation instructions on failure
|
||||
- Use `curl --resolve` to bypass DNS/proxy layers when testing direct connectivity
|
||||
|
||||
## Automation Triggers
|
||||
|
||||
If you run the same 3+ commands in sequence more than once, it should become a script. Look for:
|
||||
- Repeated command sequences in conversation history
|
||||
- Steps requiring careful ordering
|
||||
- Multi-step manual processes that are error-prone
|
||||
|
||||
## Error Handling by Tool Purpose
|
||||
|
||||
Not all scripts need the same error handling strategy:
|
||||
- **Destructive scripts** (deploy, configure, delete) should use `set -euo pipefail` — fail fast on any error.
|
||||
- **Reporting/read-only scripts** (status dashboards, aggregation, monitoring) should start without `set -e` — complex data collection from multiple sources is hard to debug under errexit. Use explicit conditional checks instead.
|
||||
- **The choice depends on the tool's purpose.** A script that writes to production needs strict error handling. A script that reads from 10 sources and aggregates results needs resilience.
|
||||
|
||||
## Dryrun Mode
|
||||
|
||||
Every script that modifies state should support `--dryrun` / `-n`:
|
||||
- Makes the script self-documenting about its side effects
|
||||
- Enables safe testing and review before execution
|
||||
- Enables test harnesses that verify output without executing changes
|
||||
- Dryrun output should show exactly what would happen, not a summary
|
||||
|
||||
## Shell Gotchas
|
||||
|
||||
- `((PASS++))` fails under `set -e` when PASS=0 — the expression evaluates to 0 (false), triggering errexit. Use `PASS=$((PASS + 1))` instead.
|
||||
- `set -e` silently terminates complex pipelines and subshells with no output — makes debugging extremely difficult. Also kills command substitutions that capture non-zero exit codes (e.g., `result=$(grep "pattern" file)` exits if grep finds nothing).
|
||||
- `grep` interprets option-like strings (starting with `-`) as flags — use `--` terminator before patterns or input that may start with dashes.
|
||||
- Always quote variables in conditionals and file paths
|
||||
- Use `trap` for cleanup of temp files and credentials
|
||||
- **Use `git diff --numstat` for binary file detection** instead of `file`. The `file` command is unreliable (marks shell scripts as "executable"), while `git diff --numstat` shows `-` for binary files using git's robust binary detection heuristics.
|
||||
- **Use `cat -A` to diagnose invisible character issues.** Reveals non-printing characters like em dashes, zero-width spaces, and smart quotes that look identical to correct characters but break YAML parsers, config files, and frontmatter. Essential when a file looks correct but tooling rejects it.
|
||||
@@ -1,53 +0,0 @@
|
||||
# Secrets Management
|
||||
|
||||
## SOPS + age
|
||||
|
||||
SOPS with age encryption is the standard across all projects. A single `.sops.yaml` at the repo root defines path-based encryption rules.
|
||||
|
||||
### File Naming
|
||||
|
||||
- `.sops.yaml` path-based rules match specific filename patterns (e.g., `**/*secret*.yaml`)
|
||||
- Non-secret files must NOT contain `secret` in their name, or the pre-commit hook will encrypt them
|
||||
- KSOPS generator files should be named `ksops-generator.yaml`, not `secret-generator.yaml`
|
||||
|
||||
### encrypted_regex Gotcha
|
||||
|
||||
When using `encrypted_regex` for selective field encryption (e.g., Ansible group_vars), variable names must contain a keyword that matches the regex (e.g., `password|private_key|api_key|secret|token`). Arbitrary key names are silently left unencrypted.
|
||||
|
||||
### SOPS Vars Plugin
|
||||
|
||||
Each Ansible project needs `vars_plugins_enabled = host_group_vars,community.sops.sops` in `ansible.cfg`. Files in `group_vars/` must be named after a group (e.g., `all.sops.yaml`), not arbitrary names.
|
||||
|
||||
### Interactive Editor Pitfalls
|
||||
|
||||
- `sops <file>` opens an interactive editor — fails in non-interactive sessions
|
||||
- `sops -e /tmp/file` fails when the temp path doesn't match `.sops.yaml` rules
|
||||
- Multiple `sops --set` calls can corrupt files — use the interactive editor for multi-field edits
|
||||
|
||||
## Credential Handling
|
||||
|
||||
- **Never pass secrets via command-line arguments** — visible in `ps` output
|
||||
- Use `@file` references, environment variables sourced at runtime, or stdin
|
||||
- For Ansible, use temp files with `trap rm` cleanup: `-e "@${tmpfile}"`
|
||||
- Read secrets at execution time and use them ephemerally — never cache or persist values
|
||||
- Reference the **existence** of a secret file in docs, never its contents
|
||||
|
||||
## Bootstrap Secrets
|
||||
|
||||
Some secrets are chicken-and-egg (e.g., the age decryption key for ArgoCD's KSOPS). These must be created manually as a bootstrap step and documented clearly.
|
||||
|
||||
## Credential Lifecycle Management
|
||||
|
||||
- **Track credential expiry dates.** OAuth client secrets, API tokens, and certificates have expiry dates that can cause silent failures. Document expiry dates when creating credentials.
|
||||
- **Set alerts before expiry.** For long-lived credentials (e.g., 720-day OAuth client secrets), set calendar reminders or automated monitoring alerts well before they expire.
|
||||
- **Rotation plan.** Know the rotation procedure before you need it — some credential types (e.g., Azure app registrations) require coordinated updates across multiple systems.
|
||||
|
||||
## Multi-Field Secret Files
|
||||
|
||||
Secret files that contain multiple fields (e.g., repo URL, token, username) cannot be used as bare values. Consumers must parse individual fields (e.g., `grep + awk` or structured YAML/JSON parsing).
|
||||
|
||||
The multi-field format is preferable because it's self-documenting — all related credentials live together. But any automation reading the file needs extraction logic, not just `cat`.
|
||||
|
||||
## Backup Considerations
|
||||
|
||||
Backup plans must include encryption keys (age private keys, etc.) so that encrypted data in Git repos remains recoverable.
|
||||
@@ -1,79 +0,0 @@
|
||||
# Security Architecture
|
||||
|
||||
## The Server Boundary Rule
|
||||
|
||||
**No server-side credential may cross the server boundary to the client. Ever.**
|
||||
|
||||
This is a hard line, not a guideline. The only credentials that cross the boundary between client and server are the client's own identity credentials (MFA tokens, login passwords, OIDC tokens, etc.) — and these flow from client to server, never the reverse.
|
||||
|
||||
### What this means in practice
|
||||
|
||||
- **API tokens stay server-side.** If a browser-based application needs to call a third-party API (Gitea, S3, database, etc.), it calls a backend proxy that holds the token. The token never appears in JavaScript, localStorage, cookies, or any client-accessible storage.
|
||||
- **Service account credentials stay server-side.** Database passwords, S3 access keys, webhook secrets, SMTP credentials — these are mounted into server-side containers via Kubernetes Secrets or environment variables and never exposed to clients.
|
||||
- **OAuth tokens for third-party services stay server-side.** If a user authenticates with Service A and the application needs to call Service B on their behalf, the application's backend holds the Service B credentials. The client only ever sees its own session token with the application.
|
||||
- **Per-user tokens mapped server-side.** When individual users need distinct third-party access (e.g., per-user Gitea tokens for audit trails), the mapping from user identity to their token lives on the server. The client authenticates with its own identity (e.g., via Authelia MFA), and the server looks up the appropriate third-party token.
|
||||
|
||||
### The identity exception
|
||||
|
||||
The only credentials that legitimately cross from client to server:
|
||||
|
||||
- **Username + password** — the user's own login credentials
|
||||
- **MFA tokens** — TOTP codes, WebAuthn assertions, security key responses
|
||||
- **OIDC/OAuth tokens** — tokens that represent the user's identity with the application itself (not with third-party services)
|
||||
- **Session cookies/JWTs** — issued by the server to represent an authenticated session
|
||||
|
||||
These all share the property: they are the user's own identity, flowing from client to server for authentication purposes.
|
||||
|
||||
### Architecture pattern: proxy with identity mapping
|
||||
|
||||
When a client-side application (SPA, CMS, admin UI) needs to interact with a backend service that requires credentials:
|
||||
|
||||
```
|
||||
Client ──→ Auth Layer (MFA) ──→ API Proxy ──→ Backend Service
|
||||
│
|
||||
├── Reads user identity from auth headers
|
||||
├── Looks up user's backend credential
|
||||
├── Forwards request with backend credential
|
||||
└── Returns response (without credential)
|
||||
```
|
||||
|
||||
The proxy:
|
||||
1. Sits behind the authentication layer (Authelia, OAuth2 Proxy, etc.)
|
||||
2. Reads the authenticated user's identity from trusted headers (e.g., `Remote-User`)
|
||||
3. Maps the identity to the appropriate backend credential
|
||||
4. Makes the backend API call with the credential
|
||||
5. Returns the response — never the credential
|
||||
|
||||
### Security layers (defense in depth)
|
||||
|
||||
A well-designed proxy architecture has multiple independent security layers:
|
||||
|
||||
1. **Authentication** — User must prove their identity (MFA, OIDC)
|
||||
2. **Session validation** — Proxy validates the session is current and legitimate
|
||||
3. **Authorization** — Proxy checks the user has access to the requested resource
|
||||
4. **Backend permissions** — The backend service enforces its own access controls
|
||||
5. **Branch/scope protection** — Fine-grained controls prevent privilege escalation (e.g., branch protection rules)
|
||||
|
||||
Each layer is independent — compromising one doesn't bypass the others.
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
- **Passing API tokens to the browser via OAuth.** Even with PKCE, the token ends up in client-accessible storage. Use a backend proxy instead.
|
||||
- **Shared service account tokens.** One token for all users means no audit trail and no granular revocation. Map per-user tokens server-side.
|
||||
- **Embedding credentials in client-side config.** API keys in `config.js`, `.env` files served statically, or hardcoded in HTML — all violate the boundary rule.
|
||||
- **Forwarding backend tokens via API responses.** Even "temporarily" returning a token in a response body breaks the rule. The client should never see it.
|
||||
- **Using the same token for client auth and backend calls.** The user's session token with your application is distinct from any token your application uses to call backend services.
|
||||
|
||||
### When credentials must be client-side
|
||||
|
||||
Some scenarios genuinely require client-side credentials (e.g., direct S3 uploads for large files, WebRTC signaling). In these cases:
|
||||
- Use **presigned URLs** or **temporary credentials** with the narrowest possible scope and shortest possible lifetime
|
||||
- The presigning/credential-issuance happens server-side
|
||||
- The temporary credential is scoped to exactly one operation (e.g., upload one file to one path)
|
||||
- Log the issuance server-side for audit
|
||||
|
||||
### Real-world example: CMS editing
|
||||
|
||||
**Wrong:** CMS authenticates directly with Gitea via OAuth popup. Gitea token lands in browser localStorage. CMS makes API calls directly to Gitea with the token.
|
||||
|
||||
**Right:** CMS sits behind Authelia (MFA). A proxy service intercepts the OAuth flow, issues a proxy session token (containing only the user's identity), and forwards all API calls to Gitea using a per-user server-side Gitea token. The Gitea token never leaves the server.
|
||||
@@ -1,46 +0,0 @@
|
||||
# Claude Code Skills
|
||||
|
||||
## Skill Structure
|
||||
|
||||
- Each skill lives in `skills/<skill-name>/SKILL.md`
|
||||
- Skills should be project-agnostic where possible — use dynamic context injection to adapt
|
||||
- After adding a new skill, run the install script to register it
|
||||
- Skills only useful for one project should live in that project's `.claude/skills/` instead
|
||||
|
||||
## Authoring Guidelines
|
||||
|
||||
- **Inline by default** — only use `context: fork` if the skill genuinely doesn't need conversation history
|
||||
- **Pre-fetch context** with `!`command`` injection to reduce tool calls during execution
|
||||
- **Restrict tools** with `allowed-tools` to the minimum needed — reduces permission prompts
|
||||
- **Use $ARGUMENTS** for user input, `$0`, `$1` etc. for positional args
|
||||
- Dynamic commands in `!`command`` run at skill load time, not during Claude's execution
|
||||
|
||||
## Portable Path Resolution
|
||||
|
||||
- **Use `CLAUDE_PROJECT_ROOT` env var** for cross-project path references in `!`command`` blocks. Hardcoded absolute paths (e.g., `~/dev/claude/...`) are user-specific. Relative paths (`../`) break depending on CWD and can trigger sandbox violations when they resolve outside allowed directories.
|
||||
- **Add a detection fallback.** Include a "Step 0" in skill instructions that detects the project root by walking up the directory tree to find the highest `CLAUDE.md` if the env var isn't set. This makes skills work even without prior setup.
|
||||
- **Keep config paths relative to the root.** Settings files should use paths relative to `CLAUDE_PROJECT_ROOT` (e.g., `projects_dir: projects`) rather than absolute paths, so they're portable across machines.
|
||||
|
||||
## Skill Discovery Timing
|
||||
|
||||
- **Skills are discovered at session start, not dynamically.** Creating or symlink a new skill mid-session requires restarting Claude Code to use it as a slash command.
|
||||
- **Broken symlinks cause silent failures under `set -e`.** `readlink -f` on a broken symlink returns empty string. The install script should validate symlinks and remove stale ones.
|
||||
|
||||
## `!`command`` Gotchas
|
||||
|
||||
- **No `$()` command substitution** — the permission checker rejects commands containing `$()`
|
||||
- **No complex shell pipelines relying on subshells** — keep commands simple and self-contained
|
||||
- **`allowed-tools` patterns must match the command binary** — each binary used in `!`command`` blocks needs its own pattern
|
||||
- **Prefer specific tool patterns over broad ones** — `Bash(git log *)` is safer than `Bash(git *)`
|
||||
- **Fallback to tool instructions for dynamic paths** — if a command needs `$ARGUMENTS` to compute a path, use a plain-text instruction telling Claude to use the Read tool instead
|
||||
- **Env var expansion works** — `${CLAUDE_PROJECT_ROOT}` expands in `!`command`` blocks because they run as shell commands. This is the recommended pattern for portable cross-project paths.
|
||||
|
||||
## Non-ASCII in YAML Frontmatter
|
||||
|
||||
Skills with em dashes (`—`), smart quotes (`"`), or other non-ASCII characters in the YAML frontmatter `description` field fail to load silently — the skill appears as "Unknown skill" with no error message. The markdown body below the frontmatter can contain any characters.
|
||||
|
||||
AI models commonly generate em dashes instead of regular dashes. Always validate skill files (e.g., with `cat -A` or a dedicated validator) before committing.
|
||||
|
||||
## Profile-Independent Skills Directories
|
||||
|
||||
Each Claude Code profile maintains a completely independent skills directory. Skills installed in one profile (e.g., default) are unavailable in other profiles (e.g., `.claude-octopus`). Install scripts must use the profile-aware config directory path rather than hardcoded paths like `~/.claude/skills/`.
|
||||
@@ -1,227 +0,0 @@
|
||||
# Spec-Driven Development with AI Agents
|
||||
|
||||
Best practices for using structured specifications to coordinate AI agent implementation work. Extracted from real project experience (agent-runtimes) and industry research (OpenSpec, Codified Context paper, Addy Osmani's workflow guides).
|
||||
|
||||
## Why Specs Matter for AI Agents
|
||||
|
||||
AI agents trust documentation absolutely. A well-written spec gives an agent everything it needs to implement a subsystem without reading the entire codebase. A stale or vague spec causes silent failures where agents generate code that is structurally valid but architecturally wrong.
|
||||
|
||||
Specs serve three functions that CLAUDE.md alone cannot:
|
||||
1. **Compressed context** — an agent reads one spec, not 300 lines of mixed concerns
|
||||
2. **Testable contracts** — numbered requirements and scenarios translate directly to pytest
|
||||
3. **Handoff boundaries** — an agent working on the dispatcher doesn't need to understand the entrypoint internals, just the interface between them
|
||||
|
||||
## Spec Structure
|
||||
|
||||
Each spec follows a consistent template. Sections are ordered so an agent can read top-down and build understanding progressively.
|
||||
|
||||
### Required Sections
|
||||
|
||||
1. **Overview** — What this subsystem does. 2-3 sentences. An agent should know if this spec is relevant after reading this.
|
||||
2. **Responsibilities** — What this subsystem owns and what it delegates. Prevents scope creep during implementation.
|
||||
3. **Dependencies** — Which other specs to read first. Keeps the reading list minimal.
|
||||
4. **Data Model** — Types, schemas, state machines, interfaces. The concrete contract.
|
||||
5. **Requirements** — Numbered functional requirements (e.g., E-1, E-2). Each must be independently testable.
|
||||
6. **Scenarios** — Concrete given/when/then examples. These become test functions.
|
||||
|
||||
### Optional Sections
|
||||
|
||||
7. **Interface** — API surface, function signatures, HTTP endpoints. Include when the subsystem has an external-facing API.
|
||||
8. **Extension Points** — How to add new capabilities without modifying existing code. Step-by-step instructions.
|
||||
9. **Error Handling** — Failure modes and expected behaviour. Prevents agents from inventing their own error strategies.
|
||||
|
||||
### Writing Guidelines
|
||||
|
||||
- **Be specific, not comprehensive.** A spec that says "handle errors appropriately" is useless. A spec that says "return exit code 124 on timeout" is testable.
|
||||
- **Include the why.** Design intent and constraints prevent agents from making structurally valid but architecturally wrong changes. Requirements without rationale are followed mechanically — agents can't judge edge cases or make trade-offs. Every constraint needs a "Why:" line. Example: `"Secrets never in payload"` needs `"because payloads may be logged and stored in task history"`.
|
||||
- **Use concrete examples.** Every data model section should include a realistic JSON/code example, not just a schema.
|
||||
- **Cross-reference, don't duplicate.** If two specs share a concept (e.g., the payload schema), one spec owns it and the other links to it.
|
||||
- **Keep each spec self-contained.** An agent should be able to implement a subsystem by reading the target spec plus its listed dependencies. If it needs to read CLAUDE.md, the spec is incomplete.
|
||||
|
||||
## Requirement Numbering
|
||||
|
||||
Each spec uses a short prefix derived from its name, followed by a sequential number:
|
||||
|
||||
| Spec | Prefix | Example |
|
||||
|---|---|---|
|
||||
| payload.md | P | P-1, P-2 |
|
||||
| entrypoint.md | E | E-1, E-2 |
|
||||
| actions.md | A | A-1, A-2 |
|
||||
| runners.md | R | R-1, R-2 |
|
||||
| dispatcher.md | D | D-1, D-2 |
|
||||
| control-plane.md | CP | CP-1, CP-2 |
|
||||
|
||||
Requirements must be:
|
||||
- **Independently testable** — each maps to one or more test functions
|
||||
- **Unambiguous** — an agent can determine pass/fail without human judgement
|
||||
- **Stable** — changing a requirement number invalidates tests, so avoid renumbering
|
||||
|
||||
## Scenarios as Test Blueprints
|
||||
|
||||
Every scenario in a spec should be directly translatable to a test function. Use this format:
|
||||
|
||||
```markdown
|
||||
### Scenario: Pre-action failure
|
||||
**Given:** Payload with clone pre-action (invalid repo URL)
|
||||
**When:** Clone fails (git returns non-zero)
|
||||
**Then:** on_error actions run, container exits 1. Runner never executes.
|
||||
```
|
||||
|
||||
This becomes:
|
||||
|
||||
```python
|
||||
def test_scenario_preaction_failure_runs_on_error_and_exits_1(...):
|
||||
"""Given clone fails, on_error runs and exits 1. Runner never executes."""
|
||||
```
|
||||
|
||||
Guidelines:
|
||||
- Each scenario tests one behaviour, not a combination
|
||||
- Include both happy paths and error paths
|
||||
- Name the scenario descriptively — it becomes the test function's docstring
|
||||
- Include enough setup detail that an agent can write the test without guessing
|
||||
|
||||
## The Spec → Test → Code Workflow
|
||||
|
||||
This is the core development loop. Tests are written from the spec before code exists.
|
||||
|
||||
### 1. Write or Update the Spec
|
||||
|
||||
Define requirements and scenarios. Get them reviewed. The spec is the source of truth for what the system should do.
|
||||
|
||||
### 2. Write Tests from the Spec
|
||||
|
||||
Translate requirements and scenarios into pytest functions. Tests should:
|
||||
- Map to requirement IDs in their names: `test_e4_preaction_failure_skips_remaining`
|
||||
- Use the scenario's given/when/then as the test body structure
|
||||
- Mock external dependencies (subprocess, HTTP, filesystem)
|
||||
- Run fast (no Docker, no network, no real APIs)
|
||||
|
||||
### 3. Run the Tests — They Should All Fail
|
||||
|
||||
This confirms the tests are actually testing something. If a test passes before implementation, it's either testing the wrong thing or the feature already exists.
|
||||
|
||||
### 4. Implement Until Tests Pass
|
||||
|
||||
Write the minimum code to make tests pass. The spec defines what, the tests verify it, the code implements it.
|
||||
|
||||
### 5. Update Spec if Implementation Reveals Issues
|
||||
|
||||
Sometimes implementation reveals that a requirement is unworkable or incomplete. Update the spec, update the test, then update the code. The spec stays authoritative.
|
||||
|
||||
## Spec Maintenance
|
||||
|
||||
### Preventing Drift
|
||||
|
||||
Specs drift from code when they're treated as planning documents that are "done" after implementation. They must be treated as living contracts.
|
||||
|
||||
**Rules:**
|
||||
- **Spec changes require test changes.** If a requirement changes, its test must change in the same commit.
|
||||
- **Code changes that affect interfaces require spec changes.** If a function signature, API endpoint, or data schema changes, the relevant spec must be updated in the same commit.
|
||||
- **New features require spec-first.** Add the requirement and scenario to the spec, write the test, then implement.
|
||||
|
||||
### CI Enforcement
|
||||
|
||||
Enforce spec hygiene with automated checks:
|
||||
|
||||
1. **Pre-commit hook** — run pytest, block commit on failure (already implemented)
|
||||
2. **Spec coverage check** — a script that verifies every numbered requirement has at least one test function referencing it
|
||||
3. **Orphan test detection** — tests referencing requirement IDs that no longer exist in specs
|
||||
|
||||
### Review Checklist
|
||||
|
||||
When reviewing a PR that touches a spec subsystem:
|
||||
- [ ] Spec updated if interface or behaviour changed
|
||||
- [ ] Test added/updated for new/changed requirements
|
||||
- [ ] Cross-references still valid
|
||||
- [ ] No requirements removed without deprecation note
|
||||
|
||||
## Context Architecture for Agents
|
||||
|
||||
Based on the Codified Context paper (108k-line system, 283 sessions), structure project knowledge in three tiers:
|
||||
|
||||
### Tier 1: Hot Context (Always Loaded)
|
||||
|
||||
CLAUDE.md — conventions, env vars, repo structure, scripts. Loaded every session. Keep under ~300 lines by linking to details elsewhere.
|
||||
|
||||
### Tier 2: Spec Context (Per-Task)
|
||||
|
||||
`spec/` files — loaded based on what the agent is working on. An agent implementing a new action reads `spec/actions.md` + `spec/payload.md`. An agent working on the dispatcher reads `spec/dispatcher.md` + `spec/container-backends.md`.
|
||||
|
||||
The spec index (SPEC.md) has a "read this when..." column to guide selection.
|
||||
|
||||
### Tier 3: Cold Context (On-Demand)
|
||||
|
||||
`memory/` files — gotchas, reflections, decisions. Loaded only when relevant. An agent hitting a weird Cilium issue checks `memory/gotchas-cilium.md`.
|
||||
|
||||
### Routing Context to Agents
|
||||
|
||||
When launching an agent to work on a subsystem:
|
||||
1. Point it at the relevant spec(s) via its prompt
|
||||
2. Include CLAUDE.md for conventions
|
||||
3. Let it pull from memory/ on-demand if it hits issues
|
||||
|
||||
Don't load everything — agents perform better with focused context than with a 50-page dump.
|
||||
|
||||
## Testing Depth
|
||||
|
||||
The spec→test→code workflow defines *when* to write tests. For *how* to write comprehensive tests — edge case discovery, property-based testing, mutation testing, AI agent testing patterns — see [Test-Driven Development](test-driven-development.md).
|
||||
|
||||
## Post-Write Spec Audit
|
||||
|
||||
After writing specs, audit them against best practices before implementation. Common gap categories:
|
||||
|
||||
1. **Missing rationale** — Constraints without "Why:" lines. Agents follow them mechanically but can't judge edge cases.
|
||||
2. **Missing error/failure scenarios** — Happy paths are covered but failure modes aren't specified.
|
||||
3. **Cross-spec interface misalignment** — Two specs describe the same interface differently.
|
||||
4. **Vague requirements** — "Handle errors appropriately" instead of specific error codes and behaviours.
|
||||
5. **Missing specs for discovered subsystems** — Implementation reveals components that weren't planned for.
|
||||
|
||||
Write-then-audit is more productive than trying to get specs perfect on the first pass. The audit step catches systematic gaps across all specs at once.
|
||||
|
||||
## Planning Session Limits
|
||||
|
||||
Architecture decisions, infrastructure research, and spec refinement each get one planning session. After three sessions of planning, start implementation. Specs are hypotheses that need code to validate them — extended planning without implementation produces diminishing returns and theoretical designs that don't survive contact with reality.
|
||||
|
||||
## Categorize Findings Before Acting
|
||||
|
||||
When a spec review or audit produces many findings, categorize them by priority (high/medium/low) before making changes. Present the categorized list for alignment before editing. Starting edits without prioritization leads to scope creep — low-priority cosmetic fixes consume time that should go to high-priority structural gaps.
|
||||
|
||||
## Multi-Agent Orchestration Practices
|
||||
|
||||
### Commit WIP Before Decomposing Tasks
|
||||
|
||||
Untracked and uncommitted files are NOT available in git worktrees. If agents work in worktrees (or container-mounted worktrees), they won't see specs, plans, or dependency outputs that haven't been committed. Commit to a staging branch before decomposition — this eliminates the dominant overhead of manually copying files into each worktree.
|
||||
|
||||
### Agents Must Self-Verify with Tests
|
||||
|
||||
Add "Run tests and fix any failures" to every implementation agent prompt. Agents that write code without running tests produce bugs that only surface during assembly. Self-verification catches issues while the agent still has full context of what it wrote.
|
||||
|
||||
### State Import and Style Conventions Explicitly
|
||||
|
||||
Agents default to standard language conventions (e.g., relative Python imports, standard packaging). If the project uses non-standard patterns (bare imports, specific naming conventions, module-level structure), state them explicitly in the prompt. A single line like "Use `from harness import X`, not `from .harness import X`" prevents import mismatches during assembly.
|
||||
|
||||
### Budget for Assembly Fixups
|
||||
|
||||
Parallel agent work produces ~3 fixups per orchestration run, each under 5 minutes. Common fixup categories: import conventions, module-level side effects, SDK exception constructor signatures, validator patterns. This is the expected cost of parallel work, not a failure. Budget 15-20 minutes for assembly and fixup after each orchestration run.
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
### Specs as documentation, not contracts
|
||||
**Symptom:** Specs describe what was built, updated after the fact. Tests don't reference spec IDs.
|
||||
**Fix:** Write specs before code. Tests reference requirement IDs. Specs are the input, not the output.
|
||||
|
||||
### Mega-spec
|
||||
**Symptom:** One large spec covering the entire system. Agents must read thousands of lines to find what they need.
|
||||
**Fix:** Split by subsystem. Each spec should be readable in under 5 minutes.
|
||||
|
||||
### Spec without scenarios
|
||||
**Symptom:** Requirements are abstract ("handle errors gracefully"). No concrete examples.
|
||||
**Fix:** Every requirement needs at least one given/when/then scenario with specific inputs and outputs.
|
||||
|
||||
### Implementation details in specs
|
||||
**Symptom:** Spec dictates variable names, algorithm choices, internal data structures.
|
||||
**Fix:** Specs define what and why, not how. The interface is specified; the implementation is free.
|
||||
|
||||
### Untested requirements
|
||||
**Symptom:** Requirements exist in the spec but no test references them. They drift without anyone noticing.
|
||||
**Fix:** Spec coverage check in CI. Every requirement ID must appear in at least one test function name.
|
||||
@@ -1,474 +0,0 @@
|
||||
# Test-Driven Development for Spec-Based Projects
|
||||
|
||||
Best practices for writing comprehensive, regression-catching tests in projects that use structured specifications. Focuses on maximising test value (catching real bugs) rather than test volume (inflating coverage numbers). Extracted from industry research, academic papers (TDAD, Codified Context), and practitioner experience.
|
||||
|
||||
## Core Principle: Tests Are the Spec's Enforcement Layer
|
||||
|
||||
In a spec-driven project, the spec defines *what* and the tests *prove it*. A requirement without a test is an aspiration. A test without a requirement is undocumented behaviour. Keep them tightly coupled:
|
||||
|
||||
- Every numbered requirement (P-1, E-3) has at least one test
|
||||
- Every test function name includes its requirement ID: `test_e3_preaction_failure_exits_1`
|
||||
- Spec changes and test changes ship in the same commit
|
||||
|
||||
## Deriving Tests from Specs
|
||||
|
||||
### Requirements to Tests
|
||||
|
||||
Each spec requirement becomes one or more test functions. The mapping isn't always 1:1 — a requirement like "must respect timeout" needs tests for: default timeout, explicit timeout, timeout=0 (no limit), timeout exceeded.
|
||||
|
||||
```python
|
||||
# From spec: R-4: Runners must respect runtime.timeout.
|
||||
# Default 3600s. Value of 0 means no timeout.
|
||||
|
||||
def test_r4_default_timeout_is_3600():
|
||||
"""R-4: When timeout not specified, default is 3600s."""
|
||||
|
||||
def test_r4_explicit_timeout_is_honoured():
|
||||
"""R-4: When timeout=60, process killed after 60s."""
|
||||
|
||||
def test_r4_zero_timeout_means_no_limit():
|
||||
"""R-4: When timeout=0, no timeout is applied."""
|
||||
|
||||
def test_r4_timeout_returns_exit_code_124():
|
||||
"""R-4 + R-5: Timeout produces exit code 124."""
|
||||
```
|
||||
|
||||
### Scenarios to Tests
|
||||
|
||||
GIVEN/WHEN/THEN scenarios translate directly to Arrange/Act/Assert:
|
||||
|
||||
```python
|
||||
def test_scenario_preaction_failure_runs_on_error():
|
||||
"""Given clone fails, on_error runs and exits 1. Runner never executes."""
|
||||
# GIVEN — arrange
|
||||
payload = make_payload(pre_actions=[{"action": "clone", "repo": "bad-url"}])
|
||||
mock_clone = Mock(side_effect=subprocess.CalledProcessError(128, "git"))
|
||||
|
||||
# WHEN — act
|
||||
exit_code = run_entrypoint(payload, clone_handler=mock_clone)
|
||||
|
||||
# THEN — assert
|
||||
assert exit_code == 1
|
||||
mock_runner.assert_not_called()
|
||||
mock_on_error.assert_called_once()
|
||||
```
|
||||
|
||||
### Parameterised Tests from Spec Enumerations
|
||||
|
||||
When a spec lists multiple valid values, use `@pytest.mark.parametrize`:
|
||||
|
||||
```python
|
||||
# From spec: task states are pending, assigned, running, succeeded, failed, timed_out, cancelled
|
||||
@pytest.mark.parametrize("terminal_state", ["succeeded", "failed", "timed_out", "cancelled"])
|
||||
def test_cp_terminal_state_cannot_be_overwritten(terminal_state):
|
||||
"""CP: Terminal states reject further transitions with 409."""
|
||||
```
|
||||
|
||||
## Systematic Edge Case Discovery
|
||||
|
||||
~80% of bugs cluster at boundaries. Use these techniques to find edge cases systematically rather than by intuition.
|
||||
|
||||
### Boundary Value Analysis
|
||||
|
||||
For every input parameter, test at the edges of its valid range:
|
||||
|
||||
| Input type | Test values |
|
||||
|---|---|
|
||||
| Integer (range 1-100) | 0, 1, 2, 99, 100, 101, -1, MAX_INT |
|
||||
| String | `""`, `"a"`, max-length string, max+1, unicode (`"\u0000"`, emoji), whitespace-only |
|
||||
| List/Array | `[]`, `[single]`, many items, duplicates, `None` |
|
||||
| Dict/Map | `{}`, missing required keys, extra unknown keys, `None` values |
|
||||
| Timeout (seconds) | 0, 1, -1, very large (999999), `None`/missing |
|
||||
| Base64 | valid, invalid chars, empty, padding variants (`=`, `==`, none) |
|
||||
|
||||
### Equivalence Partitioning
|
||||
|
||||
Group inputs into classes where all members should behave identically. Test one from each class:
|
||||
|
||||
```python
|
||||
# Payload validation: prompt field
|
||||
# Class 1: valid string → accepted
|
||||
# Class 2: empty string → rejected (spec says prompt is required)
|
||||
# Class 3: missing key → rejected
|
||||
# Class 4: wrong type (int, list, None) → rejected
|
||||
# Class 5: very long string → accepted (no length limit in spec)
|
||||
|
||||
@pytest.mark.parametrize("prompt,should_pass", [
|
||||
("Fix the bug", True), # Class 1: valid
|
||||
("", False), # Class 2: empty
|
||||
(None, False), # Class 3: missing/None
|
||||
(42, False), # Class 4: wrong type
|
||||
("x" * 100_000, True), # Class 5: long string
|
||||
])
|
||||
def test_p_prompt_validation(prompt, should_pass):
|
||||
...
|
||||
```
|
||||
|
||||
### State Transition Coverage
|
||||
|
||||
For state machines (task states, dispatcher states), test:
|
||||
|
||||
1. **Every valid transition:** `pending → assigned → running → succeeded`
|
||||
2. **Every invalid transition:** `succeeded → running` (should be rejected)
|
||||
3. **Initial state:** newly created tasks start in `pending`
|
||||
4. **Terminal states:** `succeeded`, `failed`, `timed_out`, `cancelled` cannot transition further
|
||||
5. **Re-entrant transitions:** same state → same state (should be idempotent or rejected, per spec)
|
||||
|
||||
```python
|
||||
VALID_TRANSITIONS = [
|
||||
("pending", "assigned"),
|
||||
("assigned", "running"),
|
||||
("running", "succeeded"),
|
||||
("running", "failed"),
|
||||
("running", "timed_out"),
|
||||
("assigned", "cancelled"),
|
||||
("running", "cancelled"),
|
||||
]
|
||||
|
||||
INVALID_TRANSITIONS = [
|
||||
("succeeded", "failed"),
|
||||
("failed", "running"),
|
||||
("cancelled", "pending"),
|
||||
("timed_out", "running"),
|
||||
]
|
||||
|
||||
@pytest.mark.parametrize("from_state,to_state", VALID_TRANSITIONS)
|
||||
def test_valid_state_transition(from_state, to_state):
|
||||
...
|
||||
|
||||
@pytest.mark.parametrize("from_state,to_state", INVALID_TRANSITIONS)
|
||||
def test_invalid_state_transition_rejected(from_state, to_state):
|
||||
...
|
||||
```
|
||||
|
||||
### The Edge Case Checklist
|
||||
|
||||
Walk through this for every function under test:
|
||||
|
||||
1. **Empty/null inputs** — what happens when required fields are missing?
|
||||
2. **Boundary values** — min, max, zero, negative, off-by-one
|
||||
3. **Type mismatches** — string where int expected, list where dict expected
|
||||
4. **Malformed input** — invalid JSON, bad base64, truncated data
|
||||
5. **Concurrent operations** — two tasks claiming the same resource
|
||||
6. **Ordering** — actions that depend on sequence (pre-action before runner)
|
||||
7. **Idempotency** — calling the same operation twice (kill an already-killed container)
|
||||
8. **Resource exhaustion** — at capacity, disk full, timeout expired
|
||||
9. **Partial failure** — first action succeeds, second fails (cleanup?)
|
||||
|
||||
## Property-Based Testing with Hypothesis
|
||||
|
||||
Instead of specifying individual test cases, define *properties* that must hold for all inputs. Hypothesis generates hundreds of inputs including edge cases you'd never think of.
|
||||
|
||||
### When to Use Property-Based Testing
|
||||
|
||||
- **Serialisation roundtrips:** encode → decode returns original
|
||||
- **Parsers:** should never crash on any input
|
||||
- **Data transformations:** invariants that hold regardless of input
|
||||
- **Validators:** valid inputs accepted, invalid inputs rejected (never crash)
|
||||
|
||||
### When NOT to Use It
|
||||
|
||||
- Tests where generating valid inputs is harder than the code itself
|
||||
- Tests where the "property" just restates the implementation
|
||||
- UI or integration tests
|
||||
|
||||
### Patterns
|
||||
|
||||
```python
|
||||
from hypothesis import given, strategies as st, assume, settings
|
||||
from hypothesis import example
|
||||
|
||||
# Roundtrip: base64 encode/decode preserves payload
|
||||
@given(st.text())
|
||||
def test_base64_roundtrip(payload_str):
|
||||
encoded = base64.b64encode(payload_str.encode()).decode()
|
||||
decoded = base64.b64decode(encoded).decode()
|
||||
assert decoded == payload_str
|
||||
|
||||
# Invariant: payload validation never crashes (may reject, never exception)
|
||||
@given(st.dictionaries(st.text(), st.text() | st.integers() | st.none()))
|
||||
def test_payload_validation_never_crashes(raw_payload):
|
||||
# Should return True/False or raise ValidationError — never unhandled exception
|
||||
try:
|
||||
validate_payload(raw_payload)
|
||||
except ValidationError:
|
||||
pass # Expected for invalid input
|
||||
|
||||
# Pin known edge cases alongside random generation
|
||||
@example("") # empty string
|
||||
@example("\x00") # null byte
|
||||
@example("a" * 10**6) # very long
|
||||
@given(st.text())
|
||||
def test_prompt_handling(prompt):
|
||||
...
|
||||
|
||||
# Composite strategies for domain objects
|
||||
@st.composite
|
||||
def valid_payloads(draw):
|
||||
return {
|
||||
"task_id": draw(st.uuids()).hex,
|
||||
"prompt": draw(st.text(min_size=1)),
|
||||
"runtime": {"cli": draw(st.sampled_from(["claude", "codex"]))},
|
||||
}
|
||||
|
||||
@given(valid_payloads())
|
||||
def test_valid_payload_always_accepted(payload):
|
||||
assert validate_payload(payload) is True
|
||||
```
|
||||
|
||||
### Stateful Testing for State Machines
|
||||
|
||||
Hypothesis can generate sequences of operations and check invariants after each step:
|
||||
|
||||
```python
|
||||
from hypothesis.stateful import RuleBasedStateMachine, rule, precondition
|
||||
|
||||
class TaskStateMachine(RuleBasedStateMachine):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.task = Task(state="pending")
|
||||
|
||||
@rule()
|
||||
@precondition(lambda self: self.task.state == "pending")
|
||||
def assign(self):
|
||||
self.task.transition("assigned")
|
||||
assert self.task.state == "assigned"
|
||||
|
||||
@rule()
|
||||
@precondition(lambda self: self.task.state == "running")
|
||||
def complete(self):
|
||||
self.task.transition("succeeded")
|
||||
assert self.task.state == "succeeded"
|
||||
|
||||
# Invariant: terminal states never change
|
||||
@invariant()
|
||||
def terminal_states_are_final(self):
|
||||
if self.task.state in ("succeeded", "failed", "cancelled"):
|
||||
with pytest.raises(InvalidTransition):
|
||||
self.task.transition("running")
|
||||
|
||||
TestTaskStates = TaskStateMachine.TestCase
|
||||
```
|
||||
|
||||
## Mutation Testing
|
||||
|
||||
Mutation testing answers: "If someone introduced a bug, would our tests catch it?"
|
||||
|
||||
Tools make small code changes (replacing `>` with `>=`, `True` with `False`, deleting statements) and check if tests still pass. Surviving mutants = test gaps.
|
||||
|
||||
### Setup with mutmut
|
||||
|
||||
```toml
|
||||
# pyproject.toml
|
||||
[tool.mutmut]
|
||||
paths_to_mutate = "entrypoint/"
|
||||
tests_dir = "tests/"
|
||||
runner = "python -m pytest tests/ -x -q"
|
||||
```
|
||||
|
||||
```bash
|
||||
# Run mutation testing
|
||||
mutmut run
|
||||
|
||||
# See surviving mutants
|
||||
mutmut results
|
||||
|
||||
# Inspect a specific mutant
|
||||
mutmut show 42
|
||||
```
|
||||
|
||||
### Practical Guidance
|
||||
|
||||
- **Target: mutation score above 80%.** Scores above 90% have diminishing returns (equivalent mutants).
|
||||
- **Focus on business logic** — validators, state machines, parsers. Skip glue code.
|
||||
- **Use mutation testing to audit AI-generated tests.** This is the most powerful combination: AI writes tests from spec, mutation testing verifies those tests catch real faults.
|
||||
- **Run on changed files only in CI** (full suite is slow). Full run nightly or pre-release.
|
||||
|
||||
## Test Architecture
|
||||
|
||||
### The Testing Pyramid for Spec-Driven Projects
|
||||
|
||||
| Layer | Proportion | Speed | What it catches |
|
||||
|---|---|---|---|
|
||||
| Unit tests | 60-70% | <1ms each | Logic errors, boundary violations, state machine bugs |
|
||||
| Property-based | 10-15% | ~10ms each | Edge cases humans miss, roundtrip failures, crash inputs |
|
||||
| Integration | 15-20% | ~100ms each | Component interaction bugs, mock/reality divergence |
|
||||
| E2E / acceptance | 5-10% | ~1s+ each | Full-chain failures, deployment config issues |
|
||||
|
||||
### Test Isolation Principles
|
||||
|
||||
- **No test depends on another test's state.** Each test sets up its own preconditions.
|
||||
- **No test depends on execution order.** `pytest-randomly` catches order dependencies.
|
||||
- **No test touches the real filesystem outside `tmp_path`.** Monkeypatch paths that default to production locations (like `/workspace`).
|
||||
- **No test makes network calls.** Mock HTTP, subprocess, and socket calls.
|
||||
- **Integration tests are marked** (`@pytest.mark.integration`) and excluded by default.
|
||||
|
||||
### Fixture Architecture
|
||||
|
||||
```python
|
||||
# conftest.py — shared fixtures, not test logic
|
||||
|
||||
@pytest.fixture
|
||||
def minimal_payload():
|
||||
"""Smallest valid payload — tests shouldn't need more unless testing specific fields."""
|
||||
return {"task_id": "test-123", "prompt": "do something", "runtime": {"cli": "claude"}}
|
||||
|
||||
@pytest.fixture
|
||||
def encode_payload():
|
||||
"""Helper: dict → base64 string (how the dispatcher passes payloads)."""
|
||||
def _encode(d):
|
||||
return base64.b64encode(json.dumps(d).encode()).decode()
|
||||
return _encode
|
||||
|
||||
# Per-module conftest for module-specific fixtures
|
||||
# tests/test_dispatcher/conftest.py
|
||||
@pytest.fixture
|
||||
def mock_backend():
|
||||
"""Fake container backend that records calls without Docker."""
|
||||
...
|
||||
```
|
||||
|
||||
### Negative Tests Are as Important as Positive Tests
|
||||
|
||||
For every "this works" test, write at least one "this fails correctly" test:
|
||||
|
||||
```python
|
||||
# Positive: valid payload accepted
|
||||
def test_p1_valid_payload_loads():
|
||||
...
|
||||
|
||||
# Negative: missing required field rejected
|
||||
def test_p3_missing_prompt_raises():
|
||||
...
|
||||
|
||||
# Negative: wrong type rejected
|
||||
def test_p_prompt_wrong_type_raises():
|
||||
...
|
||||
|
||||
# Negative: extra unknown fields are ignored (not rejected)
|
||||
def test_p_unknown_fields_ignored():
|
||||
...
|
||||
```
|
||||
|
||||
## AI Agent Testing Patterns
|
||||
|
||||
### The Two-Phase Rule
|
||||
|
||||
**Never let the same agent write both tests and implementation in one pass.** An agent that writes tests and code together will unconsciously write tests that verify its own broken assumptions.
|
||||
|
||||
The workflow:
|
||||
1. **Phase 1:** Agent reads spec → writes tests. Human reviews tests against spec.
|
||||
2. **Phase 2:** Agent (or different agent) reads spec + tests → writes implementation until tests pass.
|
||||
|
||||
### Hidden Test Splits
|
||||
|
||||
Hold back some tests that the implementing agent never sees. Use them as a final validation:
|
||||
|
||||
```python
|
||||
# tests/test_payload.py — agent sees these during development
|
||||
def test_p1_load_from_env_var(): ...
|
||||
def test_p2_missing_payload_exits_1(): ...
|
||||
|
||||
# tests/test_payload_hidden.py — agent never sees these, run post-implementation
|
||||
# (Marked with a custom marker, excluded from default run)
|
||||
@pytest.mark.hidden
|
||||
def test_p1_load_from_file_fallback(): ...
|
||||
@pytest.mark.hidden
|
||||
def test_p_concurrent_payload_loads(): ...
|
||||
```
|
||||
|
||||
### Regression Tests from Real Bugs
|
||||
|
||||
Every bug found in production or during integration testing becomes a permanent test case:
|
||||
|
||||
```python
|
||||
def test_regression_crlf_corruption():
|
||||
"""Regression: smtp-oauth-relay converted \\r\\n to \\n, breaking quoted-printable.
|
||||
Fixed by as_bytes(policy=email_policy.SMTP). See memory/gotchas-email-relay.md."""
|
||||
...
|
||||
```
|
||||
|
||||
These are the highest-value tests because they catch proven failure modes.
|
||||
|
||||
## Test Quality Metrics
|
||||
|
||||
### What to Measure
|
||||
|
||||
| Metric | Target | Why |
|
||||
|---|---|---|
|
||||
| Spec coverage | 100% | Every numbered requirement has at least one test |
|
||||
| Mutation score | >80% | Tests catch real faults, not just inflate coverage |
|
||||
| Line coverage | >90% | Baseline hygiene (necessary but not sufficient) |
|
||||
| Test speed | <10s total | Fast enough for pre-commit hooks |
|
||||
| Assertion density | >1 per test | Tests that don't assert don't catch anything |
|
||||
|
||||
### What NOT to Measure
|
||||
|
||||
- **100% line coverage as a goal.** Chasing 100% leads to tests that exercise code paths without meaningful assertions.
|
||||
- **Test count.** 50 well-targeted tests beat 200 shallow ones.
|
||||
- **Test-to-code ratio.** The ratio depends on the module's complexity, not a universal number.
|
||||
|
||||
## CI Integration
|
||||
|
||||
### Pre-commit (Every Commit)
|
||||
```bash
|
||||
pytest tests/ -x -q --tb=short -m "not integration"
|
||||
```
|
||||
|
||||
### PR Validation (Every Push)
|
||||
```bash
|
||||
# Unit + property-based tests
|
||||
pytest tests/ -q --tb=short -m "not integration"
|
||||
# Mutation testing on changed files only
|
||||
mutmut run --paths-to-mutate="$(git diff --name-only main... | grep '.py$' | tr '\n' ',')"
|
||||
```
|
||||
|
||||
### Nightly
|
||||
```bash
|
||||
# Full mutation testing
|
||||
mutmut run
|
||||
# Integration tests (requires Docker)
|
||||
pytest tests/ -m integration
|
||||
# Hidden test validation
|
||||
pytest tests/ -m hidden
|
||||
```
|
||||
|
||||
## Python Testing Gotchas
|
||||
|
||||
### `subprocess.run(check=True)` Is Invisible to Mocks
|
||||
|
||||
When you mock `subprocess.run`, the mock replaces the entire function — including the `check=True` logic that raises `CalledProcessError`. A mock returning `CompletedProcess(returncode=1)` won't trigger the exception even though the real code uses `check=True`. To test failure paths, use `side_effect=CalledProcessError(...)` explicitly.
|
||||
|
||||
### Use Routing Callables for Multi-Call Subprocess Mocks
|
||||
|
||||
When a function calls `subprocess.run` multiple times (e.g., git config, add, diff, commit, push), a fixed `side_effect` list is fragile and breaks when call order changes. Instead, use a routing callable that inspects the command: `mock_run.side_effect = lambda cmd, **kw: route_by_command(cmd)`. Clearer, more maintainable, and self-documenting.
|
||||
|
||||
### Pydantic v2 `@field_validator` Doesn't Fire for Default Values
|
||||
|
||||
`@field_validator('field_name')` never runs when the field takes its default value (e.g., `None`). Cross-field validation logic (e.g., "if type is X then field Y is required") silently passes when the dependent field is omitted. Use `@model_validator(mode='after')` for any validation that depends on multiple fields or needs to fire even when fields take defaults.
|
||||
|
||||
### Never `sys.exit()` at Module Level
|
||||
|
||||
`sys.exit()` in an `except ImportError` block at module level kills pytest collection entirely — all tests fail, not just the ones for that module. Use a flag pattern instead: `_HAS_DEPENDENCY = False` in the except block, then check `if not _HAS_DEPENDENCY: return 1` inside the function. This allows the module to be imported and mocked even when the optional dependency is missing.
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
### Tests that mirror implementation
|
||||
**Symptom:** Test asserts that function calls happen in a specific order, using mock.assert_has_calls with exact sequences. Breaks on any refactor.
|
||||
**Fix:** Test behaviour (inputs → outputs), not implementation details.
|
||||
|
||||
### Tests without assertions
|
||||
**Symptom:** `test_it_runs()` calls the function and checks it doesn't crash. No assertion on the result.
|
||||
**Fix:** Every test must assert something specific about the output, side effects, or raised exceptions.
|
||||
|
||||
### Overmocking
|
||||
**Symptom:** Every dependency is mocked. Tests pass but integration fails because mocks don't match real behaviour.
|
||||
**Fix:** Mock at the boundary (subprocess, HTTP, filesystem), not between your own modules. Use real objects for internal dependencies.
|
||||
|
||||
### Fragile tests
|
||||
**Symptom:** Tests break when unrelated code changes. Usually caused by asserting on implementation details, shared mutable state, or execution order.
|
||||
**Fix:** Test the public interface. Use fixtures for setup. Isolate each test completely.
|
||||
|
||||
### Testing private methods
|
||||
**Symptom:** Tests import `_internal_helper` and test it directly. These break on any refactor.
|
||||
**Fix:** Test through the public API. If a private method is complex enough to need its own tests, it should probably be a separate module with a public interface.
|
||||
@@ -1,60 +0,0 @@
|
||||
# Validation & Deployment
|
||||
|
||||
## Validate Locally, Deploy Once
|
||||
|
||||
The single biggest time sink across projects is "deploy first, validate later." Real-world stats from a 9-milestone infrastructure project showed 50-60% of commits were fixes that could have been caught locally.
|
||||
|
||||
**Always validate before pushing:**
|
||||
- `helm template` for Helm chart values
|
||||
- `kustomize build` (or `kubectl kustomize`) for Kustomize apps
|
||||
- `kubectl apply --dry-run=server` for K8s naming/schema issues
|
||||
- `docker run <app> validate-configuration` for apps that support it (Authelia, Homepage, etc.)
|
||||
- `docker inspect` for unfamiliar container images before writing init containers
|
||||
- Lint/typecheck/test for application code
|
||||
|
||||
**Batch fixes locally, push once.** Each push-sync-crash-fix cycle wastes minutes and clutters Git history.
|
||||
|
||||
## Test the Full Chain Immediately
|
||||
|
||||
After wiring up any new service or endpoint, test end-to-end from the user's perspective right away. Don't assume intermediate steps working means the whole chain works.
|
||||
|
||||
- `curl --resolve domain:443:<ip> https://domain` to test bypassing DNS/proxy layers
|
||||
- Test from the actual consumer (not same-namespace test pods for network policies)
|
||||
- Test DNS resolution after deploying FQDN-based policies
|
||||
|
||||
## Pre-Flight Checks
|
||||
|
||||
Before starting a deploy or automation phase:
|
||||
- Verify SSH keys are loaded (`ssh -T git@<host>`)
|
||||
- Confirm environment variables and credentials are available
|
||||
- Check that the target environment is in the expected state
|
||||
- Verify DNS records resolve as expected
|
||||
|
||||
## Scripts That Change Config Must Self-Verify
|
||||
|
||||
After updating and restarting a service, the script should test that the change actually took effect (e.g., curl an API endpoint, check a config value). A "success" message without verification hides failures.
|
||||
|
||||
## Check Container Image Runtime Requirements First
|
||||
|
||||
Before writing deployment manifests (StatefulSets, Deployments, init containers), check the image's runtime expectations: UID it runs as, writable directories it needs, filesystem layout. Use `docker inspect` or image documentation.
|
||||
|
||||
Modern images often run as non-root with specific writable directory requirements that aren't obvious from docs alone. Discovering these at deploy time wastes an entire push-crash-fix cycle per missed requirement.
|
||||
|
||||
## Verify Counts and Summaries Mechanically
|
||||
|
||||
After editing specification or documentation files that include summary counts (e.g., "14 requirements"), verify them with `grep` or `wc` rather than counting manually. Manual counting of dozens of items is error-prone and produces incorrect summaries that erode trust in the documentation.
|
||||
|
||||
## Test Pre-Commit Hooks Manually After Adding Dependencies
|
||||
|
||||
Run `bash .githooks/pre-commit` (or your hook path) manually after adding new dependencies or changing test imports. Hidden virtual environments (`.venv/`) that the hook discovers before system Python can cause `ModuleNotFoundError` at commit time even though tests pass from the terminal. Discovering hook failures during a real commit wastes debugging effort on environment issues rather than code issues. After adding a dependency, check all Python environments: `find . -name "activate" -o -name "pytest"` to discover venvs, and install into each.
|
||||
|
||||
## Order Multi-Step Migrations Carefully
|
||||
|
||||
When performing multi-step changes on remote systems (port changes, firewall rules, service migrations), plan explicit ordering to avoid lockout:
|
||||
|
||||
1. Open the new path first (new port, new firewall rule)
|
||||
2. Migrate the service to use the new path
|
||||
3. Add redirects or backward-compatibility rules
|
||||
4. Remove the old path
|
||||
|
||||
Doing all steps at once risks losing access if any step fails. Plan the ordering upfront, not mid-deploy.
|
||||
@@ -1,77 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
# sync-best-practices.sh — Sync best-practices/ from skynet/best-practices repo
|
||||
# Usage: scripts/sync-best-practices.sh [--dry-run]
|
||||
set -euo pipefail
|
||||
|
||||
REPO_URL="https://gitea.oreillyit.nz/skynet/best-practices.git"
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
|
||||
BP_DIR="$REPO_ROOT/best-practices"
|
||||
BESTPRACTICES_MD="$REPO_ROOT/BESTPRACTICES.md"
|
||||
DRY_RUN=false
|
||||
|
||||
if [[ "${1:-}" == "--dry-run" ]]; then
|
||||
DRY_RUN=true
|
||||
echo "[dry-run] No changes will be written or committed"
|
||||
fi
|
||||
|
||||
TMPDIR="$(mktemp -d)"
|
||||
trap 'rm -rf "$TMPDIR"' EXIT
|
||||
|
||||
echo "Cloning $REPO_URL ..."
|
||||
git clone --depth=1 "$REPO_URL" "$TMPDIR/best-practices"
|
||||
|
||||
# Copy all .md files from the source repo into best-practices/
|
||||
echo "Copying .md files to $BP_DIR ..."
|
||||
if ! $DRY_RUN; then
|
||||
find "$TMPDIR/best-practices" -maxdepth 1 -name "*.md" -exec cp {} "$BP_DIR/" \;
|
||||
# Also copy subdirectory .md files preserving structure
|
||||
find "$TMPDIR/best-practices" -mindepth 2 -name "*.md" | while read -r src; do
|
||||
rel="${src#$TMPDIR/best-practices/}"
|
||||
dest="$BP_DIR/$rel"
|
||||
mkdir -p "$(dirname "$dest")"
|
||||
cp "$src" "$dest"
|
||||
done
|
||||
else
|
||||
echo "[dry-run] Would copy:"
|
||||
find "$TMPDIR/best-practices" -name "*.md" | sed "s|$TMPDIR/best-practices/||"
|
||||
fi
|
||||
|
||||
# Update BESTPRACTICES.md topic list from INDEX.md, preserving the redirect header
|
||||
INDEX_MD="$TMPDIR/best-practices/INDEX.md"
|
||||
if [[ -f "$INDEX_MD" ]]; then
|
||||
echo "Updating $BESTPRACTICES_MD topic list from INDEX.md ..."
|
||||
if ! $DRY_RUN; then
|
||||
# Extract the redirect header (everything up to and including the first blank line after the blockquote)
|
||||
HEADER=$(awk '/^> \*\*Note:\*\*/{found=1} found{print} found && /^$/{exit}' "$BESTPRACTICES_MD")
|
||||
if [[ -z "$HEADER" ]]; then
|
||||
# Fallback: preserve first 4 lines (the note block)
|
||||
HEADER=$(head -4 "$BESTPRACTICES_MD")
|
||||
fi
|
||||
|
||||
# Write header + blank line + INDEX.md content
|
||||
{
|
||||
echo "$HEADER"
|
||||
echo ""
|
||||
cat "$INDEX_MD"
|
||||
} > "$BESTPRACTICES_MD"
|
||||
else
|
||||
echo "[dry-run] Would update $BESTPRACTICES_MD from INDEX.md"
|
||||
fi
|
||||
else
|
||||
echo "No INDEX.md found in source repo — skipping BESTPRACTICES.md update"
|
||||
fi
|
||||
|
||||
# Commit if there are changes
|
||||
if ! $DRY_RUN; then
|
||||
cd "$REPO_ROOT"
|
||||
if git diff --quiet && git diff --cached --quiet; then
|
||||
echo "No changes to commit."
|
||||
else
|
||||
git add best-practices/ BESTPRACTICES.md
|
||||
git commit -m "Sync best-practices from skynet/best-practices"
|
||||
echo "Committed sync changes."
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "Done."
|
||||
Reference in New Issue
Block a user