Files
best-practices/BESTPRACTICES.md
Paul O'Reilly 52cd0a3cb6 docs(airouter): 2026-05-08 dogfood batch + test design rules
Three additions to agent-repos.md based on the post-A1-incident dogfood
batch (8 successes + 1 operator-induced "failure"):

1. Airouter Qwen3.6 section: pattern reconfirmed across M16 Wave A1/A2/B1
   and M25 Waves A1-A5 + B1-B2. Time-to-success bands recorded for cost
   calibration (1m30s for git rm; ~5 min for Pydantic regex; ~12 min for
   class addition). Default --max-test-iterations 1 for cheap probes.

2. New section: Test Design for AI Agent Dogfood Pipelines. Triggered by
   the M16 Wave B2 (MN-4 prompt cap) failure — a 14-minute airouter run
   blamed on the agent that was actually an over-strict test asserting on
   sanitised 422 body content. CP's RequestValidationError handler strips
   Pydantic detail for security; tests asserting body content for that
   path are structurally impossible. Rules: verify test passes against a
   reference impl before pushing; status-code-only ceiling for validator-
   driven 422s; model on previous successes not stricter variants; F70
   retries don't recover structurally impossible tests.

3. New section: Dogfood Failure Path: Branch + Logs Lost. When all F70
   retries exhaust, the agent's last attempt is not pushed to the agents
   fork, the CP task record's logs field is empty, and the pod is gone.
   Operator must reproduce locally — until F70 finalize-on-failure pushes
   the failed branch.

BESTPRACTICES.md index updated to reflect the new sub-topics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-08 19:50:45 +12:00

5.6 KiB

Best Practices Index

Generalised best practices extracted from real project work via the /distill-best-practices skill. Each topic file is self-contained — read only the files relevant to the current project.

Topics

  • Validation & Deployment — Validate locally, deploy once; full-chain testing; pre-flight checks; DB migration patterns; K8s constraint planning; deployment checklists; inert-by-default feature flags; integration failure categorisation; safe persistence pattern
  • Security Architecture — Server boundary rule: no credential crosses to the client; proxy + identity mapping pattern; defense in depth; anti-patterns
  • Secrets Management — SOPS + age, credential handling, file naming, encryption gotchas, .env source injection, URL-safe passwords, per-workload secret scoping
  • Git & Source Control — Commit practices, GitOps workflows, remote conventions
  • Kubernetes Patterns — Volume mounts, deployment strategies, naming, bootstrap ordering, ArgoCD SSA quirks, etcd tuning, Secret volume gotchas, probe timeouts, Kustomize overlay image overrides, PodSecurity for monitoring, Cilium entity identities
  • Helm Charts — Schema validation, version verification, values structure
  • Ansible — Inventory, templates, idempotency, credential safety
  • Scripting — Shell conventions, verification scripts, idempotency, colour output
  • Documentation Standards — CLAUDE.md, MEMORY.md, FUTURE.md, README.md structure and tiered memory
  • Milestones & Reflections — Milestone workflow, verification, reflection process
  • Debugging Methodology — Systematic diagnosis, full-chain testing, common pitfalls, DB schema verification after deploy
  • Claude Code Skills — Skill authoring, context injection, tool restrictions, read-only review skills, formatter/hook separation
  • Linting & Formatting — Tool choices per language, PostToolUse hook, pre-commit integration, formatter contract
  • Spec-Driven Development — Spec structure, requirement numbering, test-first workflow, multi-model review, plan-first approach, agent prompt conventions, wave-based TDD dispatch
  • Test-Driven Development — Edge case discovery, property-based testing, mutation testing, AI agent testing patterns, test architecture
  • Networking & Infrastructure — nftables safety, systemd socket activation, Docker forwarding, TLS SNI vs Host header, wildcard certs
  • Docker UID Matching — UID wrapper entrypoint for mounted volumes, gosu pattern, when to use vs K8s securityContext
  • Database Selection — SQLite is not a production database; always use PostgreSQL for services with FQDNs, multiple consumers, or concurrent access
  • Docker — gosu PID 1, GIT_SSH_COMMAND scope, slim image health checks, buildx local images, Compose networking/restart gotchas, volume paths, override merge behaviour, init script privilege order, payload size limits, bind-mount rm gotcha
  • API Design — API-first methodology (OpenAPI 3.1 source of truth, Spectral governance, dogfooding, auth-by-default); transport, auth (OAuth2/JWT/mTLS), versioning, errors, idempotency, rate limiting, validation, zero-trust; docs/DX (Scalar/Mintlify, RFC 9457 errors, generated SDKs, RFC 9745 deprecation); contract testing (Schemathesis, Pact, oasdiff drift detection, test pyramid)
  • API Integration — Client-side third-party API integration: capability verification, app-layer compensation, git+SOPS polling sync, bidirectional SoR
  • Octopus Process Templates — OCL syntax, step template references, channel scoping, parameters, versioning, Platform Hub patterns
  • LLM Code Security — Security vulnerabilities in AI-generated code: injection flaws, hardcoded secrets, hallucinated packages, over-permissive defaults, IaC risks, crypto mistakes, operational vulnerabilities (idempotency, CI/CD integrity, supply chain provenance, concurrent access), review checklists
  • CI Container Builds — Registry cache with inline metadata, buildx in DinD, layer ordering, pip caching, path filter gotchas, SHA tagging strategy, runtime-mounted directory triggers
  • Agent Repos & Container Agents — Task submission, harnesses, monitoring, multi-model workflows, agent repo forks, workspace layout, artifact passing via git branches, read-only test protection, infrastructure failure modes, cost-effective model scope boundaries, airouter dogfood patterns + time-to-success bands, dogfood test design (status-code-only ceiling for sanitised 422s), failure-path branch+log loss
  • AI Parallel Agents — Parallel agent orchestration: multi-facet research dispatch, file contention, WebFetch limits, narrow reads, dataset-wide audits
  • Python Patterns — Non-reentrant Lock deadlocks, Pydantic v2 extra='ignore' silent drops, subprocess routing callables for mocking, model_validator for cross-field validation
  • Mechanical Test Generation — Spec properties that enable automated test writing: module layout tables, integration boundary marking, explicit library semantics, concrete interfaces over "implementation detail", error messages as test data, pattern tables as parametric matrices, pre-dispatch testability review