distill: 49 best practices from 5 projects (2026-03-27..2026-04-05)

Add 37 new entries and update 7 existing entries across 13 topic files.
Major contributions from agent-runtimes (K8s secrets, CI, Docker gotchas),
cluster-bootstrap (ArgoCD SSA, etcd tuning, DB migrations, Compose networking),
and cluster-apps/octopus-deploy (Helm vs raw manifests, ArgoCD source types).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

docker.md

@@ -45,3 +45,31 @@ The UID wrapper should resolve the target UID/GID using this priority:
 3. **Skip** — if not root or no mount point exists, run as the default container user.
 
 This makes UID matching a deployment concern (varies per host), not a configuration concern (baked into images). See [Docker UID Matching](docker-uid-matching.md) for the full UID wrapper pattern.
+
+## network_mode: service:* Breaks on Parent Container Restart
+
+When a container shares another's network namespace via `network_mode: "service:<parent>"`, restarting the parent recreates the namespace. The dependent container keeps stale socket bindings — its listeners are bound to a namespace that no longer exists. TCP connections fail while the dependent container appears healthy. Fix: add a healthcheck to the parent and use `depends_on: condition: service_healthy` on the dependent. Alternatively, use an internal bridge network instead of namespace sharing.
+
+## docker compose restart Is Concurrent, Not Ordered
+
+`docker compose restart svc1 svc2 svc3` restarts all named services concurrently, ignoring `depends_on` ordering. Dependent services may start before their dependencies are ready. Use `docker compose up -d` (which respects `depends_on`) or restart in explicit stages: stop dependents, restart the dependency, wait for healthy, then start dependents.
+
+## docker cp Can Corrupt Container Filesystem Ownership
+
+`docker cp` runs as root and can change ownership of parent directories in the container's filesystem layer. This is particularly dangerous for database containers (e.g., PostgreSQL UID 999) — copying a file into `/tmp/` can corrupt the data directory ownership, causing "Permission denied" errors. After any `docker cp` into a stateful container, verify and fix ownership: `chown -R <expected-uid>:<expected-gid> <data-dir>`.
+
+## Prefer Internal Bridge Networks Over Namespace Sharing
+
+For sidecar-style containers that need to communicate (e.g., app + database, app + TLS proxy), prefer an internal Docker bridge network over `network_mode: "service:<parent>"`. Bridge networks allow proper `depends_on` ordering with health checks, independent restart of each container, and clear network isolation. Namespace sharing couples container lifecycles — restarting the parent invalidates the dependent's network stack.
+
+## Verify Dockerfile COPY After Creating New Files
+
+After creating a file intended for a Docker image (scripts, configs, wrappers), immediately add the COPY line to the Dockerfile and verify with `docker run --entrypoint sh <image> -c "ls /path/to/file"`. Files placed in image source directories are not automatically included — they need explicit COPY instructions. This class of bug can remain latent until the code path is first exercised.
+
+## docker-compose.override.yaml Merges Lists Additively
+
+Docker Compose V2 merges list fields (ports, volumes, environment) by appending, not replacing. An override file with a different port mapping adds a second binding rather than replacing the original, causing conflicts. Modify the base `docker-compose.yaml` directly or use `!override` for list replacement.
+
+## Volume Source Paths Must Be Absolute
+
+Docker interprets relative paths in volume mount source fields as named volumes, not bind mounts. Use `os.path.abspath()` or equivalent when constructing volume source paths programmatically. The error message ("includes invalid characters for a local volume name") is misleading — the real issue is that the path is relative.