init: seed framework reference content from agent-runtimes main repo

harnesses/contexts/planning/v1/best-practices/kubernetes.md

@@ -0,0 +1,69 @@
+# Kubernetes Patterns
+
+## Volume Mounts
+
+- **Avoid `subPath` volume mounts** for Secrets and ConfigMaps. The kubelet does not auto-update `subPath` mounts when the source changes — the pod must be restarted. Use directory mounts instead and adjust the application's config path.
+- **Secret volume propagation is async.** After updating a Secret, the kubelet takes seconds to sync mounted volumes. A `rollout restart` issued immediately after may start pods with stale data. Add a short delay (5s) before restarting.
+
+## Deployment Strategies
+
+- **RWO PVC + RollingUpdate = Deadlock.** New pod can't attach the volume while the old pod holds it. Use `strategy: Recreate` for single-replica deployments with RWO PVCs.
+- **SSA + strategy change conflict.** Switching from RollingUpdate to Recreate via ServerSideApply fails because SSA won't remove the old `rollingUpdate` field. Must patch the live resource first.
+
+## Naming
+
+- `metadata.name` must be DNS-1035 compliant — no dots allowed. Replace dots with dashes (e.g., `oreillyit-nz` not `oreillyit.nz`). Label values CAN contain dots.
+
+## Bootstrap Ordering
+
+Some components have chicken-and-egg dependencies:
+1. CNI (e.g., Cilium) must be installed before anything else — nodes are NotReady without it
+2. GitOps controller (e.g., ArgoCD) installed second
+3. Root app applied last — the GitOps controller then "adopts" CLI-installed releases
+
+Manual bootstrap secrets (encryption keys, OIDC client secrets) must be documented as explicit steps.
+
+## Network Policies
+
+- DNS egress for `toFQDNs` rules must use `toEndpoints` targeting kube-dns pods with `rules.dns` — this triggers the DNS proxy. Using `toCIDRSet` for DNS bypasses the proxy and FQDN rules never populate.
+- Cross-namespace policies need explicit namespace matching (e.g., `matchExpressions` on namespace label).
+- Always test from the actual consumer namespace, not same-namespace test pods.
+
+## Probe Strategy
+
+- **Liveness vs readiness probes serve different purposes.** TCP checks confirm the process is listening (liveness). Exec/command checks confirm the application is ready to serve (readiness). Don't conflate them.
+- **Probes must match application host validation.** Applications that validate Host headers (e.g., Next.js `ALLOWED_HOSTS`) will reject probes sent to the pod IP. Set `httpGet.httpHeaders` with the expected Host value.
+- **Don't load credentials into liveness probes.** If readiness requires an authenticated check (e.g., `sqlcmd`), use a simple TCP check for liveness and reserve the authenticated check for readiness only.
+
+## Init Container Patterns
+
+- **Writable config via init container + emptyDir.** When apps require writable directories but ConfigMaps are read-only, use an init container to copy config into an emptyDir volume that the main container mounts read-write.
+- **Privilege separation.** Init containers can run as root to create directories or set ownership, while the main container runs as a non-root UID. Prefer this over running the entire workload as root.
+- **Non-root images have hidden filesystem requirements.** Many modern images (e.g., MSSQL 2022, UID 10001) need writable directories beyond the obvious ones. Always check image documentation or `docker inspect` before writing manifests.
+
+## StatefulSet Edge Cases
+
+- **CrashLoopBackOff pods won't auto-replace on spec update.** The StatefulSet controller won't delete and recreate a crashing pod when you update the spec — manual `kubectl delete pod` is required to force recreation.
+- **Immutable field diffs can deadlock auto-sync.** StatefulSet fields like `volumeClaimTemplates` are immutable after creation. GitOps controllers (ArgoCD) will show permanent OutOfSync if the desired state differs from the live immutable fields. Force sync or recreate the StatefulSet.
+- **SSA causes perpetual OutOfSync from defaulted fields.** Kubernetes defaults fields on StatefulSets (`persistentVolumeClaimRetentionPolicy`, `revisionHistoryLimit`, `updateStrategy.rollingUpdate.partition`) that aren't in the Helm template. With `ServerSideApply=true`, GitOps controllers see these as diffs and report OutOfSync even though the app is Healthy. The app functions correctly — this is cosmetic. Consider ArgoCD `ignoreDifferences` for these fields.
+
+## GitOps: Imperative vs Declarative
+
+- **Never use imperative operations on GitOps-managed resources.** `kubectl rollout restart` adds annotations that conflict with the GitOps controller's desired state, causing permanent OutOfSync. Use declarative paths instead — update a configmap hash annotation in Git, or change a pod template label.
+- **ArgoCD reconciliation has latency.** New Application manifests don't appear immediately due to polling intervals. Use manual refresh annotations when automation needs immediate reconciliation.
+
+## PodSecurity Alignment
+
+- **Namespace PodSecurity labels must match container security contexts.** DinD, CSI drivers, and other privileged workloads need `pod-security.kubernetes.io/enforce: privileged` on their namespace. A `baseline` or `restricted` namespace silently blocks privileged pods.
+- **Document privileged namespace requirements.** When a workload needs elevated privileges, document the specific requirement (e.g., "Docker-in-Docker for CI builds") alongside the namespace label.
+
+## ArgoCD Source Type Detection
+
+- **ArgoCD auto-detects Kustomize.** When a source directory contains `kustomization.yaml`, ArgoCD runs Kustomize automatically. Adding an explicit `directory:` source type overrides this detection and causes ArgoCD to try applying `kustomization.yaml` as a raw K8s resource, which fails with schema errors. Remove explicit directory source types from Kustomize sources.
+- **Credential template URL-prefix must match exactly.** ArgoCD repo-creds secrets use URL prefix matching. When migrating Git server URLs (hostname, protocol, or port changes), update the credential template to match the new prefix. Stale credentials cause "authentication required" errors on all apps using that prefix.
+
+## Miscellaneous
+
+- `enableServiceLinks: false` may be needed when K8s-injected service env vars conflict with app config (e.g., Authelia interprets `AUTHELIA_*` service vars as configuration).
+- Proxmox VM names must match K8s node hostnames for cloud controller manager integration.
+- Metrics-server on Talos needs `--kubelet-insecure-tls` (self-signed kubelet certs).