distill: best practices from 2026-04-19 cross-project run

Adds 3 new topic files (ai-parallel-agents, api-integration,
python-patterns) and extends 21 existing topic files with new gotchas
and patterns surfaced from memory across tracked projects. Index
updated accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

validation.md

@@ -22,6 +22,8 @@ After wiring up any new service or endpoint, test end-to-end from the user's per
 - Test from the actual consumer (not same-namespace test pods for network policies)
 - Test DNS resolution after deploying FQDN-based policies
 
+**Healthy != reachable.** A container showing `healthy` (pg_isready, HTTP 200, internal probe) can still have stale or broken external listeners — e.g., a TCP port bound to a namespace that was recreated, or an auth-backed endpoint accepting default credentials. Always probe the actual path users traverse: `nc -zw2 <ip> <port>` for TCP, `curl` the public URL, test auth APIs with known-bad and known-good creds. Probing the real path saves diagnosis time when the container's internal health signal diverges from external reachability.
+
 ## Pre-Flight Checks
 
 Before starting a deploy or automation phase:
@@ -112,3 +114,53 @@ When confirming whether a fix is deployed, `kubectl exec deploy/<name> -- ls <pa
 ## Think Through K8s Constraints Before Coding Docker-First Solutions
 
 Before implementing a feature that works in Docker, enumerate the K8s differences: read-only Secret volumes, root-owned files, no host-path mounts, separate pod filesystem, env var size limits. Design for both backends upfront. Planning for both environments from the start eliminates costly iteration cycles.
+
+## Inert-by-Default Strategy for Risky Feature Flags
+
+When shipping a significant behaviour change (new persistence layer, new auth path, changed state machine), ship the code in an inert state first — hidden behind a feature flag that defaults to off. Let the code deploy and stabilise in production without activating the behaviour. Then flip the flag in a separate commit or config change. This decouples "did the deploy succeed?" from "did the feature work?" and gives a clean rollback path (revert the flag) without reverting code.
+
+**Pattern:** `ENABLE_NEW_FEATURE=false` ships with the code. A follow-up change flips it to `true`. If the feature has issues, revert only the flag change.
+
+## Categorise Integration Failures by Root Cause Before Fixing
+
+When an integration test run produces multiple failures, resist the urge to fix them one by one. Categorise first:
+
+1. **Infrastructure** — missing env vars, wrong import paths, network timeouts
+2. **Interface drift** — a shared data model changed and callers weren't updated
+3. **Test rot** — test assumptions diverged from the current implementation
+4. **Logic bugs** — actual code defects
+
+Fixing by category is more efficient than fixing in order: all infrastructure failures have the same root fix, all interface drift failures share a data model change, etc. Categorising upfront also reveals the true scope — "3 logic bugs" is manageable; "14 failures across 4 categories" requires a different approach.
+
+## Use `additionalProperties: false` Everywhere in JSON Schemas
+
+For schema-validated YAML/JSON configs, apply `additionalProperties: false` to every `type: object` — except intentionally open maps (explicitly documented). Typos in config keys (`maxlength` vs `max_length`) are the #1 config error source; without strict rejection, they silently pass validation and produce incorrect behaviour instead of a clear error. Helm covers this for charts; generalise the pattern to all schema-validated configs.
+
+## JSON Schema Draft Choice Affects Tuple Validation Syntax
+
+Draft 2020-12 uses `prefixItems` for positional tuple validation and `items: false` to disallow extras. The `items`-as-array syntax (`items: [{type: string}, {type: number}]`) is Draft-04/07 only — using it under 2020-12 raises `jsonschema.SchemaError`. Always declare `$schema` explicitly in the schema root so readers and validators agree on the dialect.
+
+## Quote YAML Reserved-Word Keys
+
+In YAML, unquoted reserved words as keys parse to their typed values: `null: true` becomes `{None: True}` in Python, not `{"null": True}`. The same applies to `true`, `false`, `yes`, `no`, `on`, `off`. Always quote reserved-word keys: `"null": true`. Add a schema-level regex or loader-level check to catch unquoted reserved-word keys before they silently break consumers — in one project this hit 45 occurrences across 4 manifests.
+
+## Coverage Thresholds vs Marker-Filtered Test Runs
+
+`pytest -m integration` (or any marker-filtered subset) fails `--cov-fail-under` because most tests are deselected, dropping total coverage below the gate even when the filtered tests pass. Similarly, stub modules with `raise NotImplementedError` drop coverage — def/import lines still count. Use `--no-cov` for marker-filtered/smoke CI steps and run full-coverage as a separate step. When scaffolding stubs during milestones, temporarily lower the threshold and restore it after implementation.
+
+## Research Target Format and Feature Availability Before Building Artifacts
+
+When integrating with an external platform (Platform Hub, Octopus, any product with its own schema/scope rules), spend the first hour reading the target format documentation (OCL syntax, API schema, scope rules) and verifying feature availability before committing to an approach. Two failure modes this prevents:
+
+1. **Wrong-format artifacts** — e.g., 12 step templates created via API before discovering the target's process templates can't reference step templates cross-space. All the work was wasted.
+2. **Unavailable features** — ask "what features of this product are actually available to me right now?" before planning. Assuming an unreleased feature (e.g., Project Templates) is available caused a plan to evolve three times mid-implementation.
+
+## Safe Persistence Pattern: Persist-on-Change, Rehydrate-at-Startup, Clean-Slate Fallback
+
+For services that maintain in-memory state that should survive restarts (task queues, session state, model scores):
+
+1. **Persist on change** — write state to disk immediately when it changes, not on a timer
+2. **Rehydrate at startup** — read the persisted state file during service initialisation
+3. **Clean-slate fallback** — if the state file is missing, corrupt, or incompatible, start with empty state and log a warning rather than crashing
+
+This pattern handles pod restarts, rolling deploys, and upgrade scenarios without requiring an external database. The clean-slate fallback is critical — a startup crash due to a corrupt state file is worse than losing the state.