distill: best practices from 2026-04-19 cross-project run

Adds 3 new topic files (ai-parallel-agents, api-integration,
python-patterns) and extends 21 existing topic files with new gotchas
and patterns surfaced from memory across tracked projects. Index
updated accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

llm-code-security.md

@@ -777,6 +777,116 @@ Use this checklist when reviewing LLM-generated code:
 
 ---
 
+## 11. Gated Model Downloads Require Out-of-Band License Acceptance
+
+### What goes wrong
+
+HuggingFace (and similar model hubs) return **403 Forbidden** for "gated" models even when the HTTP request carries a valid user token. The hub enforces that the token's user has manually accepted the license agreement on the web UI for that *specific* model. Automation cannot bypass this — there is no API to accept the license.
+
+This breaks reproducible-build scripts, container bake pipelines, and agent workflows that pull third-party ML models: the first run on a fresh account/token fails with an opaque 403 and no hint that human action is required.
+
+### Pattern
+
+Bake a pre-flight check into every model-download script:
+
+```python
+def preflight_gated_model(model_id: str, token: str) -> None:
+    r = requests.head(f"https://huggingface.co/{model_id}/resolve/main/config.json",
+                      headers={"Authorization": f"Bearer {token}"},
+                      allow_redirects=True)
+    if r.status_code == 403:
+        url = f"https://huggingface.co/{model_id}"
+        raise SystemExit(
+            f"Model {model_id} is gated. Open {url} in a browser, sign in as "
+            f"the token owner, accept the license, then rerun this script."
+        )
+    r.raise_for_status()
+```
+
+### Rules
+
+- **Surface a human-readable error on 403** — do not retry, do not fall back to a different model silently
+- **Include the exact URL** to visit and the exact action required ("accept the license")
+- **Check every gated model** at pipeline start, not lazily at download time, so the human step is front-loaded
+- **Document which models are gated** in the project's README — license acceptance is per-user, so every new operator needs to do it once
+
+Applies to any project pulling third-party ML models from HuggingFace, Meta's Llama portal, Stability AI's hub, or similar.
+
+---
+
+## 12. Operational Vulnerabilities in AI-Generated Code (Beyond Traditional SAST)
+
+Traditional SAST tools (Bandit, Semgrep, SonarQube, Snyk) focus on known vulnerability patterns — injection, XSS, hardcoded secrets. But analysis of AI-generated code at scale (538,860 findings across 3,518 scans, SentinaLayer 2026) reveals that the **top vulnerability categories are structural and operational**, not traditional:
+
+| Category | % of P0-P2 Findings | What SAST Misses |
+|---|---|---|
+| CI/CD Integrity Gaps | 31% | Gate ordering, missing dependency chains, workflow step sequencing |
+| Backend Reliability | 27% | Missing idempotency keys, premature health checks, retry logic gaps |
+| Security Overlay | 24% | Supply chain trust gaps, unverified checksums, unsigned artifacts |
+| Supply Chain Provenance | 11% | Missing signature verification, unpinned base images in multi-stage builds |
+| Data Layer Integrity | 7% | Unsafe write paths, missing concurrent-access guards, race conditions |
+
+**The single most common P0-P2 finding: missing idempotency keys in webhook handlers (18% of all critical/high/medium findings).** This is not a vulnerability that any SAST tool checks for.
+
+### Why AI agents produce these
+
+AI coding agents optimise for **functional correctness** — does the code produce the right output for the happy path? They consistently miss:
+
+1. **Retry safety.** Webhook handlers, API endpoints, and event processors that work correctly on first invocation but corrupt data on retry. Agents don't model what happens when the same request arrives twice.
+
+2. **Ordering dependencies.** CI/CD pipelines where steps depend on prior steps' artifacts but the dependency isn't explicit. Works when steps happen to run in order, breaks under parallelism or partial failure.
+
+3. **Health check timing.** Services that report healthy before their dependencies are ready. The agent sees "return 200 from /health" and implements it, without considering that the database connection pool hasn't warmed yet.
+
+4. **Checksum and signature verification.** Agents download artifacts, pull images, and install packages without verifying integrity. The code works, but the supply chain is unverified.
+
+5. **Concurrent write safety.** File operations, database writes, and cache updates that work under single-threaded testing but corrupt under concurrent access. Agents don't think about lock ordering or write-after-read races.
+
+### Deterministic checks you can add to CI
+
+These can be implemented as fast pre-scan rules (regex + AST) that run before expensive test suites:
+
+**Idempotency:**
+- Webhook handlers without idempotency key extraction/dedup
+- POST endpoints that create resources without checking for existing duplicates
+- Event processors without at-least-once safety (no dedup by event ID)
+
+**CI/CD Integrity:**
+- GitHub/Gitea Actions steps that reference artifacts from prior steps without explicit `needs:`
+- Dockerfile `COPY --from=` referencing stages without explicit ordering
+- Helm hooks without `hook-weight` when ordering matters
+
+**Supply Chain:**
+- `curl | bash` or `wget -O- | sh` without checksum verification
+- Container images pulled by tag without digest pinning
+- Package installation without lockfile or hash verification
+- `go install` / `pip install` from URLs without integrity checks
+
+**Health Checks:**
+- HTTP health endpoints that return 200 unconditionally (no dependency readiness check)
+- Liveness probes identical to readiness probes (should be different — liveness checks "am I stuck?", readiness checks "am I ready to serve?")
+- `initialDelaySeconds: 0` on readiness probes for services with startup dependencies
+
+**Concurrent Access:**
+- File writes without advisory locking (`fcntl.flock` / `flock`)
+- Database read-then-write sequences without transactions or SELECT FOR UPDATE
+- Cache operations without atomic compare-and-swap
+
+### Review checklist (operational)
+
+- [ ] Every webhook handler extracts and deduplicates by an idempotency key
+- [ ] Every POST endpoint that creates resources checks for pre-existing duplicates
+- [ ] CI/CD steps declare explicit dependencies on prior steps' outputs
+- [ ] Downloaded artifacts are verified by checksum or signature before use
+- [ ] Container base images are pinned by digest, not just tag
+- [ ] Health check endpoints verify dependency readiness, not just "process is alive"
+- [ ] Readiness and liveness probes serve different purposes
+- [ ] File and database writes under concurrent access use appropriate locking
+- [ ] Event processors handle at-least-once delivery (idempotent or deduplicating)
+- [ ] Multi-stage Docker builds have explicit stage ordering and artifact dependencies
+
+---
+
 ## Sources
 
 - [Security Weaknesses of Copilot-Generated Code in GitHub Projects (ACM TOSEM)](https://dl.acm.org/doi/10.1145/3716848)