Populate best practices from claude-foundations

Migrates 20 topic files from claude-foundations/best-practices/ to this
standalone repo. Adds BESTPRACTICES.md index, CLAUDE.md conventions, and
updated README.md. Container agents clone this repo to /best-practices.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

scripting.md

@@ -0,0 +1,56 @@
+# Scripting Conventions
+
+## Structure
+
+- All scripts live in `scripts/` and run from the repository root
+- Scripts should be idempotent and safe to re-run
+- Exit non-zero on failure so `&&` chains work naturally
+
+## Verification Scripts
+
+- Automated checks confirming milestone or feature outcomes
+- Use colour output (green/red) for pass/fail indicators
+- Should be non-destructive and environment-resilient
+- Avoid needing sudo — test from the accessible side of a connection instead
+- Check for default/insecure credentials and print remediation instructions on failure
+- Use `curl --resolve` to bypass DNS/proxy layers when testing direct connectivity
+- **Never use `set -e` in verification scripts.** A verify script's job is to run ALL checks and report a summary. `set -e` exits on the first failure, hiding remaining issues. Use explicit conditional checks and a pass/fail counter instead. Note: `((var++))` under `set -e` is a classic bash trap — pre-increment of 0 evaluates to falsy, triggering errexit. Use `var=$((var + 1))`.
+
+## Automation Triggers
+
+If you run the same 3+ commands in sequence more than once, it should become a script. Look for:
+- Repeated command sequences in conversation history
+- Steps requiring careful ordering
+- Multi-step manual processes that are error-prone
+
+## Error Handling by Tool Purpose
+
+Not all scripts need the same error handling strategy:
+- **Destructive scripts** (deploy, configure, delete) should use `set -euo pipefail` — fail fast on any error.
+- **Reporting/read-only scripts** (status dashboards, aggregation, monitoring) should start without `set -e` — complex data collection from multiple sources is hard to debug under errexit. Use explicit conditional checks instead.
+- **The choice depends on the tool's purpose.** A script that writes to production needs strict error handling. A script that reads from 10 sources and aggregates results needs resilience.
+
+## Dryrun Mode
+
+Every script that modifies state should support `--dryrun` / `-n`:
+- Makes the script self-documenting about its side effects
+- Enables safe testing and review before execution
+- Enables test harnesses that verify output without executing changes
+- Dryrun output should show exactly what would happen, not a summary
+
+## Shell Gotchas
+
+- `((PASS++))` fails under `set -e` when PASS=0 — the expression evaluates to 0 (false), triggering errexit. Use `PASS=$((PASS + 1))` instead.
+- `set -e` silently terminates complex pipelines and subshells with no output — makes debugging extremely difficult. Also kills command substitutions that capture non-zero exit codes (e.g., `result=$(grep "pattern" file)` exits if grep finds nothing).
+- `grep` interprets option-like strings (starting with `-`) as flags — use `--` terminator before patterns or input that may start with dashes.
+- Always quote variables in conditionals and file paths
+- Use `trap` for cleanup of temp files and credentials
+- **Use `git diff --numstat` for binary file detection** instead of `file`. The `file` command is unreliable (marks shell scripts as "executable"), while `git diff --numstat` shows `-` for binary files using git's robust binary detection heuristics.
+- **Use `cat -A` to diagnose invisible character issues.** Reveals non-printing characters like em dashes, zero-width spaces, and smart quotes that look identical to correct characters but break YAML parsers, config files, and frontmatter. Essential when a file looks correct but tooling rejects it.
+- `grep` returns exit code 1 when no lines match — under `set -e`, this kills the script even when zero matches is a valid outcome. Append `|| true` to `grep` commands in pipelines where empty results are expected.
+- `while read` in a pipeline creates a subshell — variables modified inside the loop (counters, accumulators) are lost after the loop ends. Use process substitution (`while read line; do ...; done < <(command)`) or here-strings to keep the loop in the current shell.
+- **Order matters in sed/regex transformation pipelines.** Process more specific patterns before general ones. For example, if both `![[image.png]]` and `[[page]]` are valid patterns, process the image embed first — otherwise the general wikilink regex matches the inner `[[image.png]]` and the `!` prefix is left orphaned.
+
+## JSON Construction in Scripts
+
+Use Python (not shell) for constructing JSON payloads. Multi-line prompts with quotes, backticks, and special characters break shell-based JSON construction (printf/sed/heredocs). Python's `json.dump` handles escaping correctly every time. For scripts that need to construct and submit JSON payloads, write the construction logic in Python even if the rest of the script is bash.