Initial commit: Claude Code foundations and improvements research

Conventions, community best practices research (Sept 2025 - March 2026),
and prioritized improvement backlog for Claude Code workflows.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

CLAUDE.md

@@ -0,0 +1,187 @@
+# CLAUDE.md — General Project Guidelines
+
+## Session Start
+
+1. **Immediately** (without waiting for user input) list the project directories under `~/dev/claude/` (excluding `secrets/`) and present a numbered menu like:
+
+   > What are we working on today?
+   >
+   > 1. **cluster-bootstrap** — Kubernetes homelab cluster
+   > 2. **custom-claude-skills** — Reusable Claude Code skills
+   > ...
+   > N-1. **No project right now** — just chat
+   > N. **New project!** — start something new
+
+   Scan the directories at runtime so the list is always current. Include a brief description if the project has a CLAUDE.md or README.md you can glean one from.
+
+2. Based on the user's choice:
+   - **Existing project**: `cd` into the directory, read all `.md` files, and read `~/dev/claude/secrets/` (read-only reference — review every file to refresh context). Ask clarifying questions if anything is unclear or incomplete, and note context in MEMORY.md.
+   - **No project right now**: Do nothing further — just respond normally.
+   - **New project!**: Follow the "New Projects" section below. Also read `~/dev/claude/secrets/` as above.
+
+## Secrets (`~/dev/claude/secrets/`)
+
+**CRITICAL — treat this folder with extreme paranoia:**
+
+- Files in `~/dev/claude/secrets/` are **read-only**. Never edit them.
+- **Never** copy, echo, write, or reproduce secret values into any other file — not MEMORY.md, not CLAUDE.md, not commit messages, not scripts, not tool output, nowhere.
+- **Never** include secret values in git commits, diffs, or changelogs of any project.
+- **Never** pass secret values as command-line arguments (visible in `ps` output). Use `@file` references, environment variables sourced at runtime, or stdin.
+- When a task requires a secret, read it at execution time from the secrets folder and use it ephemerally. Do not cache or persist the value.
+- It is acceptable to reference the **existence** of a secret file (e.g., "credentials are in `~/dev/claude/secrets/gitea/ai_enablement`") but never its contents.
+
+## New Projects
+
+If this is a new project:
+
+1. Create a new directory under `~/dev/claude/<project-name>/`
+2. **Ask the user** which Gitea user/org the repo should be created under (e.g., `homelab`, a personal user, etc.) before setting up the remote
+3. Create the initial standard files:
+   - **CLAUDE.md** — Project-specific architecture, conventions, repo structure, and working instructions for Claude
+   - **MEMORY.md** — Persistent learnings, gotchas, reflections, and process improvements
+   - **FUTURE.md** — Ideas and improvements not on the active roadmap (Problem/Idea/Open questions/Depends on format)
+   - **README.md** — Human-readable overview, quick start, milestones, and scripts reference
+4. Parse the other CLAUDE.md files from sibling project folders in `~/dev/claude/`, and based on the type of project being considered, bring over related practices, guidelines, and learnings
+
+## Source Control
+
+- All projects are hosted on **Gitea** (`gitea.oreillyit.nz`) as the primary remote — prefer this hostname over `gitea.homelab.internal` (same instance, but the public name enables external access)
+- Migrate existing remotes from `gitea.homelab.internal` to `gitea.oreillyit.nz` when convenient
+- AI-focused projects go under the **`skynet`** org; infrastructure projects under **`homelab`**
+- SSH workflows preferred. SSH config uses host aliases per Gitea user:
+  - `gitea.oreillyit.nz-homelab` → authenticates as `cluster-administrator` (key: `~/.ssh/gitea-cluster-admin`)
+  - `gitea.oreillyit.nz-ai-enablement` → authenticates as `ai_enablement` (key: `~/.ssh/gitea.ai-enablement`)
+- Git remote URLs use the alias: `git@gitea.oreillyit.nz-<user>:<org>/<repo>.git`
+  - Example: `git@gitea.oreillyit.nz-ai-enablement:skynet/custom-claude-skills.git`
+- Optionally push-mirror to GitHub for public visibility
+- Use meaningful commit messages; prefer small, focused commits over large batches
+- Enable pre-commit hooks where appropriate (secret detection, linting, formatting)
+- Never commit secrets in plaintext — use SOPS + age or equivalent encryption
+
+## Documentation Standards
+
+Every project maintains four core markdown files:
+
+### CLAUDE.md (per-project)
+The primary reference for Claude sessions. Should contain:
+- Project overview and architecture
+- Repository structure (keep updated as the project evolves)
+- Key design decisions with rationale
+- Conventions and coding standards
+- Environment details (IPs, URLs, credentials references)
+- Common operations / how-to recipes
+
+### MEMORY.md (Tiered Memory System)
+Long-running projects accumulate significant context. To keep MEMORY.md useful rather than bloated, use a **tiered memory** structure:
+
+**MEMORY.md** is a **thin index only** — one-line descriptions with links to topic files in `memory/`. No content lives in MEMORY.md itself. Think of it as a card catalog. Keep it under ~50 lines.
+
+**memory/** contains the actual content, split by topic:
+- `memory/project-status.md` — Current milestone, what's next, blockers
+- `memory/network.md` — IPs, VIPs, subnets, topology
+- `memory/gotchas-<topic>.md` — Gotchas grouped by technology (e.g., `gotchas-cilium.md`, `gotchas-authelia.md`)
+- `memory/process-lessons.md` — How-to-work-with-this-repo lessons for Claude
+- `memory/m<N>-reflection.md` — One file per milestone reflection (these are time-bound, so per-file is natural)
+- `memory/decisions.md` — Architecture and design decisions made during planning
+
+**Principles:**
+- **Split by topic, not by time.** A Cilium gotcha belongs in `gotchas-cilium.md` whether discovered in M5 or M8.
+- **Milestone reflections are the exception** — inherently time-bound, one file per milestone.
+- **Index descriptions matter.** They're used to decide what to read. "Cilium L2/LB gotchas and externalTrafficPolicy quirks" beats "cluster stuff".
+- **Prune aggressively.** If a gotcha was fixed (e.g., chart version upgraded past the bug), delete it. Stale memory is worse than no memory.
+- **Each memory file should be self-contained and greppable.** Include enough context that the file makes sense on its own.
+- **Deduplicate with CLAUDE.md.** Conventions and patterns that are stable should live in CLAUDE.md. Memory files are for learnings, gotchas, and reflections that accumulate over time. If something in memory has graduated to a stable convention, move it to CLAUDE.md and remove it from memory.
+
+**When reading memory at session start:** Read MEMORY.md (the index), then selectively read topic files relevant to the current task. Don't read all memory files unless doing a broad review.
+
+**When writing memory after a milestone:** Create the reflection file, update any affected topic files (new gotchas, updated status), and update the index.
+
+### FUTURE.md
+Backlog of improvement ideas, each with:
+- **Problem:** What's painful or manual today
+- **Idea:** What the improvement looks like
+- **Open questions:** Unknowns to research before starting
+- **Depends on:** Other items or milestones that should come first
+
+### README.md
+Human-readable project documentation:
+- Architecture summary
+- Quick start / setup instructions
+- Milestone table with status
+- Scripts section listing every script with purpose and usage
+
+## Milestones
+
+Break projects into numbered milestones (M1, M2, ...). Every milestone completion MUST include:
+
+1. **Verification script** (`scripts/verify-m<N>.sh`) — automated checks confirming all milestone outcomes. Scripts should be idempotent, non-destructive, and return non-zero on failure. Use colour output (green/red) for pass/fail.
+2. **Milestone reflection** in `memory/m<N>-reflection.md` — review the entire conversation and capture:
+   - Process improvements (what slowed us down, wrong assumptions, backtracking)
+   - Key knowledge for reproduction (gotchas, version quirks, debugging detours)
+   - Scripts and automation opportunities (repeated command sequences → scripts)
+   - Future improvement ideas (add to FUTURE.md)
+   - Update affected topic files in `memory/` (new gotchas, updated status) and the MEMORY.md index
+3. **Updated README.md** — scripts section, milestone table, any new setup steps
+4. **Updated CLAUDE.md** — repo structure, conventions, new patterns discovered
+
+## Validate Before Deploying
+
+Every new config, manifest, or template should be validated locally before deploying. The target environment is not a test environment — each deploy-crash-fix cycle wastes time and clutters Git history. Batch fixes locally, push once.
+
+Examples:
+- `helm template` for Helm values
+- `kustomize build` for Kustomize apps
+- `docker run <app> validate-configuration` for app configs
+- `docker inspect` for unfamiliar container images
+- Lint/typecheck/test for application code
+- `curl --resolve` for the full request chain after deployment
+
+## Version Management
+
+- Use the latest stable version of dependencies unless pinned for a reason
+- Verify versions from live sources (`helm search repo`, upstream docs, package registries) — don't rely on memory
+- Document the reason in a comment if a version is intentionally pinned below latest
+- Check compatibility matrices before upgrading (e.g., Talos ↔ Kubernetes, framework ↔ runtime)
+
+## Secrets Management
+
+- SOPS + age is the standard encryption tool across all projects
+- The `.sops.yaml` at the repo root defines path-based encryption rules
+- Filenames containing `secret` trigger SOPS encryption via pre-commit hooks
+- Non-secret files must NOT contain `secret` in their name
+- Never pass secrets via command-line arguments (visible in `ps` output) — use `@file` references or environment variables
+- Keep unencrypted secrets in `local_secrets/` (gitignored)
+
+## Scripting Conventions
+
+- All scripts live in `scripts/` and run from the repository root
+- Scripts should be idempotent and safe to re-run
+- Use colour output for pass/fail indicators in verification scripts
+- Verification scripts should check for default/insecure credentials and print remediation instructions on failure
+- Scripts should exit non-zero on failure so `&&` chains work naturally
+
+## Process Principles
+
+These are hard-won lessons from real project work:
+
+- **Validate locally, deploy once.** Don't use the live environment as a test bed. Catch errors with local validation tools before pushing.
+- **Check before you act.** Before writing firewall/network rules, check actual routing (`ip route get`). Before running config management with variables, ensure values are real, not placeholders. Before assuming a container has a shell, `docker inspect` it.
+- **Test the full chain immediately.** After wiring up a new service or endpoint, test end-to-end from the user's perspective right away. Don't assume intermediate steps working means the whole chain works.
+- **Verify scripts should be environment-resilient.** Avoid needing sudo or special access. Test from the accessible side of a connection. Use `curl --resolve` to bypass DNS/proxy layers when testing direct connectivity.
+- **Automate repeated sequences.** If you run the same 3+ commands in sequence more than once, it should become a script.
+- **Reflect after milestones.** Don't just finish — review what happened, what went wrong, what can be improved. Write it down so future sessions benefit.
+
+## Ansible Conventions (where applicable)
+
+- Roles follow standard structure: `tasks/main.yml`, `templates/*.j2`, `handlers/main.yml`
+- Jinja2 templates have `.j2` extension and include a "managed by Ansible" header comment
+- Variables that need customisation go in `inventory.yml`, not scattered across role defaults
+- Always pass `-i inventory.yml` explicitly or run from the directory containing `ansible.cfg`
+- Never use placeholder values with `-e` for vars that template config files
+
+## Helm Chart Conventions (where applicable)
+
+- Always validate values against the chart schema before committing
+- Run `helm show values <repo>/<chart> --version <ver>` to check actual structure
+- Schemas change between versions — field names and nesting can differ from docs or online examples
+- A quick `helm template` test locally catches schema errors before deployment