claude-foundations/CLAUDE.md

CLAUDE.md — General Project Guidelines

Session Start

1. Immediately (without waiting for user input) use the directory tree already provided by context-load (the TREE section in your context) to identify project directories under ~/dev/claude/ (excluding secrets/). Present a numbered menu like:

   What are we working on today?

   1. cluster-bootstrap — Kubernetes homelab cluster
   2. custom-claude-skills — Reusable Claude Code skills ... N-1. No project right now — just chat N. New project! — start something new

   Do not run shell commands to list directories — the tree is already in your context. Include a brief description if the project has a CLAUDE.md or README.md you can glean one from.

2. Based on the user's choice:

   • Existing project: cd into the directory, read all .md files, and read ~/dev/claude/secrets/ (read-only reference — review every file to refresh context). Then read BESTPRACTICES.md (loaded automatically by context-load) and load any topic files relevant to the selected project's technology stack. Ask clarifying questions if anything is unclear or incomplete, and note context in MEMORY.md.
   • No project right now: Do nothing further — just respond normally.
   • New project!: Follow the "New Projects" section below. Also read ~/dev/claude/secrets/ as above. Read BESTPRACTICES.md (loaded automatically by context-load) and load topic files relevant to the new project's technology stack.

3. Set the status line topic after the user selects a project or describes their task. See the "Status Line" section below.

Secrets (~/dev/claude/secrets/)

CRITICAL — treat this folder with extreme paranoia:

• Files in ~/dev/claude/secrets/ are read-only. Never edit them.
• Never copy, echo, write, or reproduce secret values into any other file — not MEMORY.md, not CLAUDE.md, not commit messages, not scripts, not tool output, nowhere.
• Never include secret values in git commits, diffs, or changelogs of any project.
• Never pass secret values as command-line arguments (visible in ps output). Use @file references, environment variables sourced at runtime, or stdin.
• When a task requires a secret, read it at execution time from the secrets folder and use it ephemerally. Do not cache or persist the value.
• It is acceptable to reference the existence of a secret file (e.g., "credentials are in ~/dev/claude/secrets/gitea/ai_enablement") but never its contents.

New Projects

If this is a new project:

1. Create a new directory under ~/dev/claude/<project-name>/
2. Ask the user which Gitea user/org the repo should be created under (e.g., homelab, a personal user, etc.) before setting up the remote
3. Create the initial standard files:
   • CLAUDE.md — Project-specific architecture, conventions, repo structure, and working instructions for Claude
   • MEMORY.md — Persistent learnings, gotchas, reflections, and process improvements
   • FUTURE.md — Ideas and improvements not on the active roadmap (Problem/Idea/Open questions/Depends on format)
   • README.md — Human-readable overview, quick start, milestones, and scripts reference
4. Parse the other CLAUDE.md files from sibling project folders in ~/dev/claude/, and based on the type of project being considered, bring over related practices, guidelines, and learnings
5. For coding projects with multiple milestones: Read best-practices/spec-driven-development.md and best-practices/test-driven-development.md. Create SPEC.md and spec/ directory. Write specs before writing code. See the "Spec-Driven Development" section below.

Source Control

• All projects are hosted on Gitea (gitea.oreillyit.nz) as the primary remote — prefer this hostname over gitea.homelab.internal (same instance, but the public name enables external access)
• Migrate existing remotes from gitea.homelab.internal to gitea.oreillyit.nz when convenient
• AI-focused projects go under the skynet org; infrastructure projects under homelab
• SSH workflows preferred. SSH config uses host aliases per Gitea user:
  • gitea.oreillyit.nz-homelab → authenticates as cluster-administrator (key: ~/.ssh/gitea-cluster-admin)
  • gitea.oreillyit.nz-ai-enablement → authenticates as ai_enablement (key: ~/.ssh/gitea.ai-enablement)
• Git remote URLs use the alias: git@gitea.oreillyit.nz-<user>:<org>/<repo>.git
  • Example: git@gitea.oreillyit.nz-ai-enablement:skynet/custom-claude-skills.git
• Optionally push-mirror to GitHub for public visibility
• Use meaningful commit messages; prefer small, focused commits over large batches
• Enable pre-commit hooks where appropriate (secret detection, linting, formatting)
• Never commit secrets in plaintext — use SOPS + age or equivalent encryption

Documentation Standards

Every project maintains up to six standard markdown files. All root-level .md files are automatically loaded by context-load at session start, so they should be thin indexes pointing to detail files in subdirectories — not large documents themselves.

CLAUDE.md (per-project)

The primary reference for Claude sessions. Should contain:

• Project overview and architecture
• Repository structure (keep updated as the project evolves)
• Key design decisions with rationale
• Conventions and coding standards
• Environment details (IPs, URLs, credentials references)
• Common operations / how-to recipes

MEMORY.md (Tiered Memory System)

Long-running projects accumulate significant context. To keep MEMORY.md useful rather than bloated, use a tiered memory structure:

MEMORY.md is a thin index only — one-line descriptions with links to topic files in memory/. No content lives in MEMORY.md itself. Think of it as a card catalog.

memory/ contains the actual content, split by topic:

• memory/project-status.md — Current milestone, what's next, blockers
• memory/network.md — IPs, VIPs, subnets, topology
• memory/gotchas-<topic>.md — Gotchas grouped by technology (e.g., gotchas-cilium.md, gotchas-authelia.md)
• memory/process-lessons.md — How-to-work-with-this-repo lessons for Claude
• memory/m<N>-reflection.md — One file per milestone reflection (these are time-bound, so per-file is natural)
• memory/decisions.md — Architecture and design decisions made during planning

Principles:

• Split by topic, not by time. A Cilium gotcha belongs in gotchas-cilium.md whether discovered in M5 or M8.
• Milestone reflections are the exception — inherently time-bound, one file per milestone.
• Index descriptions matter. They're used to decide what to read. "Cilium L2/LB gotchas and externalTrafficPolicy quirks" beats "cluster stuff".
• Prune aggressively. If a gotcha was fixed (e.g., chart version upgraded past the bug), delete it. Stale memory is worse than no memory.
• Each memory file should be self-contained and greppable. Include enough context that the file makes sense on its own.
• Deduplicate with CLAUDE.md. Conventions and patterns that are stable should live in CLAUDE.md. Memory files are for learnings, gotchas, and reflections that accumulate over time. If something in memory has graduated to a stable convention, move it to CLAUDE.md and remove it from memory.

When reading memory at session start: Read MEMORY.md (the index), then selectively read topic files relevant to the current task. Don't read all memory files unless doing a broad review.

When writing memory after a milestone: Create the reflection file, update any affected topic files (new gotchas, updated status), and update the index.

CONTEXT.md (Active Work Focus)

Tells Claude (and independent agents) what the project is currently working on. Follows the same index pattern as MEMORY.md.

CONTEXT.md is a thin index — one-line descriptions with links to detail files in context/. It answers "what should I focus on right now?"

context/ contains the detail files, one per active work stream:

• context/<topic>.md — Background, status, what to test, future direction for that work stream

Principles:

• Keep it current. Remove entries when work is complete. CONTEXT.md should reflect what's actively in progress, not a history of past work.
• Link, don't inline. The index stays small so context-load doesn't bloat the system prompt. Detail files can be as long as needed.
• Orient agents. CONTEXT.md is the primary mechanism for pointing independent agents (including those in Docker containers) at the right work. Each agent gets the full context load, with its starting prompt referencing the relevant CONTEXT.md entry.
• Complement, don't duplicate. CLAUDE.md has stable conventions. MEMORY.md has accumulated learnings. CONTEXT.md has the current focus and active tasks.

BESTPRACTICES.md (Best Practices Index)

Index of generalised best practices extracted from real project work via the /distill-best-practices skill. Only exists in claude-foundations — other projects inherit it via context-load walking up the directory hierarchy.

BESTPRACTICES.md is a thin index — one-line descriptions with links to topic files in best-practices/.

best-practices/ contains self-contained topic files (e.g., kubernetes.md, helm.md, secrets-management.md). Read only the files relevant to the current project's technology stack.

The /distill-best-practices skill maintains both the topic files and the index.

FUTURE.md

Backlog of improvement ideas, each with:

• Problem: What's painful or manual today
• Idea: What the improvement looks like
• Open questions: Unknowns to research before starting
• Depends on: Other items or milestones that should come first

README.md

Human-readable project documentation:

• Architecture summary
• Quick start / setup instructions
• Milestone table with status
• Scripts section listing every script with purpose and usage

Script & Skill Documentation

Every script in claude-foundations and every skill in custom-claude-skills must have a corresponding memory file in claude-foundations/memory/:

• Scripts: memory/script-<name>.md — Purpose, usage, how it works, gotchas
• Skills: memory/skill-<name>.md — Purpose, usage, how it works, gotchas, which projects use it

Each memory file should be self-contained and referenced from claude-foundations/MEMORY.md (the index). This ensures that future sessions can quickly understand what tooling exists and how it works, without having to read every script and SKILL.md from scratch.

When creating a new script or skill: Create the memory file and update the MEMORY.md index as part of the same commit.

When creating or editing a skill: Run validate-skill <path/to/SKILL.md> before committing. The validator catches known restriction violations that have repeatedly broken skills — $VAR in paths, ${VAR} syntax, $() substitution, uncovered binaries in allowed-tools, and more. A skill must pass with zero errors before it is committed. Warnings should be reviewed but are acceptable.

Knowledge Distillation Pipeline

Three skills form a continuous learning pipeline across projects:

1. /log — Run at end of session. Captures decisions, gotchas, open questions to memory/log/YYYY-MM-DD.<HHMMSS>.md in the current project. Also prunes old reflected logs.
2. /reflect-logs — Run periodically. Processes unprocessed session logs into topic memory files (memory/gotchas-*.md, memory/process-lessons.md, etc.). Flags stale entries. Tracks state in .reflection-state.json.
3. /distill-best-practices — Run from any project. Reads changed memory files across all tracked projects and proposes updates to the claude-foundations repo's best-practices/ folder. Tracks state in best-practices/.distill-state.json.

State Files

• .reflection-state.json — Per-project, tracks which logs have been reflected on (md5 hashes of log file content)
• best-practices/.distill-state.json — In claude-foundations, tracks git SHAs per project at time of last distillation
• settings.yaml — In claude-foundations, configures log retention (default 7 days), max logs per reflection run, and tracked project list

Log Format

Session logs use structured markdown with parseable section headers: Summary, Decisions, Gotchas Discovered (tagged with [topic] for routing), Open Questions, Key Context, Process Notes. Empty sections are omitted.

Pruning

• Reflected logs older than log.retention_days (default: 7) are automatically deleted by /log
• Unreflected logs older than log.warn_unreflected_days (default: 14) trigger a warning instead of deletion
• /reflect-logs flags stale memory entries (version-specific bugs that have been fixed, manual processes that have been automated)

Plan Mode

When working in plan mode (permission_mode: plan):

Before calling ExitPlanMode, always write the complete plan to a file in the project root:

• Filename: [MILESTONE]-[PURPOSE]-PLAN.md — e.g. M2-auth-PLAN.md, M3-monitoring-PLAN.md
  • MILESTONE: the milestone identifier (e.g. M2) or a short label if not milestone-scoped (e.g. initial)
  • PURPOSE: a short kebab-case description of what the plan covers
• Contents: the full plan as developed in the planning conversation — steps, decisions, rationale, open questions
• Location: project root (same directory as CLAUDE.md)

This file becomes the implementation reference for the session that follows plan mode.

Keep the plan updated during implementation. After completing each phase or significant step, update the PLAN.md file:

• Mark the phase/step status as Complete
• Add key commits, references, or artifacts produced
• Document deviations from the original plan (what changed and why)
• Note verification results

This ensures the plan stays accurate as a living document — useful for resuming across sessions, reflecting on the milestone, and understanding what actually happened vs. what was planned.

Spec-Driven Development

Any coding project with multiple milestones MUST have specs before code. This is a hard requirement, not a suggestion. The workflow is: Plan → Spec → Test → Code.

At project start (or when starting a new milestone), always read:

• best-practices/spec-driven-development.md — spec structure, requirement numbering, scenarios, maintenance
• best-practices/test-driven-development.md — edge case discovery, property-based testing, AI agent testing patterns

Required Artifacts

Every multi-milestone coding project must have:

1. SPEC.md — Index file at the project root. Lists all spec files with a "when to read" column. Same thin-index pattern as MEMORY.md.
2. spec/ directory — One spec file per subsystem. Each spec follows the required structure: Overview, Responsibilities, Dependencies, Data Model, Requirements (numbered), Scenarios (given/when/then).
3. Numbered requirements — Each spec uses a prefix (e.g., IG-1 for ingestion, DB-1 for database). Requirements must be independently testable and unambiguous.
4. Test files that reference spec IDs — Every test function name includes its requirement ID: test_ig3_dedup_by_message_ts.

The Workflow

1. Plan — Architecture decisions, milestone breakdown, technology choices (PLAN.md)
2. Spec — Detailed contracts, data models, interfaces, requirements, scenarios (spec/)
3. Test — Write tests from the spec before code exists. They should all fail.
4. Code — Implement until tests pass. Minimum code to satisfy the spec.
5. Update — If implementation reveals spec issues, update spec → test → code in that order.

When to Write Specs

• Before M1 implementation begins — write specs for all subsystems in M1's scope
• Before each subsequent milestone — write specs for new subsystems, update existing specs for changes
• Spec changes and test changes ship in the same commit
• Code changes that affect interfaces require spec changes in the same commit

What Does NOT Need a Spec

• Infrastructure-only projects (Helm values, Kustomize manifests, Ansible playbooks) — these are declarative, not behavioral
• Single-script utilities — a well-commented script with a test is sufficient
• Documentation-only changes

Milestones

Break projects into numbered milestones (M1, M2, ...). Every milestone completion MUST include:

1. Verification script (scripts/verify-m<N>.sh) — automated checks confirming all milestone outcomes. Scripts should be idempotent, non-destructive, and return non-zero on failure. Use colour output (green/red) for pass/fail.
2. Milestone reflection in memory/m<N>-reflection.md — review the entire conversation and capture:
   • Process improvements (what slowed us down, wrong assumptions, backtracking)
   • Key knowledge for reproduction (gotchas, version quirks, debugging detours)
   • Scripts and automation opportunities (repeated command sequences → scripts)
   • Future improvement ideas (add to FUTURE.md)
   • Update affected topic files in memory/ (new gotchas, updated status) and the MEMORY.md index
3. Updated README.md — scripts section, milestone table, any new setup steps
4. Updated CLAUDE.md — repo structure, conventions, new patterns discovered

Version Management

• Use the latest stable version of dependencies unless pinned for a reason
• Verify versions from live sources (helm search repo, upstream docs, package registries) — don't rely on memory
• Document the reason in a comment if a version is intentionally pinned below latest
• Check compatibility matrices before upgrading (e.g., Talos ↔ Kubernetes, framework ↔ runtime)

Secrets Management

• SOPS + age is the standard encryption tool across all projects
• The .sops.yaml at the repo root defines path-based encryption rules
• Filenames containing secret trigger SOPS encryption via pre-commit hooks
• Non-secret files must NOT contain secret in their name
• Keep unencrypted secrets in local_secrets/ (gitignored)

Scripting Conventions

• All scripts live in scripts/ and run from the repository root
• Scripts should be idempotent and safe to re-run
• Use colour output for pass/fail indicators in verification scripts
• Verification scripts should check for default/insecure credentials and print remediation instructions on failure
• Scripts should exit non-zero on failure so && chains work naturally
• Never hardcode secrets, tokens, or access keys in scripts. Accept them via environment variables, stdin, or @file references. If a script needs a secret at runtime, read it from ~/dev/claude/secrets/ or accept it as a parameter — never embed it.

Process Principles

These are hard-won lessons from real project work:

• Validate locally, deploy once. Don't use the live environment as a test bed. Catch errors with local validation tools before pushing (helm template, kustomize build, lint/typecheck, docker inspect, etc.). Batch fixes locally, push once.
• Check before you act. Before writing firewall/network rules, check actual routing (ip route get). Before running config management with variables, ensure values are real, not placeholders. Before assuming a container has a shell, docker inspect it.
• Test the full chain immediately. After wiring up a new service or endpoint, test end-to-end from the user's perspective right away. Don't assume intermediate steps working means the whole chain works.
• Verify scripts should be environment-resilient. Avoid needing sudo or special access. Test from the accessible side of a connection. Use curl --resolve to bypass DNS/proxy layers when testing direct connectivity.
• Automate repeated sequences. If you run the same 3+ commands in sequence more than once, it should become a script.
• Reflect after milestones. Don't just finish — review what happened, what went wrong, what can be improved. Write it down so future sessions benefit.

Status Line

A persistent bar at the bottom of Claude Code shows the current topic, model, and context usage: [Model Name] topic | N% context.

Setting the Topic

After the user selects a project or describes their task (i.e., after the first response where the status line has had a chance to run), set the topic:

~/.claude/status/set-topic.sh "$(pwd)" "project-name: brief task description"

Examples:

• ~/.claude/status/set-topic.sh "$(pwd)" "brainiac-app: M2 web frontend"
• ~/.claude/status/set-topic.sh "$(pwd)" "cluster-bootstrap: Cilium upgrade"
• ~/.claude/status/set-topic.sh "$(pwd)" "General chat"

When to Update

• Session start: Set the topic once the user picks a project or task
• Focus change: Update if the user shifts to a different project or task mid-session
• Keep it short: Aim for project: task format, under ~40 characters

How It Works

The status line script (scripts/statusline.sh) runs after each assistant message. It writes the session ID to /tmp/claude-session-id-<md5 of cwd>, which set-topic.sh reads to find the correct per-session topic file at ~/.claude/status/<session-id>/claude-topic.txt.

Early calls are safe: If set-topic.sh is called before the status line has run (i.e., on the first message), the topic is queued to a pending file and automatically applied when the status line first runs after the next response.