Add session log and reflect: API design research session

Session log for 2026-03-29 covering API/LLM security research, agent-runtimes
audit, and review skill creation. Reflected into:
- decisions.md: best-practices topic split, review skills read-only design
- process-lessons.md: background agents for research, Edit context
  disambiguation, best-practices repo location

Pruned 2 old reflected logs (2026-03-17, 2026-03-18).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Paul O'Reilly
2026-03-29 11:22:37 +13:00
parent 884b416d38
commit 52b0f2b5c6
6 changed files with 56 additions and 58 deletions

View File

@@ -1,6 +1,6 @@
{
"version": 1,
"last_run": "2026-03-24T10:36:46Z",
"last_run": "2026-03-28T22:15:17Z",
"processed": {
"log/2026-03-12.233744.md": "ca482f03914a3b4dec89c2dbf6e0f2e5",
"log/2026-03-13.100758.md": "65c5b65fdd5984c036f1de53f8c82f2a",
@@ -8,6 +8,8 @@
"log/2026-03-15.225345.md": "ae9366aed0b4c87ecbbe535e00cb2cb2",
"log/2026-03-17.103204.md": "56c04b01459f80134e680150c338e9eb",
"log/2026-03-18.002319.md": "68ab608a650421dcc663fd4ae7131fc9",
"log/2026-03-23.123545.md": "2610a8adb53e4b831e3e4b0c1d7cad44"
"log/2026-03-23.123545.md": "2610a8adb53e4b831e3e4b0c1d7cad44",
"log/2026-03-25.110612.md": "adb467a37c6da84adb5ab5bbf20c773a",
"log/2026-03-29.111336.md": "87ad0d241f002605863f66d7fb36c863"
}
}

View File

@@ -68,6 +68,18 @@ Topic files at `~/.claude/status/<session-id>/claude-topic.txt`. Session ID uniq
Claude has no direct access to its own session ID. The status line script writes it to `/tmp/claude-session-id-<md5 of cwd>`, which `set-topic.sh` reads to find the correct per-session topic directory.
## Status line: Pending-topic queue for bootstrap race condition
`set-topic.sh` is called on the first message, before `statusline.sh` has written the session ID file. Instead of erroring, it writes to `/tmp/claude-pending-topic-<hash>`. `statusline.sh` picks this up on its next run via `mv` (atomic, no double-application). Simple and self-healing — statusline already runs after every response.
## Status line: Config must exist in every profile settings file
Each profile (`~/.claude`, `~/.claude-octopus`, `~/.claude-oreillyit`) has its own `settings.json` with no inheritance. The `statusLine` entry must be added to each independently.
Each profile (`~/.claude`, `~/.claude-octopus`, `~/.claude-oreillyit`) has its own `settings.json` with no inheritance. The `statusLine` entry and `Bash(~/.claude/status/set-topic.sh:*)` permission must be added to each independently.
## Best practices: One file per topic, not combined mega-documents
api-design.md and llm-code-security.md are separate files despite both being "security-adjacent". API design covers versioning, pagination, idempotency — not just security. Self-contained topic files are independently referenceable and can be loaded selectively by skills and agents.
## Review skills: Read-only output, not auto-edit
/review-plan and /review-spec are read-only (allowed-tools: Read, Glob, Grep). Review output informs the human rather than auto-editing plans/specs. This avoids unintended changes and reduces permission prompts. Both skills load all 5 design/security best practice files upfront — better to over-load context than miss a relevant check.

View File

@@ -1,32 +0,0 @@
# Session Log — 2026-03-17
## Summary
Built a `validate-skill` script to catch known SKILL.md restriction violations before deployment. Researched all historical skill breakages across session logs and git commits, then implemented a comprehensive validator with 43 tests. Also added "Reproduce Before Fixing" best practice, built `unreflected-logs` script, and fixed 7 real errors the validator found in existing skills.
## Decisions
- Decision: Add "Reproduce Before Fixing" to `best-practices/debugging.md` rather than TDD — Rationale: TDD covers regression tests as artifacts; debugging covers the workflow of how to approach a bug
- Decision: `validate-skill` lives in small-scripts, not claude-foundations — Rationale: It's a standalone utility script, follows the spec-first pattern, symlinked to `~/sbin`
- Decision: `$VAR` in path context is an ERROR, `~/` is a WARN, `/home/` is an ERROR suggesting `~/` — Rationale: Permission checker rejects `$HOME/path` shell expansion; `~/` is more portable than `/home/paul` but still may be rejected by some sandbox modes
- Decision: Fix `$HOME/...` bang-commands by replacing with Read tool instructions — Rationale: Read tool doesn't go through the permission checker's shell expansion check; more reliable than any path syntax in bang-commands
## Gotchas Discovered
- **[skills]** Symptom: `/log` skill fails with "Shell expansion syntax in paths requires manual approval" on `cat $HOME/dev/claude/...` — Fix: Replace bang-commands using `$VAR` in paths with plain-text instructions to use the Read tool at runtime
- **[skills]** Symptom: `$HOME` (without braces) is also rejected by permission checker, not just `${HOME}` — Fix: Added `$VAR` in path context check to `validate-skill` (regex: `\$[A-Za-z_][A-Za-z0-9_]*/`)
- **[bash]** Symptom: `grep -n '^\`'` with escaped backtick doesn't match literal backtick in files — Fix: Use `grep -n '^!` + backtick without escaping; in single-quoted strings the backslash is literal, not an escape
- **[bash]** Symptom: `((PASS++))` when PASS=0 evaluates to falsy, killed by `set -e` — Fix: Use `PASS=$((PASS + 1))` which always succeeds as an assignment
## Key Context
- `validate-skill` checks: frontmatter (delimiters, name, description), bang-commands (`${VAR}`, `$VAR` in paths, `$()`, `~/`, `../`, `/home/`), allowed-tools coverage (uncovered binaries, overly broad patterns)
- Full skill failure timeline from git history: 7 commits in custom-claude-skills, 4 are fixes. `/distill-best-practices` was fixed 3 times.
- Historical failure patterns cataloged: `$()` subshell, `${VAR}` syntax, `$VAR` in paths, absolute paths blocked by sandbox, relative paths fragile, broad `Bash(git *)`, hardcoded filenames that changed, edits needing new session
- CLAUDE.md now requires running `validate-skill` before committing any skill changes
- 3 skills fixed this session: `/log`, `/reflect-logs`, `/distill-best-practices` — all had `$HOME/...` in bang-commands replaced with Read tool instructions
## Process Notes
- Researching the full history (logs + commits) before building the validator was valuable — it revealed that `$HOME` (not just `${HOME}`) is rejected, which we wouldn't have caught from docs alone
- The validator immediately proved its value by finding 7 real errors in production skills on first run against the real codebase

View File

@@ -1,23 +0,0 @@
# Session Log — 2026-03-18
## Summary
Created a shared infrastructure catalog repo (`homelab/infrastructure-docs`) documenting all homelab systems with dependency tracking. 20 system files covering physical hosts, cluster core, networking, identity, platform services, and applications.
## Decisions
- Decision: Infrastructure knowledge goes in a dedicated repo, not MEMORY.md — Rationale: Infrastructure maps are authoritative and maintained (add/remove systems), not accumulated learnings. Different lifecycle than gotchas/reflections.
- Decision: Located at `~/dev/claude/docs/infrastructure/homelab/` with parent structure supporting future locations (e.g., customer sites) — Rationale: User expects to have infrastructure at multiple logical locations in future.
- Decision: Repo in `homelab` org on Gitea, not `skynet` — Rationale: Infrastructure documentation, not AI-focused.
- Decision: Each system file uses consistent structure (what it does, depends on, depended on by, managed by) — Rationale: Enables tracing dependencies in either direction.
## Gotchas Discovered
- **[gitea]** Symptom: Push to new repo failed with "User permission denied for writing" — Fix: The SSH alias `gitea.oreillyit.nz-homelab` authenticates as `cluster-administrator`, but repo was created by `ai_admin` via API. Added `cluster-administrator` as admin collaborator via API before push succeeded.
## Key Context
- `ai_admin` Gitea token is in `cluster-bootstrap/local_secrets/gitea_ai_admin` (not in `~/dev/claude/secrets/`)
- Created repo via Gitea API: `POST /api/v1/orgs/homelab/repos`
- Added collaborator via API: `PUT /api/v1/repos/homelab/infrastructure-docs/collaborators/cluster-administrator`
- Memory reference added to both claude-foundations MEMORY.md and Claude profile memory
## Process Notes
- Good pattern: discussing the conceptual approach (MEMORY vs SPEC vs dedicated catalog) before jumping into implementation
- The Gitea permission issue could be avoided by always adding `cluster-administrator` as collaborator when creating repos via `ai_admin` API token for the `homelab` org

View File

@@ -0,0 +1,27 @@
# Session Log — 2026-03-29
## Summary
Researched API design best practices and LLM code security, created two new best-practice files in the dedicated skynet/best-practices repo, audited agent-runtimes against the new practices (found critical auth gap + 5 unplanned medium-severity gaps), added F37-F41 to agent-runtimes FUTURE.md, and created two new skills (/review-plan, /review-spec) that review plans and specs against these best practices.
## Decisions
- Decision: Create two separate best-practice files (api-design.md and llm-code-security.md) rather than one combined security file — Rationale: Keeps topics self-contained and independently referenceable; API design covers more than just security (versioning, pagination, idempotency)
- Decision: Best practices repo (skynet/best-practices) is the target, not claude-foundations — Rationale: Best practices were migrated to a dedicated repo; BESTPRACTICES.md in claude-foundations now redirects there
- Decision: Mark F27 (deep health checks) as DONE in agent-runtimes FUTURE.md — Rationale: /health/ready already exists on both CP and dispatcher with dependency checks
- Decision: Create /review-plan and /review-spec as read-only skills — Rationale: Review output should inform the human, not auto-edit plans/specs. Read-only also reduces permission prompts.
- Decision: Both review skills load all 5 security/design best practice files upfront — Rationale: Better to over-load context than miss a relevant check; the files are not excessively large
## Gotchas Discovered
- **[best-practices]** Symptom: best-practices directory didn't exist under claude-foundations (got `No such file or directory`) — Fix: BESTPRACTICES.md redirects to skynet/best-practices repo; local clone already existed at ~/dev/claude/projects/best-practices/ but wasn't in the directory tree shown by context-load
- **[agent-runtimes]** Symptom: FUTURE.md had 3 instances of "Depends on: M9 (complete — K8s deployment exists)" making unique string replacement fail — Fix: Included more surrounding context to uniquely identify the F36 instance
## Key Context
- The best-practices repo is at ~/dev/claude/projects/best-practices/ with remote skynet/best-practices on Gitea
- ~/dev/claude/BESTPRACTICES.md is a symlink to the best-practices repo's index
- agent-runtimes spec/authentication.md has a comprehensive OIDC + mTLS + bootstrap token design (AU-1..AU-30) but zero implementation — the entire API is currently unauthenticated except for poll and heartbeat
- Research agents (background subagents) were effective for parallel web research — both completed in ~5 minutes and produced well-structured output with citations
- api-design.md references RFC 9700 (OAuth 2.0 Security BCP, January 2025) which deprecates implicit flow and password grant
## Process Notes
- Parallel background agents worked well for independent research tasks — launched both simultaneously and reviewed results as they completed
- The audit agent (foreground) was thorough — checked actual implementation files and line numbers rather than just reading specs
- Four repos touched in one session (best-practices, agent-runtimes, custom-claude-skills, claude-foundations) — each committed and pushed independently with focused commit messages

View File

@@ -39,3 +39,15 @@ Before creating a new system or document, discuss where it belongs conceptually
## Add SSH-authenticating user as collaborator when creating repos via API
When creating Gitea repos via API token (e.g., `ai_admin`), the SSH alias may authenticate as a different user (e.g., `cluster-administrator`). Always add the SSH user as admin collaborator via API before pushing.
## Use background agents for parallel independent research
When researching multiple topics that don't depend on each other, launch background subagents simultaneously. Both API design and LLM security research completed in ~5 minutes each, producing structured output with citations. Review results as they complete. Don't duplicate research work the subagent is doing.
## Include more surrounding context when Edit tool matches are ambiguous
The Edit tool fails if `old_string` matches multiple locations. When editing files with repeated patterns (like FUTURE.md items that all end with "Depends on: M9"), include unique surrounding lines (a heading, the preceding paragraph) to disambiguate. Don't use `replace_all: true` as a workaround — it changes all instances.
## Best-practices repo is separate from claude-foundations
Best practices live in skynet/best-practices (cloned at ~/dev/claude/projects/best-practices/). ~/dev/claude/BESTPRACTICES.md is a symlink to its index. context-load doesn't show the best-practices directory in the tree — remember it exists when working on best practices topics.