Add memory files, reflection state, and update docs

Memory files for decisions, bash gotchas, and process lessons.
Updated MEMORY.md index and README.md with new scripts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Paul O'Reilly
2026-03-17 11:14:44 +13:00
parent 87f7bfb7cd
commit 22db5514f8
10 changed files with 167 additions and 1 deletions

21
memory/decisions.md Normal file
View File

@@ -0,0 +1,21 @@
# Decisions
## OpenSpec format for all script specifications
Every script starts with a spec in `specs/<name>.spec.md` using OpenSpec format: purpose, usage, behaviour, dryrun behaviour, edge cases, and examples. This project is a testbed for agent-driven development where specs drive implementation and testing.
## Mandatory `--dryrun` / `-n` on every script
All scripts must support `--dryrun` which previews actions without side effects. This enables the testing strategy: test scripts exercise the main script's dryrun mode to verify behaviour matches the spec without making real changes.
## `set -uo pipefail` without `-e` for reporting tools
Read-only/reporting scripts use `set -uo pipefail` instead of `set -euo pipefail`. The `-e` flag causes silent failures in complex pipeline/subshell chains. Explicit error handling is more predictable for tools that aggregate data from many sources.
## "No remotes" and "detached HEAD" are not dirty states
In `git-status-report`, repos with no remotes or detached HEAD are considered clean unless they have local changes. "(no remotes)" is only shown as annotation when the repo already has uncommitted changes.
## Project hosted under `skynet` org
`small-scripts` lives in the `skynet` org on Gitea (`gitea.oreillyit.nz`) as an AI-focused project — specifically a testbed for agent-driven, spec-first development workflows.

21
memory/gotchas-bash.md Normal file
View File

@@ -0,0 +1,21 @@
# Bash Gotchas
## `set -e` silently kills scripts with complex subshell pipelines
`set -euo pipefail` can cause silent termination with no output when scripts use loops containing command substitutions, subshell calls, and pipelines. The script appears to produce no output and exits non-zero. For read-only/reporting tools, use `set -uo pipefail` without `-e` and handle errors explicitly.
## `grep` interprets option-like strings as flags
`grep -qF "$var"` will fail if `$var` starts with `--` (e.g., `--dryrun`). Always use `--` to terminate option parsing: `grep -qF -- "$var"`.
## `local` keyword only works inside functions
Using `local` in the main script body (e.g., inside a `for` loop that isn't wrapped in a function) causes `local: can only be used in a function`. Either wrap the loop in a function or omit `local`.
## `set -e` kills command substitution capturing non-zero exit codes
`output=$(cmd)` where `cmd` exits non-zero will trigger `errexit` before `rc=$?` on the next line executes. Use `output=$(cmd) && rc=$? || rc=$?` or `output=$(cmd) || true` to safely capture output from commands expected to fail.
## `file` command marks shell scripts as "executable"
The `file` command returns strings like "Bourne-Again shell script, Unicode text, UTF-8 text executable" for shell scripts. Grepping for `executable` to detect binaries will false-positive on text scripts. Instead, grep for `binary|image|archive` and additionally check that the output does NOT contain `text`.

View File

@@ -0,0 +1,29 @@
# Session Log — 2026-03-13
## Summary
Created the `small-scripts` project — a spec-driven utility script collection with dryrun support and automated testing. Implemented the first script `git-status-report` which recursively scans directories for git repos and reports uncommitted changes with character-level diffs and remote sync status.
## Decisions
- Decision: Use OpenSpec format for all script specifications — Rationale: Testing agent-driven development; specs define purpose, usage, behaviour, dryrun behaviour, edge cases, and examples
- Decision: Every script must support `--dryrun` / `-n` — Rationale: Enables testing via dryrun without side effects; tests validate spec compliance through dryrun output
- Decision: Removed `set -e` from git-status-report, kept `set -uo pipefail` — Rationale: `set -e` caused silent failures in complex pipeline/subshell chains; for a reporting tool, explicit error handling is safer than errexit
- Decision: "No remotes" and "detached HEAD" don't count as dirty — Rationale: These are informational states, not divergences; "(no remotes)" only shown when repo already has local changes
- Decision: Project hosted under `skynet` org on Gitea — Rationale: AI-focused project (agent-driven development testbed)
## Gotchas Discovered
- **[bash]** Symptom: `set -euo pipefail` caused silent script termination with no output during complex loops with command substitutions and subshell calls — Fix: Removed `set -e`, kept `set -uo pipefail`. Reporting tools don't need errexit; explicit error handling is more predictable.
- **[bash]** Symptom: `grep -qF "$expected"` interpreted `--dryrun` as a grep flag — Fix: Use `grep -qF -- "$expected"` to prevent option-like strings from being parsed as flags
- **[bash]** Symptom: `local` keyword used outside a function in main loop body caused `local: can only be used in a function` error — Fix: Remove `local` qualifier for variables in the main script body
- **[bash]** Symptom: `set -e` in test script killed execution when capturing output from commands expected to exit non-zero (`output=$(cmd)` where cmd exits 1) — Fix: Use `output=$(cmd) && rc=$? || rc=$?` or `output=$(cmd) || true` pattern
- **[bash]** Symptom: `file` command on shell scripts returns "executable" which matched the binary detection grep `binary\|executable\|image\|archive` — Fix: Changed to `grep -qP "binary|image|archive"` with additional `! grep -q "text"` check to avoid false positives on text executables
## Key Context
- Project structure: `specs/` (OpenSpec), `scripts/` (implementations), `tests/` (test scripts using dryrun)
- Scripts symlinked to `~/sbin` for PATH availability
- Test runner at `tests/run-all.sh` with colour pass/fail output
- First script `git-status-report` has 26 test assertions covering: help, no repos, dryrun, clean/dirty repos, untracked/deleted files, no remotes, ahead of remote, mixed repos, nested repo filtering, permission denied
## Process Notes
- The `set -e` debugging consumed significant time — the script worked in isolated tests but failed silently on real repos. Future scripts should start without `set -e` for reporting/read-only tools.
- Test development caught real bugs (grep flag parsing, local keyword misuse) — the spec-driven approach with comprehensive tests works well.
- The `file` command for binary detection is unreliable for scripts; need a better heuristic (maybe `git diff --numstat` which shows `-` for binary files).

View File

@@ -0,0 +1,25 @@
# Session Log — 2026-03-17
## Summary
Added a "reproduce before fixing" best practice to claude-foundations' debugging topic, committed and pushed all outstanding claude-foundations changes, then built a new `unreflected-logs` script for scanning projects for session logs not yet processed by `/reflect-logs`.
## Decisions
- Decision: Add "reproduce before fixing" to `best-practices/debugging.md` rather than TDD — Rationale: TDD file covers regression tests as artifacts; the debugging file covers the *workflow* of how to approach a bug
- Decision: Bulk commit all outstanding claude-foundations changes in one commit — Rationale: User requested committing everything, not just the single file change
- Decision: `unreflected-logs` script uses python3 for JSON parsing of `.reflection-state.json` — Rationale: Reliable JSON parsing vs fragile bash/jq alternatives; python3 is available on target systems
## Gotchas Discovered
- **[bash]** Symptom: `((PASS++))` when PASS=0 evaluates to falsy (arithmetic result 0), causing `set -e` to exit the script silently — Fix: Use `PASS=$((PASS + 1))` instead, which always succeeds as an assignment
## Key Context
- `unreflected-logs` script follows the small-scripts spec-first workflow: spec in `specs/`, script in `scripts/`, test in `tests/`, symlinked to `~/sbin`
- The script compares `memory/log/*.md` files against `.reflection-state.json` processed keys (format: `log/<filename>`)
- Live scan of `~/dev/claude` found 5 unreflected logs across 5 projects as of this session
## Process Notes
- The `((var++))` bash gotcha with `set -e` is a classic trap — already documented in `memory/gotchas-bash.md` but still bit us in a new test file. Worth noting that it applies to any arithmetic expression that evaluates to 0.

View File

@@ -0,0 +1,15 @@
# Session Log — 2026-03-17
## Summary
Diagnosed why the `/housekeeping` skill (created last session) failed to load — an em dash in the YAML frontmatter description silently broke parsing. Fixed the skill and added a non-ASCII frontmatter check to `validate-skill`.
## Gotchas Discovered
- **[skills]** Symptom: Skill installed correctly (symlink, SKILL.md present) but Claude Code reports "Unknown skill" — Fix: Non-ASCII characters (em dashes `—`, smart quotes, etc.) in YAML frontmatter silently prevent skill loading. Replace with ASCII equivalents. Claude commonly generates em dashes, so this is a recurring risk.
## Key Context
- The fix was replacing `—` (UTF-8 `\xe2\x80\x94`) with `-` in the `description: >` field of `custom-claude-skills/skills/housekeeping/SKILL.md`
- Non-ASCII in the skill body (below frontmatter) is fine — only the YAML-parsed frontmatter is affected
## Process Notes
- `cat -A` was the key diagnostic — showed `M-bM-^@M-^T` bytes revealing the em dash that looked identical to a regular dash in normal display
- Added the check to validate-skill with: script change, spec update, test fixture + 2 assertions — all 45 tests pass

View File

@@ -0,0 +1,22 @@
# Session Log — 2026-03-17
## Summary
Diagnosed why `/housekeeping` skill wasn't loading in the `~/.claude-octopus` profile, fixed the root cause in `install.sh`, and created a new `check-skills` script to detect missing/stale skill symlinks. Integrated check-skills into the `/housekeeping` skill.
## Decisions
- Decision: Use `CLAUDE_CONFIG_DIR` env var to detect the active Claude profile — Rationale: Claude Code sets this automatically; falls back to `~/.claude` when unset
- Decision: Create `check-skills` as a read-only diagnostic script rather than auto-fixing — Rationale: Keeps it safe for `/housekeeping` (information-only), users can run `install.sh` to fix
- Decision: Log to `small-scripts` rather than `custom-claude-skills` — Rationale: The new script and tests live in small-scripts; custom-claude-skills changes were smaller edits
## Gotchas Discovered
- **[claude-code]** Symptom: `/housekeeping` skill not found when using `~/.claude-octopus` profile — Fix: Each Claude profile has its own independent `skills/` directory. Skills must be symlinked into every profile, not just `~/.claude`. The `CLAUDE_CONFIG_DIR` env var identifies the active profile.
- **[claude-code]** Symptom: `install.sh` hardcoded `$HOME/.claude/skills` so new skills only appeared in the default profile — Fix: Changed to `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/skills`
## Key Context
- `CLAUDE_CONFIG_DIR` env var is set by Claude Code to the active profile directory (e.g., `/home/paul/.claude-octopus`)
- Multiple Claude profiles maintain completely independent `skills/` directories — no cross-profile sharing
- The `check-skills` script classifies skills as: LINKED (correct), MISSING (not in profile), STALE (wrong target), ORPHAN (not in source repo)
## Process Notes
- Spec-first workflow for `check-skills` kept implementation focused — spec, script, test, symlink, integrate
- All 17 test cases passed on first run

13
memory/process-lessons.md Normal file
View File

@@ -0,0 +1,13 @@
# Process Lessons
## Start reporting/read-only scripts without `set -e`
Debugging `set -e` failures in scripts that aggregate data from multiple sources (git repos, file stats, etc.) consumed significant time — the script worked in isolated tests but failed silently on real data. Begin without `-e` and add it only for scripts that perform destructive actions where fail-fast is critical.
## Spec-driven testing catches real bugs early
The spec → implement → test workflow caught real bugs during development (grep flag parsing, `local` keyword misuse, binary detection false positives). Writing tests that exercise dryrun against spec expectations is an effective pattern for this project.
## Use `git diff --numstat` for binary detection instead of `file`
The `file` command is unreliable for distinguishing binary from text files (marks shell scripts as "executable"). `git diff --numstat` shows `-` for binary files and is more reliable since git already has its own binary detection heuristics.