Add memory files, reflection state, and update docs
Memory files for decisions, bash gotchas, and process lessons. Updated MEMORY.md index and README.md with new scripts. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
7
.reflection-state.json
Normal file
7
.reflection-state.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"version": 1,
|
||||
"last_run": "2026-03-12T22:16:10Z",
|
||||
"processed": {
|
||||
"log/2026-03-13.111440.md": "01d9e37a3340506255d87e0100b33ef1"
|
||||
}
|
||||
}
|
||||
@@ -1,3 +1,12 @@
|
||||
# Memory Index
|
||||
|
||||
<!-- Thin index only — one-line descriptions linking to memory/ topic files -->
|
||||
|
||||
## Gotchas
|
||||
- [Bash Gotchas](memory/gotchas-bash.md) — `set -e` silent failures, grep flag parsing, `local` scope, binary detection with `file` command
|
||||
|
||||
## Decisions
|
||||
- [Decisions](memory/decisions.md) — OpenSpec format, mandatory dryrun, `set -uo pipefail` policy, dirty-state definitions, skynet org hosting
|
||||
|
||||
## Process
|
||||
- [Process Lessons](memory/process-lessons.md) — Skip `set -e` for reporting tools, spec-driven testing effectiveness, `git diff --numstat` for binary detection
|
||||
|
||||
@@ -14,7 +14,11 @@ This project uses **spec-driven development** (OpenSpec) as a testbed for agent-
|
||||
|
||||
| Script | Purpose | Status |
|
||||
|--------|---------|--------|
|
||||
| *(none yet)* | | |
|
||||
| `git-status-report` | Recursive git repo status with diff stats | Done |
|
||||
| `md-to-docx` | Markdown to DOCX conversion | Done |
|
||||
| `claude-profile` | Claude Code profile management | Done |
|
||||
| `unreflected-logs` | Scan projects for unreflected session logs | Done |
|
||||
| `validate-skill` | Validate SKILL.md files against known Claude Code restrictions | Done |
|
||||
|
||||
## Quick Start
|
||||
|
||||
|
||||
21
memory/decisions.md
Normal file
21
memory/decisions.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Decisions
|
||||
|
||||
## OpenSpec format for all script specifications
|
||||
|
||||
Every script starts with a spec in `specs/<name>.spec.md` using OpenSpec format: purpose, usage, behaviour, dryrun behaviour, edge cases, and examples. This project is a testbed for agent-driven development where specs drive implementation and testing.
|
||||
|
||||
## Mandatory `--dryrun` / `-n` on every script
|
||||
|
||||
All scripts must support `--dryrun` which previews actions without side effects. This enables the testing strategy: test scripts exercise the main script's dryrun mode to verify behaviour matches the spec without making real changes.
|
||||
|
||||
## `set -uo pipefail` without `-e` for reporting tools
|
||||
|
||||
Read-only/reporting scripts use `set -uo pipefail` instead of `set -euo pipefail`. The `-e` flag causes silent failures in complex pipeline/subshell chains. Explicit error handling is more predictable for tools that aggregate data from many sources.
|
||||
|
||||
## "No remotes" and "detached HEAD" are not dirty states
|
||||
|
||||
In `git-status-report`, repos with no remotes or detached HEAD are considered clean unless they have local changes. "(no remotes)" is only shown as annotation when the repo already has uncommitted changes.
|
||||
|
||||
## Project hosted under `skynet` org
|
||||
|
||||
`small-scripts` lives in the `skynet` org on Gitea (`gitea.oreillyit.nz`) as an AI-focused project — specifically a testbed for agent-driven, spec-first development workflows.
|
||||
21
memory/gotchas-bash.md
Normal file
21
memory/gotchas-bash.md
Normal file
@@ -0,0 +1,21 @@
|
||||
# Bash Gotchas
|
||||
|
||||
## `set -e` silently kills scripts with complex subshell pipelines
|
||||
|
||||
`set -euo pipefail` can cause silent termination with no output when scripts use loops containing command substitutions, subshell calls, and pipelines. The script appears to produce no output and exits non-zero. For read-only/reporting tools, use `set -uo pipefail` without `-e` and handle errors explicitly.
|
||||
|
||||
## `grep` interprets option-like strings as flags
|
||||
|
||||
`grep -qF "$var"` will fail if `$var` starts with `--` (e.g., `--dryrun`). Always use `--` to terminate option parsing: `grep -qF -- "$var"`.
|
||||
|
||||
## `local` keyword only works inside functions
|
||||
|
||||
Using `local` in the main script body (e.g., inside a `for` loop that isn't wrapped in a function) causes `local: can only be used in a function`. Either wrap the loop in a function or omit `local`.
|
||||
|
||||
## `set -e` kills command substitution capturing non-zero exit codes
|
||||
|
||||
`output=$(cmd)` where `cmd` exits non-zero will trigger `errexit` before `rc=$?` on the next line executes. Use `output=$(cmd) && rc=$? || rc=$?` or `output=$(cmd) || true` to safely capture output from commands expected to fail.
|
||||
|
||||
## `file` command marks shell scripts as "executable"
|
||||
|
||||
The `file` command returns strings like "Bourne-Again shell script, Unicode text, UTF-8 text executable" for shell scripts. Grepping for `executable` to detect binaries will false-positive on text scripts. Instead, grep for `binary|image|archive` and additionally check that the output does NOT contain `text`.
|
||||
29
memory/log/2026-03-13.111440.md
Normal file
29
memory/log/2026-03-13.111440.md
Normal file
@@ -0,0 +1,29 @@
|
||||
# Session Log — 2026-03-13
|
||||
|
||||
## Summary
|
||||
Created the `small-scripts` project — a spec-driven utility script collection with dryrun support and automated testing. Implemented the first script `git-status-report` which recursively scans directories for git repos and reports uncommitted changes with character-level diffs and remote sync status.
|
||||
|
||||
## Decisions
|
||||
- Decision: Use OpenSpec format for all script specifications — Rationale: Testing agent-driven development; specs define purpose, usage, behaviour, dryrun behaviour, edge cases, and examples
|
||||
- Decision: Every script must support `--dryrun` / `-n` — Rationale: Enables testing via dryrun without side effects; tests validate spec compliance through dryrun output
|
||||
- Decision: Removed `set -e` from git-status-report, kept `set -uo pipefail` — Rationale: `set -e` caused silent failures in complex pipeline/subshell chains; for a reporting tool, explicit error handling is safer than errexit
|
||||
- Decision: "No remotes" and "detached HEAD" don't count as dirty — Rationale: These are informational states, not divergences; "(no remotes)" only shown when repo already has local changes
|
||||
- Decision: Project hosted under `skynet` org on Gitea — Rationale: AI-focused project (agent-driven development testbed)
|
||||
|
||||
## Gotchas Discovered
|
||||
- **[bash]** Symptom: `set -euo pipefail` caused silent script termination with no output during complex loops with command substitutions and subshell calls — Fix: Removed `set -e`, kept `set -uo pipefail`. Reporting tools don't need errexit; explicit error handling is more predictable.
|
||||
- **[bash]** Symptom: `grep -qF "$expected"` interpreted `--dryrun` as a grep flag — Fix: Use `grep -qF -- "$expected"` to prevent option-like strings from being parsed as flags
|
||||
- **[bash]** Symptom: `local` keyword used outside a function in main loop body caused `local: can only be used in a function` error — Fix: Remove `local` qualifier for variables in the main script body
|
||||
- **[bash]** Symptom: `set -e` in test script killed execution when capturing output from commands expected to exit non-zero (`output=$(cmd)` where cmd exits 1) — Fix: Use `output=$(cmd) && rc=$? || rc=$?` or `output=$(cmd) || true` pattern
|
||||
- **[bash]** Symptom: `file` command on shell scripts returns "executable" which matched the binary detection grep `binary\|executable\|image\|archive` — Fix: Changed to `grep -qP "binary|image|archive"` with additional `! grep -q "text"` check to avoid false positives on text executables
|
||||
|
||||
## Key Context
|
||||
- Project structure: `specs/` (OpenSpec), `scripts/` (implementations), `tests/` (test scripts using dryrun)
|
||||
- Scripts symlinked to `~/sbin` for PATH availability
|
||||
- Test runner at `tests/run-all.sh` with colour pass/fail output
|
||||
- First script `git-status-report` has 26 test assertions covering: help, no repos, dryrun, clean/dirty repos, untracked/deleted files, no remotes, ahead of remote, mixed repos, nested repo filtering, permission denied
|
||||
|
||||
## Process Notes
|
||||
- The `set -e` debugging consumed significant time — the script worked in isolated tests but failed silently on real repos. Future scripts should start without `set -e` for reporting/read-only tools.
|
||||
- Test development caught real bugs (grep flag parsing, local keyword misuse) — the spec-driven approach with comprehensive tests works well.
|
||||
- The `file` command for binary detection is unreliable for scripts; need a better heuristic (maybe `git diff --numstat` which shows `-` for binary files).
|
||||
25
memory/log/2026-03-17.095436.md
Normal file
25
memory/log/2026-03-17.095436.md
Normal file
@@ -0,0 +1,25 @@
|
||||
# Session Log — 2026-03-17
|
||||
|
||||
## Summary
|
||||
|
||||
Added a "reproduce before fixing" best practice to claude-foundations' debugging topic, committed and pushed all outstanding claude-foundations changes, then built a new `unreflected-logs` script for scanning projects for session logs not yet processed by `/reflect-logs`.
|
||||
|
||||
## Decisions
|
||||
|
||||
- Decision: Add "reproduce before fixing" to `best-practices/debugging.md` rather than TDD — Rationale: TDD file covers regression tests as artifacts; the debugging file covers the *workflow* of how to approach a bug
|
||||
- Decision: Bulk commit all outstanding claude-foundations changes in one commit — Rationale: User requested committing everything, not just the single file change
|
||||
- Decision: `unreflected-logs` script uses python3 for JSON parsing of `.reflection-state.json` — Rationale: Reliable JSON parsing vs fragile bash/jq alternatives; python3 is available on target systems
|
||||
|
||||
## Gotchas Discovered
|
||||
|
||||
- **[bash]** Symptom: `((PASS++))` when PASS=0 evaluates to falsy (arithmetic result 0), causing `set -e` to exit the script silently — Fix: Use `PASS=$((PASS + 1))` instead, which always succeeds as an assignment
|
||||
|
||||
## Key Context
|
||||
|
||||
- `unreflected-logs` script follows the small-scripts spec-first workflow: spec in `specs/`, script in `scripts/`, test in `tests/`, symlinked to `~/sbin`
|
||||
- The script compares `memory/log/*.md` files against `.reflection-state.json` processed keys (format: `log/<filename>`)
|
||||
- Live scan of `~/dev/claude` found 5 unreflected logs across 5 projects as of this session
|
||||
|
||||
## Process Notes
|
||||
|
||||
- The `((var++))` bash gotcha with `set -e` is a classic trap — already documented in `memory/gotchas-bash.md` but still bit us in a new test file. Worth noting that it applies to any arithmetic expression that evaluates to 0.
|
||||
15
memory/log/2026-03-17.105314.md
Normal file
15
memory/log/2026-03-17.105314.md
Normal file
@@ -0,0 +1,15 @@
|
||||
# Session Log — 2026-03-17
|
||||
|
||||
## Summary
|
||||
Diagnosed why the `/housekeeping` skill (created last session) failed to load — an em dash in the YAML frontmatter description silently broke parsing. Fixed the skill and added a non-ASCII frontmatter check to `validate-skill`.
|
||||
|
||||
## Gotchas Discovered
|
||||
- **[skills]** Symptom: Skill installed correctly (symlink, SKILL.md present) but Claude Code reports "Unknown skill" — Fix: Non-ASCII characters (em dashes `—`, smart quotes, etc.) in YAML frontmatter silently prevent skill loading. Replace with ASCII equivalents. Claude commonly generates em dashes, so this is a recurring risk.
|
||||
|
||||
## Key Context
|
||||
- The fix was replacing `—` (UTF-8 `\xe2\x80\x94`) with `-` in the `description: >` field of `custom-claude-skills/skills/housekeeping/SKILL.md`
|
||||
- Non-ASCII in the skill body (below frontmatter) is fine — only the YAML-parsed frontmatter is affected
|
||||
|
||||
## Process Notes
|
||||
- `cat -A` was the key diagnostic — showed `M-bM-^@M-^T` bytes revealing the em dash that looked identical to a regular dash in normal display
|
||||
- Added the check to validate-skill with: script change, spec update, test fixture + 2 assertions — all 45 tests pass
|
||||
22
memory/log/2026-03-17.110414.md
Normal file
22
memory/log/2026-03-17.110414.md
Normal file
@@ -0,0 +1,22 @@
|
||||
# Session Log — 2026-03-17
|
||||
|
||||
## Summary
|
||||
Diagnosed why `/housekeeping` skill wasn't loading in the `~/.claude-octopus` profile, fixed the root cause in `install.sh`, and created a new `check-skills` script to detect missing/stale skill symlinks. Integrated check-skills into the `/housekeeping` skill.
|
||||
|
||||
## Decisions
|
||||
- Decision: Use `CLAUDE_CONFIG_DIR` env var to detect the active Claude profile — Rationale: Claude Code sets this automatically; falls back to `~/.claude` when unset
|
||||
- Decision: Create `check-skills` as a read-only diagnostic script rather than auto-fixing — Rationale: Keeps it safe for `/housekeeping` (information-only), users can run `install.sh` to fix
|
||||
- Decision: Log to `small-scripts` rather than `custom-claude-skills` — Rationale: The new script and tests live in small-scripts; custom-claude-skills changes were smaller edits
|
||||
|
||||
## Gotchas Discovered
|
||||
- **[claude-code]** Symptom: `/housekeeping` skill not found when using `~/.claude-octopus` profile — Fix: Each Claude profile has its own independent `skills/` directory. Skills must be symlinked into every profile, not just `~/.claude`. The `CLAUDE_CONFIG_DIR` env var identifies the active profile.
|
||||
- **[claude-code]** Symptom: `install.sh` hardcoded `$HOME/.claude/skills` so new skills only appeared in the default profile — Fix: Changed to `${CLAUDE_CONFIG_DIR:-$HOME/.claude}/skills`
|
||||
|
||||
## Key Context
|
||||
- `CLAUDE_CONFIG_DIR` env var is set by Claude Code to the active profile directory (e.g., `/home/paul/.claude-octopus`)
|
||||
- Multiple Claude profiles maintain completely independent `skills/` directories — no cross-profile sharing
|
||||
- The `check-skills` script classifies skills as: LINKED (correct), MISSING (not in profile), STALE (wrong target), ORPHAN (not in source repo)
|
||||
|
||||
## Process Notes
|
||||
- Spec-first workflow for `check-skills` kept implementation focused — spec, script, test, symlink, integrate
|
||||
- All 17 test cases passed on first run
|
||||
13
memory/process-lessons.md
Normal file
13
memory/process-lessons.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# Process Lessons
|
||||
|
||||
## Start reporting/read-only scripts without `set -e`
|
||||
|
||||
Debugging `set -e` failures in scripts that aggregate data from multiple sources (git repos, file stats, etc.) consumed significant time — the script worked in isolated tests but failed silently on real data. Begin without `-e` and add it only for scripts that perform destructive actions where fail-fast is critical.
|
||||
|
||||
## Spec-driven testing catches real bugs early
|
||||
|
||||
The spec → implement → test workflow caught real bugs during development (grep flag parsing, `local` keyword misuse, binary detection false positives). Writing tests that exercise dryrun against spec expectations is an effective pattern for this project.
|
||||
|
||||
## Use `git diff --numstat` for binary detection instead of `file`
|
||||
|
||||
The `file` command is unreliable for distinguishing binary from text files (marks shell scripts as "executable"). `git diff --numstat` shows `-` for binary files and is more reliable since git already has its own binary detection heuristics.
|
||||
Reference in New Issue
Block a user