diff --git a/CLAUDE.md b/CLAUDE.md index 69a0251..476940c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -144,6 +144,8 @@ Each memory file should be self-contained and referenced from `claude-foundation **When creating a new script or skill:** Create the memory file and update the MEMORY.md index as part of the same commit. +**When creating or editing a skill:** Run `validate-skill ` before committing. The validator catches known restriction violations that have repeatedly broken skills — `$VAR` in paths, `${VAR}` syntax, `$()` substitution, uncovered binaries in `allowed-tools`, and more. A skill must pass with zero errors before it is committed. Warnings should be reviewed but are acceptable. + ## Knowledge Distillation Pipeline Three skills form a continuous learning pipeline across projects: diff --git a/memory/log/2026-03-17.103204.md b/memory/log/2026-03-17.103204.md new file mode 100644 index 0000000..08d246d --- /dev/null +++ b/memory/log/2026-03-17.103204.md @@ -0,0 +1,32 @@ +# Session Log — 2026-03-17 + +## Summary + +Built a `validate-skill` script to catch known SKILL.md restriction violations before deployment. Researched all historical skill breakages across session logs and git commits, then implemented a comprehensive validator with 43 tests. Also added "Reproduce Before Fixing" best practice, built `unreflected-logs` script, and fixed 7 real errors the validator found in existing skills. + +## Decisions + +- Decision: Add "Reproduce Before Fixing" to `best-practices/debugging.md` rather than TDD — Rationale: TDD covers regression tests as artifacts; debugging covers the workflow of how to approach a bug +- Decision: `validate-skill` lives in small-scripts, not claude-foundations — Rationale: It's a standalone utility script, follows the spec-first pattern, symlinked to `~/sbin` +- Decision: `$VAR` in path context is an ERROR, `~/` is a WARN, `/home/` is an ERROR suggesting `~/` — Rationale: Permission checker rejects `$HOME/path` shell expansion; `~/` is more portable than `/home/paul` but still may be rejected by some sandbox modes +- Decision: Fix `$HOME/...` bang-commands by replacing with Read tool instructions — Rationale: Read tool doesn't go through the permission checker's shell expansion check; more reliable than any path syntax in bang-commands + +## Gotchas Discovered + +- **[skills]** Symptom: `/log` skill fails with "Shell expansion syntax in paths requires manual approval" on `cat $HOME/dev/claude/...` — Fix: Replace bang-commands using `$VAR` in paths with plain-text instructions to use the Read tool at runtime +- **[skills]** Symptom: `$HOME` (without braces) is also rejected by permission checker, not just `${HOME}` — Fix: Added `$VAR` in path context check to `validate-skill` (regex: `\$[A-Za-z_][A-Za-z0-9_]*/`) +- **[bash]** Symptom: `grep -n '^\`'` with escaped backtick doesn't match literal backtick in files — Fix: Use `grep -n '^!` + backtick without escaping; in single-quoted strings the backslash is literal, not an escape +- **[bash]** Symptom: `((PASS++))` when PASS=0 evaluates to falsy, killed by `set -e` — Fix: Use `PASS=$((PASS + 1))` which always succeeds as an assignment + +## Key Context + +- `validate-skill` checks: frontmatter (delimiters, name, description), bang-commands (`${VAR}`, `$VAR` in paths, `$()`, `~/`, `../`, `/home/`), allowed-tools coverage (uncovered binaries, overly broad patterns) +- Full skill failure timeline from git history: 7 commits in custom-claude-skills, 4 are fixes. `/distill-best-practices` was fixed 3 times. +- Historical failure patterns cataloged: `$()` subshell, `${VAR}` syntax, `$VAR` in paths, absolute paths blocked by sandbox, relative paths fragile, broad `Bash(git *)`, hardcoded filenames that changed, edits needing new session +- CLAUDE.md now requires running `validate-skill` before committing any skill changes +- 3 skills fixed this session: `/log`, `/reflect-logs`, `/distill-best-practices` — all had `$HOME/...` in bang-commands replaced with Read tool instructions + +## Process Notes + +- Researching the full history (logs + commits) before building the validator was valuable — it revealed that `$HOME` (not just `${HOME}`) is rejected, which we wouldn't have caught from docs alone +- The validator immediately proved its value by finding 7 real errors in production skills on first run against the real codebase