diff --git a/CONTEXT.md b/CONTEXT.md index dc8b3de..191cd44 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -5,3 +5,4 @@ Current work focus and active tasks. Used by `context-load` to orient agents at ## Active - [Context loading improvements](context/context-loading.md) — Testing and refining the `context-load` and `start-claude` scripts +- [Transcript tracking — first-run verification](context/transcript-tracking.md) — System built and committed; verify Sonnet subagent writes to correct absolute path on first `/log` run with real backups diff --git a/README.md b/README.md index 415f194..cb82ccb 100644 --- a/README.md +++ b/README.md @@ -21,10 +21,12 @@ claude-foundations/ pre-compact-backup.sh # PreCompact: backup transcript before compaction require-plan-file.sh # PreToolUse/ExitPlanMode: enforce *-PLAN.md exists before leaving plan mode scripts/ - install-hooks.sh # Symlink hooks into ~/.claude/hooks/ - setup-formatters.sh # Set up formatters for a project - statusline.sh # Status line renderer (symlinked from ~/.claude/status/) - set-topic.sh # Set per-session topic for the status line + install-hooks.sh # Symlink hooks into ~/.claude/hooks/ and skill-helper scripts into ~/.claude/scripts/ + setup-formatters.sh # Set up formatters for a project + statusline.sh # Status line renderer (symlinked from ~/.claude/status/) + set-topic.sh # Set per-session topic for the status line + extract-transcripts.py # Transcript backup tracking: register, list, extract, mark-processed + list-transcripts-here.sh # Wrapper: extract-transcripts --list $(pwd) (skill bang-command safe) context/ # Active work focus detail files memory/ # Session logs and reflections settings.yaml # Knowledge pipeline configuration @@ -44,7 +46,7 @@ cd ~/dev/claude/projects/claude-foundations scripts/install-hooks.sh ``` -This symlinks all hooks into `~/.claude/hooks/` and prints the `settings.json` config to add. The PostToolUse hook is what triggers auto-formatting on every Edit/Write. +This symlinks all hooks into `~/.claude/hooks/`, installs skill-helper scripts into `~/.claude/scripts/`, and prints the `settings.json` config to add. The PostToolUse hook is what triggers auto-formatting on every Edit/Write. ### 2. Opt a project into auto-formatting @@ -140,7 +142,9 @@ rm .pre-lint | Script | Purpose | |--------|---------| -| `scripts/install-hooks.sh` | Symlink all hooks to `~/.claude/hooks/` and print settings.json config | +| `scripts/install-hooks.sh` | Symlink hooks to `~/.claude/hooks/` and skill-helper scripts to `~/.claude/scripts/`; print settings.json config | | `scripts/setup-formatters.sh` | Create formatter symlinks in a target project | | `scripts/statusline.sh` | Status line renderer — shows topic, model, context %. Symlinked from `~/.claude/status/` | | `scripts/set-topic.sh` | Set the session topic: `set-topic.sh "topic text"` | +| `scripts/extract-transcripts.py` | Transcript backup tracking — register backups, list unprocessed, extract conversation text, mark processed | +| `scripts/list-transcripts-here.sh` | Wrapper calling `extract-transcripts.py --list $(pwd)` — used in `/log` skill pre-gathered context | diff --git a/context/transcript-tracking.md b/context/transcript-tracking.md new file mode 100644 index 0000000..eb64e5a --- /dev/null +++ b/context/transcript-tracking.md @@ -0,0 +1,28 @@ +# Transcript Tracking System — Pending Verification + +## Status +Built and committed (2026-04-13, commit `6fd0dac`). Pending first real-run verification. + +## What was built +- `pre-compact-backup.sh` — fixed to derive transcript path from `session_id+cwd`; now registers each backup in `~/.claude/transcript-backups/tracking.json` +- `extract-transcripts.py` — manages tracking.json; modes: `--register`, `--list`, `--extract`, `--mark-all-processed` +- `list-transcripts-here.sh` — wrapper for `--list $(pwd)` (needed because SKILL.md bang commands reject `$()` substitution) +- `/log` skill (in custom-claude-skills) — spawns Sonnet subagent to read JSONL backups and write `HHMMSS-transcripts.md` companion log + +## What to verify on first real run + +When `/log` is run in a session where context compression fired at least once: + +1. Check `~/.claude/transcript-backups/tracking.json` has an entry for the session with `processed: false` +2. Run `/log` — it should show unprocessed transcripts in the pre-gathered metadata +3. The Sonnet subagent should write `memory/log/YYYY-MM-DD.HHMMSS-transcripts.md` in the **project directory** (absolute path) +4. After `/log` completes, `tracking.json` entry should show `processed: true` with a `log_file` path + +**Known risk:** The subagent prompt template asks it to write to `[PROJECT_ABS_PATH]/memory/log/...` — if it uses a relative path from the wrong cwd, the file lands in the wrong place. If that happens, update the subagent prompt in `/log` SKILL.md to include an explicit `cd [CWD]` instruction. + +## Files +- Hook: `~/dev/claude/projects/claude-foundations/hooks/pre-compact-backup.sh` +- Scripts: `~/dev/claude/projects/claude-foundations/scripts/extract-transcripts.py` and `list-transcripts-here.sh` +- Symlinked to: `~/.claude/scripts/` +- Skill: `~/dev/claude/projects/custom-claude-skills/skills/log/SKILL.md` +- Tracking state: `~/.claude/transcript-backups/tracking.json` diff --git a/memory/log/2026-03-23.123545.md b/memory/log/2026-03-23.123545.md deleted file mode 100644 index 6c83a0d..0000000 --- a/memory/log/2026-03-23.123545.md +++ /dev/null @@ -1,18 +0,0 @@ -# Session Log — 2026-03-23 - -## Summary -Set up a persistent status line for Claude Code that displays session topic, model name, and context window percentage. Scripts were created, moved into claude-foundations, and documentation updated across CLAUDE.md, MEMORY.md, and README.md. Status line config was added to all three profile settings files (~/.claude, ~/.claude-octopus, ~/.claude-oreillyit). - -## Decisions -- Decision: Use per-session topic files at `~/.claude/status//claude-topic.txt` — Rationale: Session ID from the status line JSON uniquely identifies each session, allowing multiple concurrent sessions to have independent topics -- Decision: Status line script writes session ID to `/tmp/claude-session-id-` — Rationale: Claude has no direct access to its own session ID; the status line script bridges this by writing it to a predictable path keyed by working directory -- Decision: Canonical scripts live in `claude-foundations/scripts/`, symlinked from `~/.claude/status/` — Rationale: Follows the same pattern as hooks (canonical in repo, symlinked to ~/.claude/) for version control and consistency -- Decision: Status line config added to all three profile settings files — Rationale: Each profile (default, octopus, oreillyit) has its own settings.json; statusLine must be present in each - -## Gotchas Discovered -- **[claude-code]** Symptom: `set-topic.sh` fails on the first message of a session because the status line hasn't run yet to write the session ID file — Fix: This is expected; set the topic from the second message onward. The bootstrap caveat is documented in CLAUDE.md and the memory file. - -## Key Context -- Claude Code status line receives session JSON on stdin with fields: `session_id`, `model`, `context_window`, `cwd`, `cost`, `rate_limits`, etc. -- Status line script runs after each assistant message, debounced at 300ms -- Three profile settings files need to stay in sync: `~/.claude/settings.json`, `~/.claude-octopus/settings.json`, `~/.claude-oreillyit/settings.json` diff --git a/memory/log/2026-03-25.110612.md b/memory/log/2026-03-25.110612.md deleted file mode 100644 index 61bbb99..0000000 --- a/memory/log/2026-03-25.110612.md +++ /dev/null @@ -1,19 +0,0 @@ -# Session Log — 2026-03-25 - -## Summary -Fixed the `set-topic.sh` bootstrap race condition where calling it before `statusline.sh` had run caused an error. Implemented a pending-topic queue mechanism and added the permission rule across all three Claude profiles. - -## Decisions -- Decision: Queue pending topics to `/tmp/claude-pending-topic-` instead of erroring — Rationale: `set-topic.sh` is called early in sessions (first message) before `statusline.sh` has written the session ID file. Erroring was noisy and unhelpful since the topic could just be deferred one response. -- Decision: Have `statusline.sh` pick up and apply pending topics via `mv` — Rationale: Keeps the logic simple — statusline already runs after every response, so it's the natural place to consume the pending file. Using `mv` is atomic and avoids double-application. -- Decision: Add `Bash(~/.claude/status/set-topic.sh:*)` to all three profile settings files — Rationale: The script is called every session start and is safe to auto-approve. - -## Gotchas Discovered -- **[statusline]** Symptom: `set-topic.sh` always errored on first message with "No session ID found" because `/tmp/claude-session-id-` doesn't exist until `statusline.sh` runs after the first response — Fix: Write topic to a pending file; `statusline.sh` picks it up on its next run. - -## Key Context -- Three profile settings files need to stay in sync: `~/.claude/settings.json`, `~/.claude-octopus/settings.json`, `~/.claude-oreillyit/settings.json` -- The `statusline.sh` hash uses `echo -n "$CWD" | md5sum` (note the `-n` flag — important for hash consistency with `set-topic.sh`) - -## Process Notes -- Clean fix — two small script changes plus doc updates. Tested both the happy path (session ID exists) and the pending path (no session ID yet). diff --git a/memory/log/2026-03-29.111336.md b/memory/log/2026-03-29.111336.md deleted file mode 100644 index 74b1ba7..0000000 --- a/memory/log/2026-03-29.111336.md +++ /dev/null @@ -1,27 +0,0 @@ -# Session Log — 2026-03-29 - -## Summary -Researched API design best practices and LLM code security, created two new best-practice files in the dedicated skynet/best-practices repo, audited agent-runtimes against the new practices (found critical auth gap + 5 unplanned medium-severity gaps), added F37-F41 to agent-runtimes FUTURE.md, and created two new skills (/review-plan, /review-spec) that review plans and specs against these best practices. - -## Decisions -- Decision: Create two separate best-practice files (api-design.md and llm-code-security.md) rather than one combined security file — Rationale: Keeps topics self-contained and independently referenceable; API design covers more than just security (versioning, pagination, idempotency) -- Decision: Best practices repo (skynet/best-practices) is the target, not claude-foundations — Rationale: Best practices were migrated to a dedicated repo; BESTPRACTICES.md in claude-foundations now redirects there -- Decision: Mark F27 (deep health checks) as DONE in agent-runtimes FUTURE.md — Rationale: /health/ready already exists on both CP and dispatcher with dependency checks -- Decision: Create /review-plan and /review-spec as read-only skills — Rationale: Review output should inform the human, not auto-edit plans/specs. Read-only also reduces permission prompts. -- Decision: Both review skills load all 5 security/design best practice files upfront — Rationale: Better to over-load context than miss a relevant check; the files are not excessively large - -## Gotchas Discovered -- **[best-practices]** Symptom: best-practices directory didn't exist under claude-foundations (got `No such file or directory`) — Fix: BESTPRACTICES.md redirects to skynet/best-practices repo; local clone already existed at ~/dev/claude/projects/best-practices/ but wasn't in the directory tree shown by context-load -- **[agent-runtimes]** Symptom: FUTURE.md had 3 instances of "Depends on: M9 (complete — K8s deployment exists)" making unique string replacement fail — Fix: Included more surrounding context to uniquely identify the F36 instance - -## Key Context -- The best-practices repo is at ~/dev/claude/projects/best-practices/ with remote skynet/best-practices on Gitea -- ~/dev/claude/BESTPRACTICES.md is a symlink to the best-practices repo's index -- agent-runtimes spec/authentication.md has a comprehensive OIDC + mTLS + bootstrap token design (AU-1..AU-30) but zero implementation — the entire API is currently unauthenticated except for poll and heartbeat -- Research agents (background subagents) were effective for parallel web research — both completed in ~5 minutes and produced well-structured output with citations -- api-design.md references RFC 9700 (OAuth 2.0 Security BCP, January 2025) which deprecates implicit flow and password grant - -## Process Notes -- Parallel background agents worked well for independent research tasks — launched both simultaneously and reviewed results as they completed -- The audit agent (foreground) was thorough — checked actual implementation files and line numbers rather than just reading specs -- Four repos touched in one session (best-practices, agent-runtimes, custom-claude-skills, claude-foundations) — each committed and pushed independently with focused commit messages diff --git a/memory/log/2026-04-13.140716.md b/memory/log/2026-04-13.140716.md new file mode 100644 index 0000000..2d37e1e --- /dev/null +++ b/memory/log/2026-04-13.140716.md @@ -0,0 +1,26 @@ +# Session Log — 2026-04-13 + +## Summary + +Built a complete pre-compaction transcript backup tracking system: fixed the broken `pre-compact-backup.sh` hook, created `extract-transcripts.py` and `list-transcripts-here.sh`, updated the `/log` skill to dispatch a Sonnet subagent for transcript analysis, moved all scripts to their canonical homes in claude-foundations, and ensured both active profiles are in sync. + +## Decisions + +- Decision: Use Sonnet (not Haiku) for transcript analysis subagent in `/log` — Rationale: gotcha detection requires judgment about backtracking and failed attempts; Haiku tends to see only the final outcome and miss the failure mode that led to it +- Decision: `/log` pre-gathers only transcript metadata (compact JSON) not content — Rationale: full transcript content would overflow context; content reading delegated to the Sonnet subagent in its own fresh context window +- Decision: Tracking file at `~/.claude/transcript-backups/tracking.json` (machine-readable JSON) — Rationale: both hook (bash) and skill (python) need to update it; JSON is simpler to parse than markdown tables from bash +- Decision: Scripts live in `claude-foundations/scripts/`, symlinked to `~/.claude/scripts/` — Rationale: consistent with hooks pattern; `~/.claude/scripts/` is a fixed path accessible from all profiles since `~/.claude/` always resolves to the default profile dir +- Decision: `install-hooks.sh` extended with a curated `SKILL_HELPERS` array to manage skill-helper script symlinks — Rationale: not all claude-foundations scripts belong in `~/.claude/scripts/`, only ones referenced by skills + +## Gotchas Discovered + +- **[hooks]** Symptom: `pre-compact-backup.sh` was silently skipping every run — Fix: Claude Code's PreCompact hook no longer provides `transcript_path` in the JSON input (changed from older versions); derive path from `session_id` + `cwd` using `~/.claude/projects/$(cwd | tr '/' '-')/.jsonl` +- **[python]** Symptom: cwd extraction loop printed multiple lines instead of stopping at first match — Fix: `sys.exit(0)` inside `except: pass` is caught as `SystemExit`; use a `break` or don't use bare `except` when early exit is needed inside exception handlers +- **[skills]** Symptom: SKILL.md validator error on bang command using `$(pwd)` — Fix: `$()` substitution in SKILL.md bang commands is rejected by the Claude Code permission checker; create a wrapper shell script that runs `$(pwd)` internally and call the wrapper from the bang command instead + +## Key Context + +- Old transcript backups with `-auto` suffix (before tracking system) are not in `tracking.json` and will never appear in `--list`; they'll be pruned by the 30-day cleanup in the hook +- Profile parity fix: `~/.claude-octopus/skills/switch-mode` symlink was missing; added +- `~/.claude/scripts/` is the canonical location for skill-helper scripts; `~/.claude/` always resolves to the default profile dir, making it accessible from all profiles regardless of `CLAUDE_CONFIG_DIR` +- The Sonnet subagent in `/log` writes to the project's absolute `memory/log/` path — worth verifying on first real run that it uses the absolute path correctly, not a relative one from the wrong cwd