skynet/best-practices
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

distill: flush prior-session best-practice additions

Browse Source 
Additive/refining entries left uncommitted in the working tree from an earlier
distill session (found during the 2026-07 sweep commit): agent-repos 5-step
plan pattern; ai-parallel-agents cheap-model scope; spec-driven spec-inversion
and caller/callee cross-reference rules.
This commit is contained in:
Paul O'Reilly
2026-07-02 15:58:52 +12:00
parent 7e348f5ee3
commit c8691368e4
3 changed files with 93 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
agent-repos.md
View File

@@ -436,10 +436,24 @@ Invest in pre-dispatch validation scripts that catch the top-N infrastructure fa
## Cross-Model Reviews Catch ~38% More Issues Than a Single Model
Running the same security or spec review with two different models (e.g., Opus + MiniMax) and comparing outputs catches ~38% more issues than either alone — in measured reviews, only 62% of findings overlap. Models converge on obvious issues but diverge on edge cases and design concerns. Worth the extra cost for security-critical specs and architecture reviews; overkill for routine code review.
Running the same security or spec review with two different models (e.g., Opus + MiniMax) and comparing outputs catches ~38% more issues than either alone — in measured reviews, only 62% of findings overlap. Reconfirmed at ~40% additional findings across multiple milestones in a second independent project — the effect is consistent, not noise. Models converge on obvious issues but diverge on edge cases and design concerns. Worth the extra cost for security-critical specs and architecture reviews; overkill for routine code review.
**Pattern:** dispatch parallel review tasks to different models with identical prompts, union the findings, deduplicate against a shared issue key (file + line + category). Present the merged list to the human reviewer along with per-model attribution so reviewers can see where models agreed vs. diverged.
**Run in parallel, not sequentially.** Sequential review wastes wall time AND biases the second reviewer if the first's output lands in the shared context. Concurrent dispatch with identical prompts is the correct shape; merge findings after both complete.
## Plan → Review → Decisions → V2 → Consistency-Check (5-Step Pattern)
For non-trivial plans (multi-subsystem features, security-critical changes, cross-team boundaries), use a named 5-step workflow rather than ad-hoc iteration:
1. **Plan** — produce the initial PLAN.md from the architecture conversation.
2. **Review** — dispatch cross-model reviews in parallel (Opus + a different family). Both reviewers receive the same plan and the same review prompt.
3. **Decisions** — write a DECISIONS.md capturing the outcome of each review finding: accepted, rejected (with rationale), or deferred. The decisions file is a permanent record of why the plan looks the way it does.
4. **V2** — produce PLAN-v2.md applying all accepted decisions. Keep V1 alongside V2; don't overwrite.
5. **Consistency-check** — automated grep/script pass to confirm: revoked phrases don't reappear, paired schema shapes match across docs, all referenced requirement IDs resolve. Skipping this lets spec inversions ship — the revoked text remains alongside the new policy and agents follow the wrong one half the time.
Phase 5 is the one most teams skip and the one that prevents the largest class of multi-day fix-up cycles. Treat it as a mandatory CI step on any PR that touches a plan or spec.
## Agent Worktree Branches Contain Files, Not Commits — Copy, Don't Merge
**Symptom:** Orchestrator merges an agent's branch and sees "Already up to date" because the agent wrote files to its worktree but never ran `git add` / `git commit`. Downstream tasks that depend on the upstream artifact then fail or silently use stale data.