855 lines
36 KiB
YAML
855 lines
36 KiB
YAML
name: spec-planning
|
|
version: 4
|
|
description: >
|
|
Multi-model spec planning with interview, cross-model reviews, human gates,
|
|
and post-synthesis security review with auto-fix.
|
|
17-node DAG: 2 interviews, 1 question consolidation, 2 plans, 6 cross-model
|
|
reviews (3 disciplines x 2 models — each reviews the OTHER's plan),
|
|
1 escalation with human decision points, 1 final synthesis by best-scoring model,
|
|
2 post-synthesis security reviews (parallel, cross-model), 1 auto-fix for
|
|
non-escalated findings, 1 final fix implementation after human review of escalations.
|
|
|
|
params:
|
|
required:
|
|
task_description:
|
|
type: string
|
|
description: "What to spec — the feature or subsystem to design"
|
|
project_id:
|
|
type: string
|
|
description: "Target project identifier"
|
|
optional:
|
|
model_a:
|
|
type: string
|
|
default: null
|
|
description: "Override model A endpoint (auto-selected from registry if omitted)"
|
|
model_b:
|
|
type: string
|
|
default: null
|
|
description: "Override model B endpoint (auto-selected from registry if omitted)"
|
|
best_model:
|
|
type: string
|
|
default: null
|
|
description: "Model override for final synthesis — set after reading escalation output (the best-scoring model)"
|
|
repo:
|
|
type: string
|
|
default: null
|
|
description: "Git repo URL to clone for project context (optional)"
|
|
existing_specs:
|
|
type: string
|
|
default: ""
|
|
description: "Paste existing spec content for context (e.g., current WF-1..WF-22)"
|
|
scope_notes:
|
|
type: string
|
|
default: ""
|
|
description: "Any constraints, prior decisions, or scope boundaries from the human"
|
|
|
|
nodes:
|
|
# ── Phase 0: Interview ───────────────────────────────────────────────
|
|
#
|
|
# Both models independently review the task and generate clarifying
|
|
# questions. Each sees the task description, scope notes, existing
|
|
# specs, and best practices (via the planning harness). If a repo is
|
|
# provided, they can also read the project's CLAUDE.md and specs.
|
|
|
|
interview_a:
|
|
name: "Interview Questions ({{ model_a or 'auto' }})"
|
|
prompt: &interview_prompt |
|
|
You are a senior software architect preparing to write a detailed specification.
|
|
Before you start, you need to ask clarifying questions to avoid costly assumptions.
|
|
|
|
## Task to Spec
|
|
{{ task_description }}
|
|
|
|
{% if scope_notes %}
|
|
## Scope Notes from Human
|
|
{{ scope_notes }}
|
|
{% endif %}
|
|
|
|
{% if existing_specs %}
|
|
## Existing Specifications (for context)
|
|
{{ existing_specs }}
|
|
{% endif %}
|
|
|
|
## Best Practices
|
|
Read these files from `/opt/harness/context/planning/best-practices/`:
|
|
1. `spec-driven-development.md` — spec structure, requirement numbering, scenarios
|
|
2. `test-driven-development.md` — deriving tests from specs, edge case discovery
|
|
3. `security-architecture.md` — server boundary rule, defense in depth, auth patterns
|
|
4. `llm-code-security.md` — injection flaws, input validation, OWASP for AI code
|
|
|
|
{% if repo %}
|
|
## Project Context
|
|
The project repo has been cloned. Read `CLAUDE.md` and any existing `spec/` files
|
|
to understand the current architecture, conventions, and design decisions.
|
|
{% endif %}
|
|
|
|
## Your Task
|
|
Generate questions that will improve the quality of the spec you'll write later.
|
|
Think about what you'd ask a product owner, tech lead, or domain expert before
|
|
committing to a design.
|
|
|
|
Focus on:
|
|
- **Ambiguous requirements** — what does X mean in this context?
|
|
- **Missing scope boundaries** — is Y in or out of scope?
|
|
- **Business logic decisions** — should Z behave as A or B?
|
|
- **Technical constraints** — performance targets, compatibility, resource limits?
|
|
- **Dependencies and integration** — how does this interact with existing subsystems?
|
|
- **Priority and phasing** — which parts are essential vs nice-to-have?
|
|
- **Security implications** — who are the threat actors, what's the trust boundary?
|
|
- **Testing strategy** — what's the expected test infrastructure?
|
|
|
|
## Output Format
|
|
Write a numbered list of questions to `/workspace/project/output.md`:
|
|
|
|
```
|
|
## Clarifying Questions
|
|
|
|
1. **[Question]**
|
|
Why it matters: [what design decision hinges on the answer]
|
|
Suggested default: [what you'd assume if no answer is given]
|
|
|
|
2. **[Question]**
|
|
...
|
|
```
|
|
|
|
Aim for 8-20 questions. Prioritize questions whose answers would change the most
|
|
design decisions. Don't ask about things that are clearly stated in the task
|
|
description or scope notes.
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 7, spec_adherence: 8 }
|
|
model_override: "{{ model_a }}"
|
|
|
|
interview_b:
|
|
name: "Interview Questions ({{ model_b or 'auto' }})"
|
|
prompt: *interview_prompt
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 7, spec_adherence: 8 }
|
|
model_override: "{{ model_b }}"
|
|
|
|
# ── Phase 0.5: Question Consolidation ────────────────────────────────
|
|
#
|
|
# A single model deduplicates and organizes questions from both
|
|
# interviewers. Questions asked by both models are flagged as
|
|
# high-signal. Output goes through a human gate — the human answers
|
|
# the questions, and their answers are appended to the artifact.
|
|
|
|
consolidate_questions:
|
|
name: "Consolidate Interview Questions"
|
|
depends_on: [interview_a, interview_b]
|
|
prompt: |
|
|
You are consolidating clarifying questions from two independent reviewers
|
|
into a single, organized list for a human to answer.
|
|
|
|
## Questions from Model A
|
|
<<ARTIFACT:interview_a:output>>
|
|
|
|
## Questions from Model B
|
|
<<ARTIFACT:interview_b:output>>
|
|
|
|
## Instructions
|
|
1. **Deduplicate** — merge questions that ask the same thing in different words
|
|
2. **Flag consensus** — when both models asked the same question, note "(Asked by both)"
|
|
as this is a strong signal the question matters
|
|
3. **Group by theme** — organize into sections (Scope, Architecture, Security,
|
|
Testing, Integration, Priority, etc.)
|
|
4. **Preserve context** — keep each question's "why it matters" and "suggested default"
|
|
5. **Order by impact** — within each group, questions whose answers change the most
|
|
design decisions come first
|
|
|
|
## Output Format
|
|
Write to `/workspace/project/output.md`:
|
|
|
|
```
|
|
## Questions for Human Review
|
|
|
|
### High Impact (answers change multiple design decisions)
|
|
1. **[Question]** (Asked by: A / B / both)
|
|
Why it matters: [explanation]
|
|
Suggested default: [what to assume if unanswered]
|
|
|
|
### Medium Impact
|
|
...
|
|
|
|
### Low Impact / Confirmations
|
|
...
|
|
```
|
|
|
|
Do not add your own questions — only consolidate what the two models asked.
|
|
Do not answer the questions — that's the human's job.
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 7 }
|
|
|
|
# ── Phase 1: Independent Planning ──────────────────────────────────
|
|
#
|
|
# Two models write specs independently. Both get ALL four best-practice
|
|
# docs, the interview answers (via the consolidated questions artifact,
|
|
# which the human has annotated with answers), and project context.
|
|
# Differences reflect genuine design disagreements, not knowledge gaps.
|
|
|
|
plan_a:
|
|
name: "Spec Draft A ({{ model_a or 'auto' }})"
|
|
depends_on: [consolidate_questions]
|
|
prompt: &plan_prompt |
|
|
You are a senior software architect writing a detailed specification.
|
|
|
|
## Task
|
|
{{ task_description }}
|
|
|
|
{% if scope_notes %}
|
|
## Scope Notes from Human
|
|
{{ scope_notes }}
|
|
{% endif %}
|
|
|
|
{% if existing_specs %}
|
|
## Existing Specifications (for context — extend, don't duplicate)
|
|
{{ existing_specs }}
|
|
{% endif %}
|
|
|
|
## Interview Answers
|
|
The following questions were asked during the interview phase.
|
|
The human has provided answers — use them to guide your design decisions.
|
|
Where a question has no answer, use the suggested default.
|
|
|
|
<<ARTIFACT:consolidate_questions:output>>
|
|
|
|
## Methodology
|
|
Read and apply these best-practice documents from `/opt/harness/context/planning/best-practices/`:
|
|
1. `spec-driven-development.md` — spec structure, requirement numbering, scenarios
|
|
2. `test-driven-development.md` — deriving tests from specs, edge case discovery
|
|
3. `security-architecture.md` — server boundary rule, defense in depth, auth patterns
|
|
4. `llm-code-security.md` — injection flaws, input validation, OWASP for AI code
|
|
|
|
{% if repo %}
|
|
## Project Context
|
|
The project repo has been cloned. Read `CLAUDE.md` and any existing `spec/` files
|
|
to understand the current architecture and conventions.
|
|
{% endif %}
|
|
|
|
## Output Requirements
|
|
Write a complete spec with these sections:
|
|
1. **Overview** — 2-3 sentences on what this subsystem does
|
|
2. **Responsibilities** — what it owns vs delegates
|
|
3. **Dependencies** — other specs to read
|
|
4. **Data Model** — types, schemas, with concrete JSON/YAML examples
|
|
5. **Requirements** — numbered (e.g., XX-1, XX-2), each independently testable, each with a "Why:" rationale
|
|
6. **Scenarios** — given/when/then for every requirement
|
|
7. **Security Considerations** — input validation, injection risks, access control, resource limits
|
|
8. **Test Strategy** — how each requirement maps to tests, edge cases to cover
|
|
|
|
Be opinionated. Make concrete design decisions with rationale. Call out trade-offs.
|
|
Do NOT leave things vague ("handle errors appropriately") — be specific ("return HTTP 413 with error body").
|
|
|
|
Write the complete spec to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 7, spec_adherence: 8 }
|
|
model_override: "{{ model_a }}"
|
|
|
|
plan_b:
|
|
name: "Spec Draft B ({{ model_b or 'auto' }})"
|
|
depends_on: [consolidate_questions]
|
|
prompt: *plan_prompt
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 7, spec_adherence: 8 }
|
|
model_override: "{{ model_b }}"
|
|
|
|
# ── Phase 2: Cross-Model Reviews ───────────────────────────────────
|
|
#
|
|
# 6 review nodes: 3 disciplines x 2 models. Each model reviews ONLY
|
|
# the OTHER model's plan. This eliminates self-review bias — a model
|
|
# reviewing its own plan tends to confirm its own decisions.
|
|
|
|
spec_review_a:
|
|
name: "Spec Review by A (of Plan B)"
|
|
depends_on: [plan_b]
|
|
prompt: |
|
|
You are reviewing a specification for adherence to spec-driven development practices.
|
|
You did NOT write this plan. Review it critically.
|
|
|
|
## Best Practice Reference
|
|
Read `/opt/harness/context/planning/best-practices/spec-driven-development.md` thoroughly before reviewing.
|
|
|
|
## Spec Under Review (written by another model)
|
|
<<ARTIFACT:plan_b:output>>
|
|
|
|
## Review Checklist
|
|
Evaluate:
|
|
1. **Structure** — Does it have all required sections (Overview, Responsibilities, Dependencies, Data Model, Requirements, Scenarios)?
|
|
2. **Requirements quality** — Are they numbered, independently testable, unambiguous? Does each have a "Why:" rationale?
|
|
3. **Scenarios** — Does every requirement have at least one given/when/then? Are edge cases covered?
|
|
4. **Cross-references** — Does it reference other specs correctly without duplicating?
|
|
5. **Concrete examples** — Are data models shown with realistic JSON/code, not just abstract schemas?
|
|
6. **Completeness** — Are there gaps? Requirements that should exist but don't?
|
|
|
|
## Output Format
|
|
List numbered improvement points with severity (CRITICAL/HIGH/MEDIUM/LOW).
|
|
Then: "## Questions for Human Review" — decisions needing human input (unclear requirements, business logic, scope).
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 9 }
|
|
model_override: "{{ model_a }}"
|
|
|
|
spec_review_b:
|
|
name: "Spec Review by B (of Plan A)"
|
|
depends_on: [plan_a]
|
|
prompt: |
|
|
You are reviewing a specification for adherence to spec-driven development practices.
|
|
You did NOT write this plan. Review it critically.
|
|
|
|
## Best Practice Reference
|
|
Read `/opt/harness/context/planning/best-practices/spec-driven-development.md` thoroughly before reviewing.
|
|
|
|
## Spec Under Review (written by another model)
|
|
<<ARTIFACT:plan_a:output>>
|
|
|
|
## Review Checklist
|
|
Evaluate:
|
|
1. **Structure** — Does it have all required sections (Overview, Responsibilities, Dependencies, Data Model, Requirements, Scenarios)?
|
|
2. **Requirements quality** — Are they numbered, independently testable, unambiguous? Does each have a "Why:" rationale?
|
|
3. **Scenarios** — Does every requirement have at least one given/when/then? Are edge cases covered?
|
|
4. **Cross-references** — Does it reference other specs correctly without duplicating?
|
|
5. **Concrete examples** — Are data models shown with realistic JSON/code, not just abstract schemas?
|
|
6. **Completeness** — Are there gaps? Requirements that should exist but don't?
|
|
|
|
## Output Format
|
|
List numbered improvement points with severity (CRITICAL/HIGH/MEDIUM/LOW).
|
|
Then: "## Questions for Human Review" — decisions needing human input (unclear requirements, business logic, scope).
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 9 }
|
|
model_override: "{{ model_b }}"
|
|
|
|
security_review_a:
|
|
name: "Security Review by A (of Plan B)"
|
|
depends_on: [plan_b]
|
|
prompt: |
|
|
You are performing a security review of a specification.
|
|
You did NOT write this plan. Review it critically.
|
|
|
|
## Best Practice References
|
|
Read these thoroughly before reviewing:
|
|
- `/opt/harness/context/planning/best-practices/security-architecture.md` — server boundary rule, defense in depth, proxy patterns
|
|
- `/opt/harness/context/planning/best-practices/llm-code-security.md` — injection flaws, OWASP for AI code, hallucinated packages
|
|
|
|
## Spec Under Review (written by another model)
|
|
<<ARTIFACT:plan_b:output>>
|
|
|
|
## Review Focus
|
|
Evaluate:
|
|
1. **Injection risks** — SQL, command, template, prompt, SSRF, path traversal
|
|
2. **Input validation** — are all inputs validated at system boundaries? Size limits?
|
|
3. **Authentication/Authorization** — who can access what? Multi-tenancy isolation?
|
|
4. **Secret handling** — are credentials ever exposed? Server boundary rule compliance?
|
|
5. **Resource exhaustion** — unbounded loops, unlimited sizes, missing timeouts?
|
|
6. **Data integrity** — race conditions, TOCTOU, atomic operations?
|
|
7. **LLM-specific risks** — prompt injection via user input, hallucinated dependencies, over-permissive defaults
|
|
|
|
## Output Format
|
|
For each finding: Severity (CRITICAL/HIGH/MEDIUM/LOW/INFO), attack vector, affected requirement ID, concrete fix.
|
|
Then: "## Questions for Human Review" — security decisions needing human judgement.
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 8 }
|
|
model_override: "{{ model_a }}"
|
|
|
|
security_review_b:
|
|
name: "Security Review by B (of Plan A)"
|
|
depends_on: [plan_a]
|
|
prompt: |
|
|
You are performing a security review of a specification.
|
|
You did NOT write this plan. Review it critically.
|
|
|
|
## Best Practice References
|
|
Read these thoroughly before reviewing:
|
|
- `/opt/harness/context/planning/best-practices/security-architecture.md` — server boundary rule, defense in depth, proxy patterns
|
|
- `/opt/harness/context/planning/best-practices/llm-code-security.md` — injection flaws, OWASP for AI code, hallucinated packages
|
|
|
|
## Spec Under Review (written by another model)
|
|
<<ARTIFACT:plan_a:output>>
|
|
|
|
## Review Focus
|
|
Evaluate:
|
|
1. **Injection risks** — SQL, command, template, prompt, SSRF, path traversal
|
|
2. **Input validation** — are all inputs validated at system boundaries? Size limits?
|
|
3. **Authentication/Authorization** — who can access what? Multi-tenancy isolation?
|
|
4. **Secret handling** — are credentials ever exposed? Server boundary rule compliance?
|
|
5. **Resource exhaustion** — unbounded loops, unlimited sizes, missing timeouts?
|
|
6. **Data integrity** — race conditions, TOCTOU, atomic operations?
|
|
7. **LLM-specific risks** — prompt injection via user input, hallucinated dependencies, over-permissive defaults
|
|
|
|
## Output Format
|
|
For each finding: Severity (CRITICAL/HIGH/MEDIUM/LOW/INFO), attack vector, affected requirement ID, concrete fix.
|
|
Then: "## Questions for Human Review" — security decisions needing human judgement.
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 8 }
|
|
model_override: "{{ model_b }}"
|
|
|
|
tdd_review_a:
|
|
name: "TDD Review by A (of Plan B)"
|
|
depends_on: [plan_b]
|
|
prompt: |
|
|
You are reviewing a specification for testability and test strategy quality.
|
|
You did NOT write this plan. Review it critically.
|
|
|
|
## Best Practice Reference
|
|
Read `/opt/harness/context/planning/best-practices/test-driven-development.md` thoroughly before reviewing.
|
|
|
|
## Spec Under Review (written by another model)
|
|
<<ARTIFACT:plan_b:output>>
|
|
|
|
## Review Focus
|
|
Evaluate:
|
|
1. **Requirement testability** — Can each requirement be tested without human judgement?
|
|
Flag requirements that say "appropriate", "reasonable", "as needed".
|
|
2. **Test coverage plan** — Does the test strategy cover all requirements? Any gaps?
|
|
3. **Edge cases** — Are boundary conditions, error paths, and concurrency scenarios covered?
|
|
4. **Test naming** — Do proposed test names include requirement IDs (e.g., test_wf23_artifact_upload)?
|
|
5. **Parameterised tests** — Where specs enumerate valid values, are parameterised tests suggested?
|
|
6. **Integration test boundary** — Is the line between unit and integration tests clear?
|
|
7. **Property-based testing** — Are there invariants that would benefit from hypothesis/property testing?
|
|
8. **Mock boundaries** — What should be mocked vs tested with real dependencies?
|
|
|
|
## Output Format
|
|
List numbered improvement points with severity.
|
|
Then: "## Questions for Human Review" — test scope decisions needing human input.
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 8 }
|
|
model_override: "{{ model_a }}"
|
|
|
|
tdd_review_b:
|
|
name: "TDD Review by B (of Plan A)"
|
|
depends_on: [plan_a]
|
|
prompt: |
|
|
You are reviewing a specification for testability and test strategy quality.
|
|
You did NOT write this plan. Review it critically.
|
|
|
|
## Best Practice Reference
|
|
Read `/opt/harness/context/planning/best-practices/test-driven-development.md` thoroughly before reviewing.
|
|
|
|
## Spec Under Review (written by another model)
|
|
<<ARTIFACT:plan_a:output>>
|
|
|
|
## Review Focus
|
|
Evaluate:
|
|
1. **Requirement testability** — Can each requirement be tested without human judgement?
|
|
Flag requirements that say "appropriate", "reasonable", "as needed".
|
|
2. **Test coverage plan** — Does the test strategy cover all requirements? Any gaps?
|
|
3. **Edge cases** — Are boundary conditions, error paths, and concurrency scenarios covered?
|
|
4. **Test naming** — Do proposed test names include requirement IDs (e.g., test_wf23_artifact_upload)?
|
|
5. **Parameterised tests** — Where specs enumerate valid values, are parameterised tests suggested?
|
|
6. **Integration test boundary** — Is the line between unit and integration tests clear?
|
|
7. **Property-based testing** — Are there invariants that would benefit from hypothesis/property testing?
|
|
8. **Mock boundaries** — What should be mocked vs tested with real dependencies?
|
|
|
|
## Output Format
|
|
List numbered improvement points with severity.
|
|
Then: "## Questions for Human Review" — test scope decisions needing human input.
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 8 }
|
|
model_override: "{{ model_b }}"
|
|
|
|
# ── Phase 3: Escalation ────────────────────────────────────────────
|
|
#
|
|
# Reads both plans AND all six cross-reviews. Since each model only
|
|
# reviewed the OTHER's plan, the escalation must correlate findings:
|
|
# "Model A found X in Plan B; Model B found Y in Plan A — is this
|
|
# the same underlying issue?" Also recommends which model should
|
|
# write the final synthesis based on review quality.
|
|
|
|
escalate:
|
|
name: "Decision Briefing for Human"
|
|
depends_on: [spec_review_a, spec_review_b, security_review_a, security_review_b, tdd_review_a, tdd_review_b]
|
|
prompt: |
|
|
You are preparing a decision briefing for a human reviewer.
|
|
You have two spec drafts and SIX cross-model reviews (each model reviewed
|
|
only the OTHER model's plan — no self-review). Your job: surface what needs
|
|
human decisions vs what the reviewers agree on.
|
|
|
|
## Plan A (written by Model A, reviewed by Model B)
|
|
<<ARTIFACT:plan_a:output>>
|
|
|
|
## Plan B (written by Model B, reviewed by Model A)
|
|
<<ARTIFACT:plan_b:output>>
|
|
|
|
## Reviews of Plan B (by Model A)
|
|
### Spec Review
|
|
<<ARTIFACT:spec_review_a:output>>
|
|
### Security Review
|
|
<<ARTIFACT:security_review_a:output>>
|
|
### TDD Review
|
|
<<ARTIFACT:tdd_review_a:output>>
|
|
|
|
## Reviews of Plan A (by Model B)
|
|
### Spec Review
|
|
<<ARTIFACT:spec_review_b:output>>
|
|
### Security Review
|
|
<<ARTIFACT:security_review_b:output>>
|
|
### TDD Review
|
|
<<ARTIFACT:tdd_review_b:output>>
|
|
|
|
## Output Format
|
|
|
|
### 1. Design Disagreements
|
|
For each disagreement between Plan A and Plan B:
|
|
- **Topic:** (e.g., "Artifact storage: dedicated table vs metadata inline")
|
|
- **Plan A approach:** (summary)
|
|
- **Plan B approach:** (summary)
|
|
- **What A's reviewer of B said:** (summary)
|
|
- **What B's reviewer of A said:** (summary)
|
|
- **Recommendation:** (your assessment, with rationale)
|
|
- **Decision needed:** YES / NO (YES if reviewers disagree or reasonable people could disagree)
|
|
|
|
### 2. Correlated Findings
|
|
Where Model A's review of Plan B and Model B's review of Plan A found
|
|
related issues (same underlying problem manifesting in both plans):
|
|
- **Issue:** (description)
|
|
- **In Plan A:** (what B's reviewer found)
|
|
- **In Plan B:** (what A's reviewer found)
|
|
- **Fix:** (recommended resolution)
|
|
|
|
### 3. Open Questions
|
|
Consolidate all "Questions for Human Review" from all six reviews.
|
|
Deduplicate. For each:
|
|
- **Question:** (clear, actionable)
|
|
- **Context:** (why this matters)
|
|
- **Options:** (concrete choices)
|
|
- **Default if no answer:** (what the synthesizer would pick)
|
|
|
|
### 4. Security Decisions
|
|
All CRITICAL and HIGH findings from both security reviews.
|
|
Note whether the finding affects Plan A, Plan B, or both.
|
|
|
|
### 5. Consensus Items
|
|
Areas where both plans agree AND reviewers found no issues.
|
|
These don't need human review.
|
|
|
|
### 6. Model Scoring Recommendation
|
|
Based on review findings, recommend which model should write the final synthesis:
|
|
- **Plan A review score:** (fewer/less severe findings = better)
|
|
- **Plan B review score:** (fewer/less severe findings = better)
|
|
- **Recommendation:** Model A or Model B for final synthesis, with rationale
|
|
|
|
Be concise. Lead with decisions, not context.
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 9, spec_adherence: 9 }
|
|
|
|
# ── Phase 4: Final Synthesis ───────────────────────────────────────
|
|
#
|
|
# In manual-workflow mode, the human reviews the escalation output,
|
|
# answers open questions, resolves conflicts, and sets best_model
|
|
# before this node runs. The best-scoring model writes the final spec.
|
|
|
|
synthesize:
|
|
name: "Final Spec Synthesis"
|
|
depends_on: [escalate]
|
|
prompt: |
|
|
You are producing the FINAL specification by combining two competing
|
|
drafts, six cross-model reviews, and an escalation briefing with human decisions.
|
|
|
|
## Plan A
|
|
<<ARTIFACT:plan_a:output>>
|
|
|
|
## Plan B
|
|
<<ARTIFACT:plan_b:output>>
|
|
|
|
## Reviews of Plan B (by Model A)
|
|
<<ARTIFACT:spec_review_a:output>>
|
|
<<ARTIFACT:security_review_a:output>>
|
|
<<ARTIFACT:tdd_review_a:output>>
|
|
|
|
## Reviews of Plan A (by Model B)
|
|
<<ARTIFACT:spec_review_b:output>>
|
|
<<ARTIFACT:security_review_b:output>>
|
|
<<ARTIFACT:tdd_review_b:output>>
|
|
|
|
## Escalation Briefing (with human decisions)
|
|
<<ARTIFACT:escalate:output>>
|
|
|
|
## Best Practice References
|
|
Read ALL of these from `/opt/harness/context/planning/best-practices/`:
|
|
- `spec-driven-development.md`
|
|
- `test-driven-development.md`
|
|
- `security-architecture.md`
|
|
- `llm-code-security.md`
|
|
|
|
## Instructions
|
|
|
|
1. **Start from the stronger plan** — the one with fewer and less severe review findings.
|
|
Don't average; pick the better foundation and incorporate the best from the other.
|
|
|
|
2. **Resolve all disagreements** using the escalation briefing.
|
|
Where the briefing says "Decision needed: YES" and human decisions are present,
|
|
follow the human's choice. Where no human response is present,
|
|
use the briefing's "Default if no answer" and mark with
|
|
`<!-- HUMAN DECISION PENDING: [topic] -->`.
|
|
|
|
3. **Address correlated findings** — issues found in both plans indicate a
|
|
fundamental problem that needs a different approach, not just a patch.
|
|
|
|
4. **Address all CRITICAL and HIGH security findings** as concrete requirement changes.
|
|
|
|
5. **Every requirement must have:** unique ID, "Why:" rationale, given/when/then scenario, test mapping.
|
|
|
|
6. **Include a Security Requirements section** and a **Test Strategy section**.
|
|
|
|
7. **Do NOT include** review commentary, improvement points, or process notes.
|
|
|
|
Write the complete final spec to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 9, spec_adherence: 9, creativity: 7 }
|
|
model_override: "{{ best_model }}"
|
|
|
|
# ── Phase 4.5: Post-Synthesis Security Reviews ─────────────────────
|
|
#
|
|
# Two models independently review the FINAL synthesized spec for
|
|
# security issues. This catches problems introduced during synthesis
|
|
# (merging two plans can create inconsistencies, gaps, or new attack
|
|
# surfaces that weren't in either original plan). Each model reviews
|
|
# the same spec — not cross-review, because there's only one spec now.
|
|
|
|
post_security_a:
|
|
name: "Post-Synthesis Security Review ({{ model_a or 'auto' }})"
|
|
depends_on: [synthesize]
|
|
prompt: &post_security_prompt |
|
|
You are performing a security review of a FINAL synthesized specification.
|
|
This spec was produced by merging two competing drafts and applying
|
|
review feedback. Your job: find security issues that survived synthesis
|
|
or were introduced by the merge process.
|
|
|
|
## Synthesized Spec
|
|
<<ARTIFACT:synthesize:output>>
|
|
|
|
## Best Practice References
|
|
Read these thoroughly before reviewing:
|
|
- `/opt/harness/context/planning/best-practices/security-architecture.md`
|
|
- `/opt/harness/context/planning/best-practices/llm-code-security.md`
|
|
- `/opt/harness/context/planning/best-practices/api-design.md` (if it exists)
|
|
|
|
{% if repo %}
|
|
## Project Context
|
|
The project repo has been cloned. Read `CLAUDE.md` and any existing `spec/` files
|
|
for architectural context (existing auth patterns, data flows, trust boundaries).
|
|
{% endif %}
|
|
|
|
## Review Focus
|
|
1. **Injection risks** — SQL, command, template, prompt, SSRF, path traversal
|
|
2. **Input validation** — all inputs validated at system boundaries? Size limits?
|
|
3. **Authentication/Authorization** — who can access what? Identity spoofing?
|
|
4. **Secret handling** — credentials exposed? Server boundary rule?
|
|
5. **Resource exhaustion** — unbounded loops, unlimited sizes, missing timeouts?
|
|
6. **Data integrity** — race conditions, TOCTOU, atomic operations?
|
|
7. **LLM-specific risks** — prompt injection, hallucinated deps, over-permissive defaults
|
|
8. **Synthesis artifacts** — inconsistencies between merged sections, conflicting
|
|
requirements, gaps where one plan's approach was dropped but the replacement
|
|
was incomplete
|
|
|
|
## Output Format
|
|
Structure your output into two clear sections:
|
|
|
|
### Fixes (implement directly — no human decision needed)
|
|
For each finding that has an unambiguous fix:
|
|
- **ID:** F-NN
|
|
- **Severity:** CRITICAL / HIGH / MEDIUM / LOW / INFO
|
|
- **Requirement:** affected requirement ID(s)
|
|
- **Issue:** what's wrong
|
|
- **Current text:** quote the problematic spec text
|
|
- **Fixed text:** exact replacement text
|
|
- **Why:** rationale for the fix
|
|
|
|
### Escalations (need human decision)
|
|
For each finding where reasonable people could disagree on the fix:
|
|
- **ID:** E-NN
|
|
- **Severity:** CRITICAL / HIGH / MEDIUM / LOW
|
|
- **Requirement:** affected requirement ID(s)
|
|
- **Issue:** what's wrong
|
|
- **Options:** concrete choices (A, B, C) with trade-offs
|
|
- **Default recommendation:** what you'd pick and why
|
|
|
|
Be specific. Quote exact spec text. Provide exact replacement text for fixes.
|
|
Do NOT suggest vague improvements ("consider adding validation") — specify
|
|
exactly what validation, on what field, with what error code.
|
|
|
|
Write to /workspace/project/output.md
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 8 }
|
|
model_override: "{{ model_a }}"
|
|
|
|
post_security_b:
|
|
name: "Post-Synthesis Security Review ({{ model_b or 'auto' }})"
|
|
depends_on: [synthesize]
|
|
prompt: *post_security_prompt
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { spec_adherence: 8 }
|
|
model_override: "{{ model_b }}"
|
|
|
|
# ── Phase 4.6: Auto-Fix (non-escalated findings) ──────────────────
|
|
#
|
|
# A single agent reads both security reviews, deduplicates findings,
|
|
# applies all fixes that don't need human decisions directly to the
|
|
# spec text, and collects escalations into a briefing for the human.
|
|
# Output: the updated spec + an escalation summary.
|
|
|
|
auto_fix:
|
|
name: "Apply Security Fixes"
|
|
depends_on: [post_security_a, post_security_b]
|
|
prompt: |
|
|
You are implementing security fixes on a specification.
|
|
Two independent security reviews have been performed on the synthesized spec.
|
|
Your job: apply all non-controversial fixes and prepare escalations for human review.
|
|
|
|
## Synthesized Spec (the document to modify)
|
|
<<ARTIFACT:synthesize:output>>
|
|
|
|
## Security Review A
|
|
<<ARTIFACT:post_security_a:output>>
|
|
|
|
## Security Review B
|
|
<<ARTIFACT:post_security_b:output>>
|
|
|
|
## Instructions
|
|
|
|
### Step 1: Triage and Deduplicate
|
|
Compare both reviews. Many findings will overlap (same issue found by both).
|
|
Create a single merged list with:
|
|
- Deduplicated findings (note when both reviewers found the same issue)
|
|
- Severity from the stricter reviewer (if A says HIGH and B says MEDIUM, use HIGH)
|
|
|
|
### Step 2: Categorize
|
|
Split findings into:
|
|
- **Fixes** — clear, unambiguous improvements. Apply these directly.
|
|
- **Escalations** — findings where the fix involves a design trade-off,
|
|
changes the external API, or where the two reviewers disagree on the approach.
|
|
|
|
### Step 3: Apply Fixes
|
|
For each fix, modify the spec text directly. Track what you changed:
|
|
- Requirement ID
|
|
- What changed (old text → new text, summarized)
|
|
- Which review(s) identified the issue
|
|
|
|
### Step 4: Write Escalation Summary
|
|
For each escalation, include:
|
|
- Finding description
|
|
- Which reviewer(s) raised it
|
|
- Options with trade-offs
|
|
- Default recommendation
|
|
|
|
## Output
|
|
Write TWO files:
|
|
|
|
**`/workspace/project/output.md`** — The complete updated spec with all
|
|
non-escalated fixes applied. This should be the full spec text, ready to save
|
|
as the final spec file. Include a comment at the top:
|
|
`<!-- Security fixes applied: N fixes from post-synthesis review. M escalations pending human review. -->`
|
|
|
|
**`/workspace/project/escalations.md`** — Escalation summary for human
|
|
review. Include:
|
|
- Total findings: N (X fixes applied, Y escalations)
|
|
- Deduplication stats (how many found by both reviewers)
|
|
- Each escalation with options and default recommendation
|
|
- "Accepted risks" section for INFO-level findings that don't need action
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 9, spec_adherence: 9 }
|
|
|
|
# ── Phase 4.7: Final Fix Implementation ────────────────────────────
|
|
#
|
|
# After the human reviews escalations (via the human gate between
|
|
# auto_fix and final_fix in manual-workflow mode), this agent applies
|
|
# the human's decisions to the spec. The human annotates the
|
|
# escalations artifact with their choices before this node runs.
|
|
|
|
final_fix:
|
|
name: "Apply Escalation Decisions"
|
|
depends_on: [auto_fix]
|
|
prompt: |
|
|
You are applying human decisions to a specification that has already had
|
|
non-controversial security fixes applied.
|
|
|
|
## Spec with Auto-Fixes Applied
|
|
<<ARTIFACT:auto_fix:output>>
|
|
|
|
## Escalation Summary (with human decisions)
|
|
<<ARTIFACT:auto_fix:escalations>>
|
|
|
|
## Instructions
|
|
|
|
1. Read the escalation summary. The human has annotated each escalation
|
|
with their decision (which option to implement, or a custom approach).
|
|
|
|
2. For each escalation where the human provided a decision:
|
|
- Apply the chosen fix to the spec text
|
|
- If the human chose a custom approach, implement it faithfully
|
|
|
|
3. For escalations where the human did NOT provide a decision:
|
|
- Apply the default recommendation noted in the escalation
|
|
- Mark with `<!-- DEFAULT APPLIED: [topic] — human did not override -->`
|
|
|
|
4. For any "Accepted risks" the human flagged as needing action after all:
|
|
- Implement the fix
|
|
|
|
5. Verify requirement ID uniqueness — no duplicate IDs after all changes.
|
|
|
|
6. Verify all scenarios still match their requirements after text changes.
|
|
|
|
## Output
|
|
Write the FINAL spec to `/workspace/project/output.md`.
|
|
This is the production-ready spec. No review commentary, no TODOs,
|
|
no pending decisions. Every requirement has an ID, rationale, and scenario.
|
|
|
|
At the top, include a summary comment:
|
|
`<!-- Final spec: N escalations resolved, M defaults applied. Ready for merge. -->`
|
|
harness: planning/v1
|
|
requirements:
|
|
min_scores: { complexity: 9, spec_adherence: 9 }
|
|
model_override: "{{ best_model }}"
|
|
|
|
scoring:
|
|
type: comparative_review
|
|
reviews:
|
|
# Model A reviewed Plan B (spec, security, TDD)
|
|
- review_node: spec_review_a
|
|
scored_model_node: plan_b
|
|
dimension: spec_adherence
|
|
- review_node: security_review_a
|
|
scored_model_node: plan_b
|
|
dimension: spec_adherence
|
|
- review_node: tdd_review_a
|
|
scored_model_node: plan_b
|
|
dimension: test_pass_rate
|
|
# Model B reviewed Plan A (spec, security, TDD)
|
|
- review_node: spec_review_b
|
|
scored_model_node: plan_a
|
|
dimension: spec_adherence
|
|
- review_node: security_review_b
|
|
scored_model_node: plan_a
|
|
dimension: spec_adherence
|
|
- review_node: tdd_review_b
|
|
scored_model_node: plan_a
|
|
dimension: test_pass_rate
|
|
method: fewer_improvement_points_wins
|