Rework single-agent workflows; add write-tests; remove comparative-plan
- claude-agent → standard-agent (complexity≥8, spec_adherence≥9 → Sonnet-class) - smart-agent: complexity≥9 → Opus; documents MiniMax not in registry - functional-agent: labels=[airouter], cost_efficiency≥9 → Qwen3.6 - write-tests: two-node DAG (write_tests → review_coverage), spec-driven test authoring with coverage gap report in JSON - Remove comparative-plan (superseded by tdd-impl audit pattern) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
117
workflows/write-tests.yaml
Normal file
117
workflows/write-tests.yaml
Normal file
@@ -0,0 +1,117 @@
|
||||
name: write-tests
|
||||
version: 1
|
||||
description: >
|
||||
Two-node spec-driven test writing workflow. First node writes tests from the spec
|
||||
(one test per requirement, edge cases, property tests where applicable). Second node
|
||||
reviews coverage — checking every numbered requirement has a corresponding test and
|
||||
flagging any gaps.
|
||||
|
||||
Output: test files committed to the agent-repo branch; coverage gap report in the
|
||||
review node's output artifact.
|
||||
|
||||
params:
|
||||
required:
|
||||
spec_file:
|
||||
type: string
|
||||
description: "Path to the spec file in the repo (e.g. spec/my-feature.md)"
|
||||
project_id:
|
||||
type: string
|
||||
description: "Target project identifier"
|
||||
repo_url:
|
||||
type: string
|
||||
description: "Git repo URL containing the spec"
|
||||
agent_repo_url:
|
||||
type: string
|
||||
description: "Agent repo URL for branch-per-task output persistence"
|
||||
optional:
|
||||
task_description:
|
||||
type: string
|
||||
default: ""
|
||||
description: "Additional context about what is being tested (supplements the spec)"
|
||||
test_output_dir:
|
||||
type: string
|
||||
default: "tests/"
|
||||
description: "Directory to write test files into (default: tests/)"
|
||||
test_framework:
|
||||
type: string
|
||||
default: "pytest"
|
||||
description: "Test framework to use (default: pytest)"
|
||||
writer_harness:
|
||||
type: string
|
||||
default: "test-writing-opus-repo/v1"
|
||||
description: "Harness for the test-writing node (default: test-writing-opus-repo/v1)"
|
||||
reviewer_harness:
|
||||
type: string
|
||||
default: "planning/v1"
|
||||
description: "Harness for the coverage review node (default: planning/v1)"
|
||||
|
||||
nodes:
|
||||
write_tests:
|
||||
name: "Write tests from spec"
|
||||
prompt: |
|
||||
You are writing a test suite from a spec file. Tests are the contract — they will
|
||||
be run against an agent's implementation. Write them now, before the implementation exists.
|
||||
|
||||
## Spec file
|
||||
Read the spec at `{{ spec_file }}` in the cloned repo at `/workspace/project/`.
|
||||
|
||||
{% if task_description %}
|
||||
## Additional context
|
||||
{{ task_description }}
|
||||
{% endif %}
|
||||
|
||||
## Instructions
|
||||
1. Read the spec carefully. Identify every numbered requirement (e.g. MY-1, MY-2...).
|
||||
2. For each requirement, write at least one test function named `test_<req_id>_<short_description>`.
|
||||
3. Cover edge cases explicitly — empty inputs, boundary values, error paths.
|
||||
4. Add property-based tests (Hypothesis) where the requirement involves ranges or invariants.
|
||||
5. Write tests to `{{ test_output_dir }}` — one file per logical group (mirrors spec sections).
|
||||
6. Use `{{ test_framework }}` conventions. Tests should fail immediately (no implementation exists).
|
||||
7. Do NOT write any implementation code. Only test files.
|
||||
|
||||
After writing all tests, write a coverage summary to `/workspace/project/output.md`:
|
||||
- List every requirement ID from the spec
|
||||
- For each: which test function(s) cover it
|
||||
- Note any requirements you could not write a test for (and why)
|
||||
harness: "{{ writer_harness }}"
|
||||
requirements:
|
||||
min_scores:
|
||||
spec_adherence: 9
|
||||
test_pass_rate: 8
|
||||
|
||||
review_coverage:
|
||||
name: "Review test coverage"
|
||||
depends_on: [write_tests]
|
||||
prompt: |
|
||||
You are reviewing the coverage of a freshly written test suite against its spec.
|
||||
|
||||
## Spec file
|
||||
Read the spec at `{{ spec_file }}` in the cloned repo at `/workspace/project/`.
|
||||
|
||||
## Test summary written by the previous agent
|
||||
<<ARTIFACT:write_tests:output>>
|
||||
|
||||
## Instructions
|
||||
1. Read the spec and extract every numbered requirement.
|
||||
2. Cross-reference each requirement against the test summary above.
|
||||
3. For each requirement, determine: fully covered / partially covered / not covered.
|
||||
4. Flag any test functions that appear to test behaviour NOT in the spec (over-specification).
|
||||
|
||||
Write your review to `/workspace/project/output.md` as:
|
||||
|
||||
```json
|
||||
{
|
||||
"covered": ["REQ-1", "REQ-2"],
|
||||
"partial": [{"id": "REQ-3", "reason": "happy path only, error case missing"}],
|
||||
"missing": ["REQ-4", "REQ-5"],
|
||||
"over_specified": ["test_something_not_in_spec"],
|
||||
"summary": "one paragraph"
|
||||
}
|
||||
```
|
||||
|
||||
Then list specific, actionable improvements for any partial or missing coverage.
|
||||
harness: "{{ reviewer_harness }}"
|
||||
requirements:
|
||||
min_scores:
|
||||
complexity: 8
|
||||
spec_adherence: 9
|
||||
Reference in New Issue
Block a user