docs(airouter): add scope-decomposition guidance for Qwen3.6 reliability
Airouter Qwen3.6 reliable for single-file tasks; unreliable for multi-file/multi-rule. D7 (single harness + one rule) = success. D4 (two files, multiple rules) = silent no-output. Pattern: decompose airouter tasks to one file per dispatch. BUG-21 reference included. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -473,3 +473,23 @@ Smaller/cheaper models (e.g., MiniMax, Haiku) need tighter scope constraints tha
|
|||||||
- **Explicit output location** — "Write to `results/output.md`" not "Write your findings"
|
- **Explicit output location** — "Write to `results/output.md`" not "Write your findings"
|
||||||
|
|
||||||
Treat scope boundaries as a harness concern, not an agent concern — encode them in the prompt template or harness context, not in ad-hoc task prompts.
|
Treat scope boundaries as a harness concern, not an agent concern — encode them in the prompt template or harness context, not in ad-hoc task prompts.
|
||||||
|
|
||||||
|
## Airouter Qwen3.6 — Scope Decomposition for Reliable Output
|
||||||
|
|
||||||
|
Qwen3.6 via airouter harness (`runtime_cli: agentic`, model `Qwen3.6`) produces reliable output for single, concrete file-creation tasks but has shown silent failures on multi-part instructions requiring sustained context. In measured dispatches:
|
||||||
|
- **Reliable**: Create one new file, one hard rule, clear output path
|
||||||
|
- **Unreliable**: Create multiple files with distinct content, multi-step instructions, tasks requiring internal state management across turns
|
||||||
|
|
||||||
|
**Pattern observed (2026-05-04):**
|
||||||
|
- D7 (single harness.yaml + single CLAUDE.md with one hard rule) → fully successful, correctly committed
|
||||||
|
- D4 (same template + same model, but two files with multiple layered rules each) → silent no-output, only unrelated `.gitignore` change appeared
|
||||||
|
|
||||||
|
**Dispatch strategy for airouter agents:**
|
||||||
|
1. **One file per task** — split multi-file tasks into separate dispatches. Each task creates exactly one file.
|
||||||
|
2. **Reference the template, don't copy it** — give the agent the path to an existing similar file to read, rather than including all structure in the prompt.
|
||||||
|
3. **State the file path explicitly** — "Create `harnesses/contexts/cp-harness/v1/harness.yaml`" at the start, not implied by context.
|
||||||
|
4. **Verify output before treating as done** — check the agent-repo branch for the expected file immediately after task succeeds.
|
||||||
|
|
||||||
|
If a multi-file task is unavoidable, use a **sequential chain**: dispatch task-1 to create file-A, wait for success, then dispatch task-2 to create file-B referencing task-1's output. This bounds the agent's scope to one artifact at a time.
|
||||||
|
|
||||||
|
**BUG-21** (2026-05-04): D4 task (`017f63cb`) exited 0, showed correct understanding in logs, but produced no harness files. Likely causes: prompt complexity, early exit, or internal runner issue. Fix: decompose airouter tasks to single-file units.
|
||||||
|
|||||||
Reference in New Issue
Block a user