best-practices/ai-parallel-agents.md

AI Parallel Agents

Patterns for orchestrating parallel AI agents inside a Claude Code (or similar) workflow — research fan-out, file-contention avoidance, tool-capability limits, context-budget discipline, and dataset-wide audits via background agents.

Cross-references: Agent Repos & Container Agent Operations covers the container-agent execution environment (harnesses, agent repos, CP dispatch). This file covers the orchestration patterns a main-thread Claude Code session uses when spawning and coordinating subagents.

---

1. Dispatch Parallel Agents for Multi-Facet Research

Principle: When researching a topic with multiple independent facets, dispatch parallel research agents rather than running sequential queries from the main thread.

Why it matters: Sequential investigation delays cross-facet contradiction discovery to the point where it is expensive to address (often after a plan has been written). Scoping each agent narrowly to one facet — per API, per jurisdiction, per sub-topic, per source-type filter — both parallelises the work and improves signal-to-noise because each agent's context is tightly focused on one domain.

How to implement:

• Enumerate the facets of the research topic before dispatching. For an integration project, facets are usually "per external API". For a compliance question, facets are usually "per jurisdiction". For a market survey, facets are usually "per source category" (vendor docs, academic papers, blog posts).
• Spawn one agent per facet, each with a narrow prompt: "Research X specifically in the context of Y. Do not cover adjacent topics."
• For integration work that touches 2+ external APIs, spawn one research agent per API, plus a parallel plan-reviewer agent that checks cross-API compatibility as soon as the per-API agents report back.
• Merge results in the main thread — deduplicate, flag contradictions, and surface cross-facet constraints.

Anti-patterns:

• Running a single agent with "research topics X, Y, and Z" — context dilution, and any blocker in X delays Y and Z.
• Sequential WebFetch calls from the main thread when the facets are independent.
• Dispatching parallel agents with overlapping scope — they return the same information and the dedup cost eats the parallelism savings.
• Waiting for research agent N to finish before dispatching reviewer agents — run them concurrently when they have no data dependency.

---

2. File Contention: Agents Return Text, Main Thread Writes

Principle: Never have two parallel agents edit the same file. Have agents RETURN prepared text in their final message and apply edits sequentially from the main thread.

Why it matters: Subagent writes to a shared file silently collide — one edit wins, the other is lost, and there is no error or warning. Verified in practice during a 195-claim verification across 16 files: 4 parallel agents, each handling 3-4 files, returning their prepared edits as text, applied by the main thread — zero conflicts, one round.

How to implement:

• Partition the file set so each agent owns a non-overlapping subset. Never assign the same file to two agents.
• Instruct agents explicitly: "Return your proposed edits as text in your final message. Do not write to disk." Include an example of the expected return format (e.g., file path + old/new block per edit).
• When overlap is unavoidable (e.g., a cross-cutting change to every file), have agents RETURN their changes; apply edits from the main thread in a deterministic order.
• For large batches, consider a two-phase pattern: parallel agents produce proposed edits as text; main thread reviews and applies.

Anti-patterns:

• "Each agent edits the files relevant to its findings" — guaranteed to produce silent write collisions when scopes overlap.
• Letting agents write to a shared directory without partitioning — last write wins, earlier writes vanish.
• Assuming filesystem-level locking will save you — Claude Code agents do not hold locks across calls.
• Relying on git to detect the collision — an agent that reads the pre-collision state and writes after the other agent has the correct "new" content, and git sees only the last write.

---

3. Subagents Cannot WebFetch — Perform Fetches in Main Thread

Principle: Subagents cannot use WebFetch or WebSearch (permission denied by the harness). Perform web fetches in the main conversation and delegate file processing — reading, verification, extraction, summarisation — to agents.

Why it matters: Telling a subagent to "go fetch X from the web" fails silently from the orchestrator's perspective — the subagent returns what it "knows" (training data) rather than what's on the web today. For container agents the restriction is stricter still: no network tools at all, and training-data version claims have been measured at ~30% inaccuracy.

How to implement:

• Use WebFetch / WebSearch in the main thread to pull live content into files or into the context.
• Pass the fetched content to subagents as text (in their prompt) or as file paths they can Read.
• For research agents that need multi-source fetches, fetch everything in the main thread first, then dispatch agents to process/summarise the local content.
• For container agents, treat all external network access as unavailable — run a separate web-validation pass from the main conversation after the container agent completes.

Anti-patterns:

• "Dispatch a subagent to research X on the web" — subagent has no web tools, will fabricate from training data.
• Telling container agents to "use web search" — they have none and will not tell you so.
• Trusting version numbers, release dates, or "actively maintained" claims from a subagent that had no network access — validate separately.

---

4. Narrow Read Instructions to Prevent Context Blowup

Principle: Give agents narrow read instructions so each agent's context stays small. Name specific sections, grep patterns, or line ranges rather than "read the file."

Why it matters: Parallel agents that each read every full file consume tokens without improving output quality. A 16-file, 195-claim verification run blows up if every agent reads every file; it stays cheap if each agent reads only the section relevant to its claim. Context is the scarce resource in multi-agent workflows.

How to implement:

• In the agent prompt, specify the exact section or line range: "Read only the ## Key Data Points section of each file" or "Read lines 120-180 of spec/ingestion.md."
• When the agent needs to find the relevant section itself, instruct it to grep first and Read only matching line windows — not the whole file.
• For large source trees, supply a curated file list; do not let the agent glob an entire repo.
• Review agent prompts for accidental "read everything" phrasing — "look through the spec" is unbounded, "read Section 3.2 of the spec" is bounded.

Anti-patterns:

• "Read all the files in the spec directory and tell me X" — N agents × M files = huge context burn.
• Open-ended research prompts with no read boundaries — agents default to reading everything.
• Long agent prompts that themselves include full file contents when a section would suffice.
• Failing to cross-reference against an existing index file (CLAUDE.md, MEMORY.md) — forcing every agent to rediscover the repo structure.

---

5. Background Agents for Dataset-Wide Audits

Principle: When a question requires surveying every record in a large dataset — field gap analysis, distribution across files, unknown-unknowns discovery — dispatch a background (container) agent with a single well-scoped sweep prompt and a research-document output. One complete pass produces better results than piecemeal interactive exploration.

Why it matters: Interactive main-thread exploration of large datasets is slow, context-expensive, and prone to anchoring on whatever the operator looked at first. A background agent with a single sweep prompt can process the full dataset in one pass, produce a structured research artifact, and surface patterns the interactive session would miss.

How to implement:

• Frame the audit as a complete question: "For every ticket in the dataset, classify by [dimensions]. Produce a research document at research/<audit-name>.md with sections [X, Y, Z] and a summary table."
• Give the agent the full dataset location (repo path, data export, dataset manifest) and explicit output instructions.
• Prefer one deeper pass over many shallow passes — the marginal cost of one big research run is usually lower than the cumulative cost of back-and-forth.
• Store research outputs in a dedicated folder (e.g., research/) separate from code and docs. Humans review, distill the useful findings, and archive or discard the raw research.
• For recurring audits (weekly, per-milestone), template the prompt so it can be re-run with a date/scope parameter.

Anti-patterns:

• Running the sweep interactively in the main thread — slow and context-expensive.
• Multiple overlapping sweeps with no consolidation — hard to reason about what's covered.
• No dedicated output folder — research artifacts get lost alongside code.
• Never reading the research output after dispatch — a "fire and forget" audit with no human review has no value.

Cheap-Model Sub-Sessions: When to Use, When to Skip

Tools that fork a cheap-model session for grunt work (/ask-minimax, similar MiniMax-, Qwen-, or Haiku-backed delegators) shine on a narrow band of tasks. Reach for them when the file payload dwarfs the answer payload.

Use for:

• Summarising large files (logs, dumps, generated reports >500 lines)
• Extracting specific facts from many files or long reference docs
• Generating big files (migrations, fixtures, large docs) whose content does not need to flow back
• Format conversion of large files (CSV↔JSON, XML↔YAML)
• Bulk find-and-extract where only the matches matter

Skip for:

• Iterative design or debugging — the parent session needs the content in context
• Small files (<500 lines) — overhead exceeds savings
• Tasks where you will immediately re-read the result to act on it
• Anything requiring tools the cheap model lacks (web fetch, MCP, browser, agent dispatch)
• Architecture or quality-sensitive output — cheap models are for grunt work, not nuanced reasoning
• Per-test verification loops — the LLM round-trip cost dwarfs the test's own cost; the cheap model also stalls silently on tasks requiring sustained state across iterations. Replace with an AST script.

Delegate by reference, not content: pass file paths, not file contents. If you read the input files yourself before invoking, you have already paid the token cost the skill exists to avoid.

---

6. Cost-Effective Models: Single-File, Single-Rule Task Scope

Principle: Cheap/small models (MiniMax, Qwen3.x, Haiku via airouter) reliably produce output when scoped to ONE file and ONE rule per task. Multi-file or multi-rule tasks silently produce no output even when the agent's reasoning logs look correct.

Why it matters: Cost-effective models trade context-management ability for token cost. With multi-file scope, the model loses track of which file it has already edited and produces empty output or commits without the expected files. The failure is silent — exit code 0, reasoning logs look fine, no output file. Verified in a 9-dispatch batch: single-file scope succeeded 8/8, multi-file scope failed 2/2 with no output.

How to implement:

• One file per task — split multi-file changes into separate dispatches. Each task creates or edits exactly one file.
• Reference the template, don't inline it — give the agent the path to an existing similar file to read, rather than including all structure in the prompt.
• State the file path explicitly — "Create <exact/path/to/file>" at the start of the prompt, not implied by context.
• Verify output before treating as done — check the agent-repo branch (or output directory) for the expected file immediately after task success. Don't trust the exit code alone.

Sequential chain for unavoidable multi-file work: dispatch task-1 to produce file-A, wait for success, then dispatch task-2 to produce file-B referencing task-1's output (passed via git branch artifact). This bounds the agent's scope to one artifact at a time and lets each task verify the previous one's output before continuing.

Anti-patterns:

• Dispatching a single cheap-model task with "create files X, Y, Z" — silent no-output is the typical result.
• Trusting exit code = 0 from a cheap-model dispatch — always verify the expected file landed on disk or the agent-repo branch.
• Using a cheap model for tasks that require sustained state across many turns — the failure mode is silent stall, not error.

Cross-reference: Agent Repos "Airouter Qwen3.6 — Scope Decomposition" has the production dispatch pattern and time-to-success bands.

---

Summary

Parallel agent orchestration multiplies throughput when agents have narrowly scoped work, disjoint file sets, and the right tool capabilities. The main thread holds responsibilities subagents cannot do: web fetches, file writes on contested paths, and final integration of returned text. For large datasets, a single background sweep with a research-document output usually beats interactive back-and-forth. Cost-effective models need tighter scope still — one file, one rule, verified output.