best-practices/agent-repos.md

Agent Repos & Container Agent Operations

Best practices for running AI agents via the agent-runtimes control plane — submitting tasks, using agent repos for persistence, running multi-model workflows, and monitoring progress.

Architecture Overview

The agent-runtimes system runs AI coding agents inside ephemeral Docker/K8s containers, orchestrated by a control plane (CP). The three-tier model:

1. Control Plane — central task queue, dispatcher registry, model routing. External systems (including Claude Code sessions) submit tasks here.
2. Dispatchers — poll the CP for tasks, resolve harness+model, create containers. All connections outbound (works behind firewalls).
3. Agent Containers — ephemeral, isolated. Receive a JSON payload, execute work, exit.

You (Claude Code session) → POST /tasks → Control Plane → Dispatcher → Agent Container
                           ← GET /tasks/{id} ← (poll for results)

Access

Environment	CP URL
K8s (port-forward)	kubectl port-forward -n agent-runtimes svc/controlplane 8100:8100 then http://localhost:8100
K8s (MFA)	https://agents.oreillyit.nz (behind Authelia)
Local dev (Docker Compose)	http://localhost:8100 after docker compose up -d

---

Submitting Tasks

Minimal Task

curl -s -X POST http://localhost:8100/tasks \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Fix test_auth.py",
    "project_id": "my-project",
    "prompt": "Fix the failing test in test_auth.py",
    "harness": "planning-opus-repo/v1",
    "runtime": {"cli": "claude"},
    "pre_actions": [
      {"type": "clone", "repo": "git@gitea.oreillyit.nz-ai-enablement:skynet/my-project.git", "depth": 1}
    ]
  }'

Full Task Payload

{
  "task_id": "optional-uuid",
  "name": "Short name (shown in monitor)",
  "project_id": "groups tasks in monitor",
  "prompt": "The instruction for the agent",
  "harness": "composite-harness-name/v1",
  "runtime": {
    "cli": "claude",
    "model": "sonnet",
    "timeout": 1800
  },
  "priority": 0,
  "metadata": {
    "workflow": "my-workflow",
    "phase": "plan",
    "model": "opus"
  },
  "pre_actions": [
    {"type": "clone", "repo": "git@...", "branch": "main", "depth": 1}
  ],
  "on_success": [
    {"type": "commit_pr", "branch": "feature-branch", "title": "PR title", "base": "main"}
  ],
  "on_error": [
    {"type": "report", "webhook_url": "https://..."}
  ],
  "max_retries": 1
}

Key Fields

Field	Required	Description
name	Recommended	Short task name shown in the monitor
project_id	Recommended	Groups tasks in the monitor; enables filtering
prompt	Yes	The instruction sent to the agent
harness	Recommended	Composite harness (defines credentials, context, model provider)
runtime.cli	No	CLI runner: claude (default), openai_compat, agentic
pre_actions	No	Setup actions before agent runs (e.g., clone)
on_success	No	Post-agent actions on success (e.g., commit_pr, report)
on_error	No	Post-agent actions on failure
metadata	No	Arbitrary JSONB — used for workflow tracking, filtering
priority	No	-100 to 100 (higher = picked first, default 0)

Python Task Submission

For programmatic use from a Claude Code session:

import requests, json, uuid

CP = "http://localhost:8100"
task_id = str(uuid.uuid4())

payload = {
    "task_id": task_id,
    "name": "My agent task",
    "project_id": "my-project",
    "prompt": "...",
    "harness": "planning-opus-repo/v1",
    "runtime": {"cli": "claude"},
    "pre_actions": [{"type": "clone", "repo": "git@gitea.oreillyit.nz-ai-enablement:skynet/my-project.git"}],
    "metadata": {"workflow": "my-workflow", "phase": "plan"}
}

resp = requests.post(f"{CP}/tasks", json=payload, timeout=10)
data = resp.json()
print(f"Submitted: {data['task_id']}")

---

Available Harnesses

Harnesses define what credentials, context, and model provider an agent gets. Composites combine multiple layers.

Composite Harnesses (ready to use)

Harness	Model Provider	Capabilities
planning-opus-repo/v1	Anthropic Claude (subscription)	Planning context + SSH clone
planning-minimax-repo/v1	MiniMax	Planning context + SSH clone
python-code-review/v1	Default (Anthropic)	Python dev tools + Gitea admin

Context Layers (building blocks)

Layer	What it provides
planning/v1	Best-practices files for spec/plan writing
gitea-ssh/v1	SSH key for clone/push to Gitea
gitea-admin/v1	Gitea admin context (SSH + API token)
anthropic-cloud/v1	Anthropic API (subscription pricing)
minimax/v1	MiniMax API (SOPS-encrypted credentials)
code-methodology/v1	Coding methodology CLAUDE.md

Capability Layers

Layer	What it provides
python-dev/v1	pytest, ruff, mypy, hypothesis, uv

---

Monitoring Tasks

agent-monitor (terminal UI)

# Live view, filtered to your project
~/dev/claude/projects/agent-runtimes/scripts/agent-monitor --filter "project=my-project" --filter "age<20m"

# Single snapshot
~/dev/claude/projects/agent-runtimes/scripts/agent-monitor --once

# All running tasks
~/dev/claude/projects/agent-runtimes/scripts/agent-monitor --filter "state=running"

API Queries

CP=http://localhost:8100

# Check task status
curl -s "$CP/tasks/{task_id}" | python3 -m json.tool

# List tasks (most recent)
curl -s "$CP/tasks?limit=10" | python3 -m json.tool

# Cancel a task
curl -s -X DELETE "$CP/tasks/{task_id}"

Extracting Agent Output from Logs

Agents write to /workspace/.agent-output/output.md. To extract this from stream-json logs:

import requests, json

def extract_output(cp_url, task_id):
    """Extract output.md content from completed task logs."""
    r = requests.get(f"{cp_url}/tasks/{task_id}", timeout=10).json()
    logs = r.get("logs", "") or ""
    output = None
    for line in logs.split("\n"):
        if not line.startswith("{"):
            continue
        try:
            event = json.loads(line)
        except:
            continue
        if event.get("type") == "assistant":
            for block in event.get("message", {}).get("content", []):
                if isinstance(block, dict) and block.get("type") == "tool_use":
                    if block.get("name") == "Write" and "output.md" in str(block.get("input", {}).get("file_path", "")):
                        output = block["input"]["content"]
    return output

---

Workflows (Multi-Model Spec Planning)

Workflow templates define multi-step DAGs where different models collaborate, cross-review, and a human resolves disputes.

spec-planning v3

The flagship workflow for spec development. 14-node DAG:

Phase 0:   interview_a + interview_b (parallel, different models)
Phase 0.5: consolidate_questions
           ↓ HUMAN GATE — answer questions ↓
Phase 1:   plan_a + plan_b (parallel, different models)
Phase 2:   6 cross-reviews (spec/security/TDD × 2 models, each reviews OTHER's plan)
Phase 3:   escalate (surfaces disagreements, recommends best model)
           ↓ HUMAN GATE — resolve disputes ↓
Phase 4:   synthesize (best model writes final spec)

Running a Workflow Manually

Since the CP doesn't yet expand workflows natively, run each phase from a Claude Code session:

1. Render prompts — substitute {{ task_description }}, {{ scope_notes }}, etc. from the template YAML
2. Submit tasks — POST to CP with rendered prompts, correct harness per model
3. Wait — poll with agent-monitor or curl
4. Extract artifacts — read output.md from completed task logs
5. Inject artifacts — replace <<ARTIFACT:node_id:key>> sentinels in next phase's prompts
6. Human gates — present escalation output to user, collect answers, append to artifacts
7. Repeat for each phase

Artifact Passing Between Stages

When a downstream task needs an upstream task's output:

# Extract upstream output
upstream_output = extract_output(CP, upstream_task_id)

# Build downstream prompt with artifact injected
downstream_prompt = f"""
## Plan A (from upstream)
{upstream_output}

## Your Task
Review the above plan for security issues...
"""

Workflow Templates Location

Templates live in ~/dev/claude/projects/agent-runtimes/workflows/:

Template	Description
spec-planning.yaml	Full 14-node spec planning with cross-model review
comparative-plan.yaml	Simpler 5-node comparative planning

---

Agent Repos — Git-Based Persistence

What Is an Agent Repo

An agent repo is a Gitea fork of a project's main repo, named {repo}-agents (e.g., agent-runtimes-agents). Agents work on task-specific branches in the fork. All agent output is auto-committed to git before the container exits, so nothing is lost when ephemeral containers are removed.

Creating an Agent Repo

One-time setup per project. The fork must exist before agents can use it.

# Authenticate with the ai_admin token (see ~/dev/claude/secrets/gitea/ai_admin)
TOKEN="<token from secrets>"

# Check if fork exists (HTTP 200 = yes, 404 = no)
curl -sk -H "Authorization: token $TOKEN" \
  "https://gitea.oreillyit.nz/api/v1/repos/{org}/{repo}-agents"

# Create the fork (HTTP 202 = accepted, HTTP 409 = already exists)
curl -sk -X POST \
  -H "Authorization: token $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"organization":"{org}","name":"{repo}-agents"}' \
  "https://gitea.oreillyit.nz/api/v1/repos/{org}/{repo}/forks"

Naming Convention

Main repo	Agent repo
skynet/agent-runtimes	skynet/agent-runtimes-agents
skynet/brainiac-app	skynet/brainiac-app-agents

Always use the -agents suffix. The fork relationship enables cross-fork PRs.

Workspace Layout

/workspace/
├── reference/              # Owned by root — read-only (agent gets error if they try to write)
│   ├── main/               # Main repo default branch
│   ├── plan-opus-4f3a/     # Another agent's output branch (if needed)
│   └── best-practices/     # Best practices repo (if cloned)
└── working/                # Agent's branch of the agent repo (read-write)
    ├── CLAUDE.md           # Project code from the fork
    ├── spec/
    └── results/            # Starts empty — guaranteed output location
        ├── output.md
        ├── session-log.md
        └── changelog.md

• /workspace/reference/ — each subdirectory is a separate clone. Owned by root so agents get immediate permission errors if they try to modify.
• /workspace/working/ — the agent's branch. All changes auto-committed on exit.
• /workspace/working/results/ — always starts empty. Write outputs here.

Branch Naming

• With workflow context: {workflow_id}-{stage}-{short_task_id} (e.g., f58-plan-4f3a1b2c)
• Without workflow context: task-{short_task_id} (e.g., task-4f3a1b2c)

Auto-Commit (Finalize Phase)

After the agent exits (success or failure), the entrypoint runs finalize scripts:

1. Check for changes in /workspace/working/
2. git add -A && git commit with message: "Agent task {id} ({status}): {prompt_summary}"
3. git push origin {branch}
4. Write metadata to /workspace/.agent-output/ci_metadata.json

Finalize runs unconditionally — partial work from failed agents is preserved. Finalize failure does not override the agent's exit code.

Metadata Propagation

After container exit, the dispatcher reads ci_metadata.json and reports to CP:

{
  "agent_branch": "task-4f3a1b2c",
  "agent_sha": "a1b2c3d4...",
  "agent_repo_url": "git@gitea.oreillyit.nz:skynet/agent-runtimes-agents.git",
  "agent_branch_pushed": true
}

Stored in CPTask.metadata (JSONB dict). Query via GET /tasks/{id}.

Credential Separation

Actor	Credential	Purpose
Dispatcher	GITEA_API_TOKEN (read-only)	Verify fork and branches exist
Agent container	SSH key (via gitea-ssh harness)	Clone repos, push branches

The dispatcher verifies prerequisites but doesn't create forks or branches.

Retry Behaviour

On retry, the agent gets a fresh workspace (clean checkout from base branch). The previous attempt's branch is cloned into /workspace/reference/previous-attempt/ with a note in CLAUDE.md that the last attempt was incomplete and any work it created can be found there.

Cross-Repo PRs

When an agent working in the agent repo needs to create a PR against the main repo, Gitea supports cross-fork PRs natively. This is not automated initially — escalate to a human or handle on demand.

Branch Cleanup

Out of scope for initial implementation. The task- prefix and deterministic naming make automated pruning straightforward when needed.

---

Known Agent Repos

Project	Agent Repo	Created
skynet/agent-runtimes	skynet/agent-runtimes-agents	2026-04-05

---

Quick Reference

# Port-forward to CP
kubectl port-forward -n agent-runtimes svc/controlplane 8100:8100 &

# Submit a task
curl -s -X POST http://localhost:8100/tasks -H "Content-Type: application/json" \
  -d '{"name":"my-task","project_id":"my-project","prompt":"...","harness":"planning-opus-repo/v1","runtime":{"cli":"claude"},"pre_actions":[{"type":"clone","repo":"git@gitea.oreillyit.nz-ai-enablement:skynet/my-project.git"}]}'

# Monitor
~/dev/claude/projects/agent-runtimes/scripts/agent-monitor --filter "project=my-project" --filter "age<20m"

# Check result
curl -s http://localhost:8100/tasks/{id} | python3 -m json.tool

# Cancel
curl -s -X DELETE http://localhost:8100/tasks/{id}