Compare commits
9 Commits
feat/cpw-s
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c67d9736fd | ||
|
|
d0424879c2 | ||
|
|
37a94dcd07 | ||
|
|
44334a2143 | ||
|
|
418d634afa | ||
|
|
d7decc1dec | ||
| c8c9e7f7d0 | |||
|
|
57374c8bfb | ||
|
|
e11b448f00 |
@@ -49,8 +49,12 @@ task-templates/ # dispatch-task --template <name> YAML files
|
||||
workflows/ # Workflow DAG definitions
|
||||
model-registry/ # Model scoring YAMLs (capability dimensions)
|
||||
models/ # Model endpoint YAMLs (URLs, auth, parameters)
|
||||
benchmarks/ # Reference data for the scores/costs pass (NOT read by any loader):
|
||||
# artificial-analysis/ snapshot + README, unverified/estimate files
|
||||
```
|
||||
|
||||
**Model slugs:** a registry slug is not a provider model id. MiniMax silently maps unknown model names to MiniMax-M3, so each `models/<slug>.yaml` must carry the real id in `default_flags.model` (agent-runtimes translates slug -> id on every runner). One `models/` + `model-registry/` file per effort level (e.g. `minimax-m31-flash-{max,medium}`).
|
||||
|
||||
## Harness Conventions
|
||||
|
||||
### harness.yaml fields
|
||||
|
||||
7
CONTEXT.md
Normal file
7
CONTEXT.md
Normal file
@@ -0,0 +1,7 @@
|
||||
# CONTEXT.md — agent-runtime-framework
|
||||
|
||||
## Active
|
||||
|
||||
- **MiniMax Flash migration done; scores/costs are placeholders.** `minimax-m31-flash-{max,medium}` (models/ + model-registry/) and context `minimax-m31-flash/v1` are live on the CP (d7decc1). Registry scores are copies of the M3 entries with costs just below them. Planner FUTURE `629829d3` (agent-runtimes) tracks replacing them and the stop-gap `benchmarks/estimates.csv` M3.1-Flash row (0.9x M3 cost, index 61.5 — implausible). See `benchmarks/artificial-analysis/README.md` for the AA snapshot, slug mapping and caveats.
|
||||
- **Unverified end-to-end:** dispatch a real hatchmate task and check the model the API reports is `MiniMax-M3.1-Flash-Preview` (agent-runtimes 0.0.5.1817 translates slug -> provider id on all runners).
|
||||
- Untracked, left alone: `.claude/` (worktrees), `tests/__init__.py`.
|
||||
35
benchmarks/artificial-analysis/2026-10-13.csv
Normal file
35
benchmarks/artificial-analysis/2026-10-13.csv
Normal file
@@ -0,0 +1,35 @@
|
||||
model,creator,context,intelligence_index,cost_per_task_usd,cost_per_intelligence_point_usd,median_tokens_per_s
|
||||
Claude Opus 5.5 (max with fallback),Anthropic,1M,58,5.98,0.1031,96
|
||||
Claude Sonnet 5.5 (max with fallback),Anthropic,1M,56,5.46,0.0975,135
|
||||
Claude Opus 5.5 (xhigh with fallback),Anthropic,1M,56,3.46,0.0618,81
|
||||
Claude Opus 5.5 (high with fallback),Anthropic,1M,54,1.82,0.0337,82
|
||||
Claude Fable 5.1 (max with fallback),Anthropic,1M,53,7.63,0.1440,67
|
||||
Claude Fable 5.1 (xhigh with fallback),Anthropic,1M,53,5.98,0.1128,62
|
||||
Claude Sonnet 5.5 (xhigh with fallback),Anthropic,1M,52,2.01,0.0387,103
|
||||
Claude Opus 5.5 (medium with fallback),Anthropic,1M,51,1.34,0.0263,79
|
||||
Claude Fable 5.1 (high with fallback),Anthropic,1M,51,3.91,0.0767,52
|
||||
Claude Fable 5.1 (medium with fallback),Anthropic,1M,49,2.98,0.0608,54
|
||||
Claude Fable 5.1 (low with fallback),Anthropic,1M,47,2.37,0.0504,56
|
||||
Claude Sonnet 5.5 (high with fallback),Anthropic,1M,47,0.88,0.0187,96
|
||||
Claude Haiku 5.5 (max),Anthropic,1M,43,0.21,0.0049,236
|
||||
Claude Opus 5.5 (low with fallback),Anthropic,1M,42,0.55,0.0131,80
|
||||
Claude Haiku 5.5 (xhigh),Anthropic,1M,41,0.12,0.0029,187
|
||||
Claude Sonnet 5.5 (medium with fallback),Anthropic,1M,41,0.48,0.0117,95
|
||||
Claude Haiku 5.5 (high),Anthropic,1M,38,0.08,0.0021,175
|
||||
Claude Sonnet 5.5 (low with fallback),Anthropic,1M,36,0.35,0.0097,100
|
||||
Claude Haiku 5.5 (medium),Anthropic,1M,34,0.05,0.0015,171
|
||||
Claude Haiku 5.5 (low),Anthropic,1M,29,0.02,0.0007,164
|
||||
MiniMax-M3,MiniMax,1M,29,0.51,0.0176,107
|
||||
Qwen3.8 Max (0902),Alibaba,984k,45,5.41,0.1202,36
|
||||
Qwen3.8 2.4T A95B,Alibaba,262k,40,2.16,0.0540,37
|
||||
Qwen3.8-Flash-Next,Alibaba,256k,40,0.37,0.0092,51
|
||||
Qwen3.8 27B (xhigh),Alibaba,256k,34,1.01,0.0297,46
|
||||
Qwen3.8 27B (medium),Alibaba,256k,28,1.13,0.0404,50
|
||||
Qwen3.8 27B (low),Alibaba,256k,26,1.05,0.0404,53
|
||||
Qwen3.7 Plus,Alibaba,1M,25,0.22,0.0088,55
|
||||
Qwen3.8 27B (non-reasoning),Alibaba,256k,20,2.49,0.1245,52
|
||||
Qwen3.5 397B A17B,Alibaba,262k,18,0.47,0.0261,88
|
||||
Qwen3.6 35B A3B,Alibaba,262k,18,0.48,0.0267,129
|
||||
Qwen3.5 122B A10B,Alibaba,262k,16,0.32,0.0200,129
|
||||
Qwen3.5 9B,Alibaba,262k,11,0.21,0.0191,58
|
||||
Qwen3 Coder Next,Alibaba,256k,9,0.78,0.0867,73
|
||||
|
28
benchmarks/artificial-analysis/README.md
Normal file
28
benchmarks/artificial-analysis/README.md
Normal file
@@ -0,0 +1,28 @@
|
||||
# Artificial Analysis benchmark snapshot
|
||||
|
||||
Reference data for the model-registry scores/costs pass. NOT read by any loader
|
||||
(RegistryEntry ignores unknown fields, so this lives beside the registry, not in it).
|
||||
|
||||
- **Source:** https://artificialanalysis.ai/leaderboards/models, extracted from the rendered
|
||||
table on 2026-10-13 (Anthropic, Alibaba/Qwen and MiniMax rows only; 34 rows with a score and a cost).
|
||||
- **File:** `2026-10-13.csv` — `intelligence_index` is the AA Intelligence Index; `cost_per_task_usd` is the
|
||||
site's "Cost per Task (USD)".
|
||||
- **`cost_per_intelligence_point_usd`** = `cost_per_task_usd / intelligence_index`. Derived here, not a site
|
||||
column. Chosen as the cost factor to record (user decision, 2026-10-13).
|
||||
- Rows the site marks with `*` (partial results, no cost shown) are omitted.
|
||||
- Names are the site's labels. "with fallback" on most Claude rows is the site's wording; effort level is the
|
||||
bracketed suffix.
|
||||
- Attribution: data © Artificial Analysis. Re-extract rather than editing by hand.
|
||||
|
||||
## Mapping to registry slugs (informational, not authoritative)
|
||||
|
||||
| Registry slug | AA row(s) | Note |
|
||||
|---|---|---|
|
||||
| `minimax-m3`, `minimax-m3-max` | MiniMax-M3 (index 29, $0.51, 0.0176) | AA lists one M3 row with no effort level; both slugs share it |
|
||||
| `minimax-m31-flash-{max,medium}` | none (see `../estimates.csv`) | MiniMax-M3.1-Flash-Preview is NOT on the leaderboard yet (checked 2026-10-13; no model page). **TODO(revisit):** `../estimates.csv` holds a placeholder estimate (cost 0.9x M3, index scaled by the KingBench ratio) — replace with measured values |
|
||||
| `airouter-qwen38`, `airouter-qwen38-max` | Qwen3.8 family | `default_flags.model: "Qwen3.8"` does not say which size; AA has Qwen3.8 27B (low/medium/xhigh), 2.4T A95B, Max (0902), Flash-Next — confirm before mapping |
|
||||
| `claude-fable-5-1` | Claude Fable 5.1 (low..max) | registry sets `reasoning_effort: low` (inert on the claude runner) |
|
||||
| `claude-sonnet-5`, `claude-opus-5`, `claude-haiku-4-5` | Sonnet 5.5, Opus 5.5, Haiku 5.5 | slug/model-id names predate the 5.5 releases (`default_flags.model` is `claude-sonnet-5` etc.); verify which API model each actually calls before using these rows |
|
||||
|
||||
Cost per task is not comparable to `cost_per_1k_tokens`: it folds in how many tokens the model spends. Use it
|
||||
for `cost_efficiency` scoring, not for the per-token price fields.
|
||||
2
benchmarks/estimates.csv
Normal file
2
benchmarks/estimates.csv
Normal file
@@ -0,0 +1,2 @@
|
||||
model,creator,context,intelligence_index,cost_per_task_usd,cost_per_intelligence_point_usd,status,basis
|
||||
MiniMax-M3.1-Flash-Preview (effort unspecified),MiniMax,1M,61.5,0.46,0.0075,ESTIMATE-TODO,"cost = 0.9 x MiniMax-M3 AA cost per task ($0.51); index = M3 AA index (29) x KingBench-3 ratio (66.25/31.25 = 2.12); both inputs unverified/placeholder"
|
||||
|
23
benchmarks/minimax-m3-1-flash-unverified.md
Normal file
23
benchmarks/minimax-m3-1-flash-unverified.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# MiniMax-M3.1-Flash-Preview — ad-hoc, UNVERIFIED figures
|
||||
|
||||
Supplied by the operator on 2026-10-13 with no source URL. Not from Artificial Analysis (the model is not on
|
||||
that leaderboard yet — see `artificial-analysis/README.md`). Do not use for registry scores until confirmed.
|
||||
|
||||
| Item | Figure | Note |
|
||||
|---|---|---|
|
||||
| KingBench 3, M3.1 Flash | 66.25% (53/80) | "community and third-party testing"; arithmetic checks (53/80 = 66.25%) |
|
||||
| KingBench 3, M3 baseline | 31.25% (25/80) | arithmetic checks; the source also mentions "alternative baseline figures depending on the evaluation set" |
|
||||
| M3 SWE-bench Pro | ~59% | reference baseline for M3, described as official |
|
||||
| M3 "Browse.com" | 83.5 | benchmark name looks mistyped (BrowseComp?) — confirm before citing |
|
||||
| M3 Terminal Bench 2.1 | "strong" | no number given |
|
||||
|
||||
Caveats: unknown effort level for the KingBench runs (Flash always thinks; server default is max); KingBench 3 is
|
||||
small (80 items) so one item is 1.25 points; none of these figures share a scale with the AA Intelligence Index
|
||||
(M3 = 29 there), so they cannot be mixed into one cost-per-intelligence figure.
|
||||
|
||||
## TODO(revisit) — placeholder estimate in use (2026-10-13)
|
||||
`estimates.csv` carries a stop-gap row for M3.1 Flash: cost per task = 90% of M3's ($0.51 -> $0.46) and
|
||||
intelligence index = M3's AA index (29) scaled by the KingBench-3 ratio (66.25/31.25 = 2.12) = 61.5. That index
|
||||
would rank it above Claude Opus 5.5 max (58), which is implausible for a Flash-class model — treat both numbers as
|
||||
rough placeholders. Replace with (a) the Artificial Analysis row when the model is listed, or (b) our own measured
|
||||
cost per task at each effort level (fixed task set via `ask-minimax-run --output-format json`).
|
||||
@@ -6,6 +6,6 @@ description: "Best practices review agent on MiniMax M3 (max effort) + repo clon
|
||||
layers:
|
||||
- context: best-practices/v1
|
||||
- context: minimax/v1
|
||||
- context: minimax-m3-max/v1
|
||||
- context: minimax-m31-flash/v1
|
||||
- context: gitea-ssh/v1
|
||||
- context: agent-repo/v1
|
||||
|
||||
@@ -9,7 +9,7 @@ layers:
|
||||
# personal-subscription OAuth — integration runs are frequent/mechanical
|
||||
# and were draining the interactive 5-hour window (2026-09-02).
|
||||
- context: minimax/v1
|
||||
- context: minimax-m3-max/v1
|
||||
- context: minimax-m31-flash/v1
|
||||
- context: gitea-ssh/v1
|
||||
- context: agent-repo/v1
|
||||
- context: direct-push/v1
|
||||
|
||||
@@ -8,8 +8,3 @@ layers:
|
||||
- context: anthropic-cloud-paul-oauth/v1
|
||||
- context: gitea-ssh/v1
|
||||
- context: agent-repo/v1
|
||||
# AU-51/AU-51b: this composite backs bug-verify-sonnet@1, whose agent node
|
||||
# self-transitions its own item's flow_state (PATCH /v1/work-items/{id})
|
||||
# as its final step — see work item 88ecd23e ("agent self-transition"
|
||||
# pattern, spec/typed-workflows.md). Needs a CP write credential.
|
||||
- context: cp-service-token/v1
|
||||
|
||||
@@ -8,6 +8,6 @@ layers:
|
||||
# MiniMax M3 — scaffolding is mechanical (stub files so tests import);
|
||||
# moved off the personal Anthropic subscription 2026-09-02.
|
||||
- context: minimax/v1
|
||||
- context: minimax-m3-max/v1
|
||||
- context: minimax-m31-flash/v1
|
||||
- context: gitea-ssh/v1
|
||||
- context: agent-repo/v1
|
||||
|
||||
@@ -8,6 +8,3 @@ layers:
|
||||
- context: anthropic-cloud-paul-oauth/v1
|
||||
- context: gitea-ssh/v1
|
||||
- context: agent-repo/v1
|
||||
# AU-51/AU-51b: this composite backs scope-decompose-sonnet@1, which
|
||||
# calls back POST /v1/work-items/{id}/decompose — needs a credential.
|
||||
- context: cp-service-token/v1
|
||||
|
||||
@@ -9,6 +9,6 @@ layers:
|
||||
# 2026-09-02 (the "credential layer swappable" swap this composite was
|
||||
# designed for).
|
||||
- context: minimax/v1
|
||||
- context: minimax-m3-max/v1
|
||||
- context: minimax-m31-flash/v1
|
||||
- context: gitea-ssh/v1
|
||||
- context: agent-repo/v1
|
||||
|
||||
@@ -0,0 +1,14 @@
|
||||
kind: composite
|
||||
name: workflow-script-nocp-norepo
|
||||
version: 1
|
||||
description: "Internal composite for kind: script nodes with cp_write=false and NO effective agent_repo (typed-workflows WF-COMP-31e, planner item e993c9fa). Brings only the shared-variables filesystem context: no repo-readonly clone, no CP credential, and therefore ZERO secrets_required entries. That makes the node eligible for an M45 tenant namespace (the NS-12 gate rejects any ESO account secret) and for the M33 egress broker sidecar, which supplies the node's credentials through bindings. Not model-bearing; the compiled shape is cli=\"shell\", params.cmd=<cmd>."
|
||||
|
||||
# Counterpart of workflow-script-nocp/v1 for templates that declare no agent_repo.
|
||||
# The compiler (controlplane/script_composites.py resolve_script_composite) selects
|
||||
# this only when effective_repo is False; with AGENT_REPO_URL unset the entrypoint
|
||||
# V2 guard does not require /workspace/project. Never layer repo-readonly/v1 here:
|
||||
# its gitea-ssh-readonly secret is exactly what this composite exists to avoid.
|
||||
layers:
|
||||
# spec/workflow-variables.md WV-19 / spec/harness.md HD-14 — provides the
|
||||
# [shared-variables] tag.
|
||||
- context: shared-variables/v1
|
||||
@@ -1,35 +0,0 @@
|
||||
kind: composite
|
||||
name: workflow-script
|
||||
version: 1
|
||||
description: "Internal composite for the flag-on kind: script path (typed-workflows WF-COMP-31 / workflow-variables WV-19). Brings the shared-variables filesystem context AND the AU-51 CP write credential (cp-service-token/v1 sets CP_SERVICE_TOKEN_FILE in the entrypoint process env before shell_runner runs). NOT a model-bearing composite — the kind: script compiled shape is cli=\"shell\", params.cmd=<cmd>; this composite provides the harness layers that path needs and nothing else."
|
||||
|
||||
# Order matters only for HC-6 env last-writer-wins / HC-8 git identity. The two
|
||||
# layers here have disjoint env vars (shared-variables sets WORKFLOW_VARIABLES_*
|
||||
# paths, cp-service-token sets CP_SERVICE_TOKEN_FILE) and no overlapping git
|
||||
# identity, so the listed order has no observable merge effect — both layers
|
||||
# reach the script-node container either way.
|
||||
layers:
|
||||
# spec/workflow-variables.md WV-19 / spec/harness.md HD-14 — agent-facing
|
||||
# /variables/ context (CLAUDE.md mounted at /opt/harness/context/shared-variables/CLAUDE.md)
|
||||
# plus WORKFLOW_VARIABLES_ROOT=/variables and templated WORKFLOW_VARIABLES_SELF=<step_path>.
|
||||
# Provides the [shared-variables] tag used by the dispatcher's extra_layers injection.
|
||||
- context: shared-variables/v1
|
||||
# spec/typed-workflows.md WF-COMP-31 "AU-51 credential env seam" — this layer's
|
||||
# init.sh restages the ESO-mounted CP-service-token secret (root-owned mode 0400)
|
||||
# to an agent-owned 0600 copy at /run/agent/cp-service-token/token and exports
|
||||
# CP_SERVICE_TOKEN_FILE pointing at it. Verified: env.CP_SERVICE_TOKEN_FILE is set
|
||||
# on the entrypoint process environment before shell_runner runs (entrypoint reads
|
||||
# os.environ via env=os.environ.copy() in subprocess.run at entrypoint/runners/shell.py:73,
|
||||
# so the same credential channel an agent node gets reaches a script node unchanged).
|
||||
- context: cp-service-token/v1
|
||||
# spec/typed-workflows.md WF-COMP-34 (bug 3bee78fa, 2026-10-02) — read-only
|
||||
# project-repo clone for harness-path kind: script nodes. Layered LAST so
|
||||
# its clone runs after the two AU-51 wiring layers and before the script
|
||||
# starts (init.sh order: shared-variables -> cp-service-token -> repo-readonly).
|
||||
# When AGENT_REPO_URL is unset (the pre-fix runtime, or any template that
|
||||
# does not declare agent_repo) this layer is a no-op; the V2 guard
|
||||
# (entrypoint/entrypoint.py:901-906) does NOT trip because the unset case
|
||||
# is a no-op there too. Layered-in instead of layering agent-repo/v1
|
||||
# because that context's fork management and push side effects are wrong
|
||||
# for a read-only script node (see bug item body).
|
||||
- context: repo-readonly/v1
|
||||
@@ -65,6 +65,41 @@ if [ -n "${AGENT_REPO_URL:-}" ] && [ -n "${AGENT_BRANCH:-}" ]; then
|
||||
echo "Cloning agent working repo: $AGENT_REPO_URL (branch: $AGENT_BRANCH)"
|
||||
if git clone --depth 1 --branch "$AGENT_BRANCH" "$AGENT_REPO_URL" /workspace/project 2>/dev/null; then
|
||||
echo "Cloned existing branch $AGENT_BRANCH (continuing prior work)"
|
||||
# AR-42: archive the stale task branch and reset it to the current base.
|
||||
# Opt-in per node (AGENT_BRANCH_RESET_ON_START=true, set by the workflow
|
||||
# template on its FIRST node only): every node of a workflow shares this
|
||||
# one branch, so an unconditional reset would wipe earlier nodes' commits.
|
||||
if [ "${AGENT_BRANCH_RESET_ON_START:-}" = "true" ]; then
|
||||
mkdir -p /workspace/.agent-output
|
||||
AR42_TOOL=/opt/agent/agent_repo_base.py
|
||||
AR42_FAIL() {
|
||||
echo "ERROR: AR-42 $1" >&2
|
||||
printf '{"error": "ar42_archive_and_reset_failed", "detail": %s}\n' \
|
||||
"$(printf '%s' "$1" | python3 -c 'import json,sys; print(json.dumps(sys.stdin.read()[:500]))')" \
|
||||
> /workspace/.agent-output/init-error.json
|
||||
exit 1
|
||||
}
|
||||
AR42_SEL=$(python3 "$AR42_TOOL" select-base \
|
||||
--workdir /workspace/project \
|
||||
--branch "$AGENT_BRANCH" \
|
||||
--upstream-url "${AGENT_UPSTREAM_REPO_URL:-}" \
|
||||
--upstream-branch "${AGENT_UPSTREAM_BRANCH:-main}" \
|
||||
--reference-dir /workspace/reference/main \
|
||||
--out /workspace/.agent-output/branch_base.json 2>&1) \
|
||||
|| AR42_FAIL "select-base failed: $AR42_SEL"
|
||||
AR42_SHA=$(printf '%s\n' "$AR42_SEL" | sed -n 's/^sha=//p' | head -1)
|
||||
[ -n "$AR42_SHA" ] || AR42_FAIL "select-base returned no sha: $AR42_SEL"
|
||||
AR42_OUT=$(python3 "$AR42_TOOL" archive-and-reset \
|
||||
--workdir /workspace/project \
|
||||
--branch "$AGENT_BRANCH" \
|
||||
--base-sha "$AR42_SHA" \
|
||||
--fork-url "$AGENT_REPO_URL" \
|
||||
--archive-prefix archive \
|
||||
--base-metadata /workspace/.agent-output/branch_base.json \
|
||||
--out /workspace/.agent-output/branch_base.json 2>&1) \
|
||||
|| AR42_FAIL "archive-and-reset failed: $AR42_OUT"
|
||||
echo "AR-42: $AGENT_BRANCH reset to base $AR42_SHA"
|
||||
fi
|
||||
else
|
||||
echo "Branch $AGENT_BRANCH does not exist — seeding fresh branch from upstream reference"
|
||||
# Full clone (not --depth 1) so we get a working remote for finalize.sh push.
|
||||
|
||||
48
harnesses/contexts/cp-docs-token/v1/CLAUDE.md
Normal file
48
harnesses/contexts/cp-docs-token/v1/CLAUDE.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# Step documents (cp-docs-token)
|
||||
|
||||
This step can read and write planner documents through `python3 -m lib.cp_cli`
|
||||
in step-docs mode (PLN-CLI-16). The mode is active because `CP_DOCS_TOKEN_FILE`
|
||||
and `CP_DOCS_CA_FILE` are set and the listener URL is in `CP_DOCS_URL`.
|
||||
|
||||
## Credential
|
||||
|
||||
- The token is a file at `$CP_DOCS_TOKEN_FILE`. cp-cli reads it on every call.
|
||||
- Never print, copy, log or pass the token on a command line.
|
||||
- Do not use `~/.cpconfig`; step-docs mode does not read it.
|
||||
|
||||
## Allowed commands
|
||||
|
||||
Only these commands work; anything else exits 2 `cp_docs_mode_command_not_allowed`:
|
||||
|
||||
```
|
||||
python3 -m lib.cp_cli planner items documents list <item-uuid>
|
||||
python3 -m lib.cp_cli planner items documents get <item-uuid> <name> [--json]
|
||||
python3 -m lib.cp_cli planner items documents put <item-uuid> <name> --file <path> (--create-only | --if-match <rev>)
|
||||
python3 -m lib.cp_cli planner items documents related [--json]
|
||||
```
|
||||
|
||||
Item arguments are full UUIDs. Do not pass a project id.
|
||||
|
||||
## Scope
|
||||
|
||||
- You may write only documents on this step's own item.
|
||||
- You may read documents on this step's item, its parent and its siblings.
|
||||
- `related` prints `item_uuid`, `parent_uuid`, `sibling_uuids` and `siblings_truncated`.
|
||||
|
||||
## Writes need a precondition
|
||||
|
||||
Exactly one of these is required on every `put` (neither or both exits 2):
|
||||
|
||||
- `--create-only` creates a new document (sends `If-None-Match: *`).
|
||||
- `--if-match <rev>` updates an existing one (sends `If-Match`). Take `<rev>`
|
||||
from `get --json` (the ETag revision); surrounding quotes are stripped.
|
||||
|
||||
## Errors
|
||||
|
||||
- 412 `precondition_failed`: the document changed or already exists. Re-run
|
||||
`get --json`, merge your change into the current content, and `put` again
|
||||
with the new revision.
|
||||
- 428 (precondition missing): add `--create-only` or `--if-match`.
|
||||
- 429 `step_docs_write_budget_exhausted` or `step_docs_rate_limited`: stop
|
||||
writing; do not retry in a loop.
|
||||
- 404 `not_found`, 400 `invalid_document_name`: fix the item uuid or name.
|
||||
16
harnesses/contexts/cp-docs-token/v1/harness.yaml
Normal file
16
harnesses/contexts/cp-docs-token/v1/harness.yaml
Normal file
@@ -0,0 +1,16 @@
|
||||
kind: context
|
||||
name: cp-docs-token
|
||||
version: 1
|
||||
description: "Step-docs mode env contract for lib.cp_cli (PLN-CLI-16): per-step wfcd_ token file and CA file delivered by the dispatcher; document access scoped to this step's item (spec/dispatcher.md D-40, spec/authentication.md AU-59)"
|
||||
requires: []
|
||||
provides: [cp-docs-token]
|
||||
|
||||
# Env-only layer: the dispatcher delivers the token and CA files into the
|
||||
# container; this layer sets only the paths. No secrets_required, no init script.
|
||||
env:
|
||||
CP_DOCS_TOKEN_FILE: /run/agent/cp-docs/token
|
||||
CP_DOCS_CA_FILE: /run/agent/cp-docs/ca.crt
|
||||
|
||||
context_files:
|
||||
- source: ./CLAUDE.md
|
||||
target: /opt/harness/context/cp-docs-token/CLAUDE.md
|
||||
@@ -1,30 +0,0 @@
|
||||
kind: context
|
||||
name: cp-service-token
|
||||
version: 1
|
||||
description: "CP-internal service bearer token — authenticates agent-container write calls back to the control plane (AU-51/AU-51b, bug b3a96acc)"
|
||||
requires: []
|
||||
provides: []
|
||||
|
||||
# AU-51b: the ESO-mounted secret is root-owned mode 0400 (init.sh runs as
|
||||
# root, before the gosu drop to the agent user, uid 1000) — unreadable by
|
||||
# the agent process directly. Confirmed live 2026-09-03: a real agent
|
||||
# session hit exactly this ("cp-cli invocation can't proceed... token is
|
||||
# root-owned mode 0400 so it's not readable by the agent user"). Mirrors
|
||||
# anthropic-cloud-paul-oauth/v1's pattern: init.sh restages a 0600
|
||||
# agent-owned copy outside the ESO mount. CP_SERVICE_TOKEN_FILE points at
|
||||
# the restaged copy, not the raw mount.
|
||||
#
|
||||
# The value at this path MUST match the CP's CP_INTERNAL_BEARER_TOKEN
|
||||
# (same account: cp-decompose-service-token, account_id
|
||||
# 1d963673-6ac9-4f85-875a-2ce5323e76ad, owner (cp, cp)).
|
||||
env:
|
||||
CP_SERVICE_TOKEN_FILE: /run/agent/cp-service-token/token
|
||||
|
||||
scripts:
|
||||
init: "./init.sh"
|
||||
|
||||
secrets_required:
|
||||
- name: cp-service-token
|
||||
account_ref: "1d963673-6ac9-4f85-875a-2ce5323e76ad"
|
||||
mount_path: /run/agent/secrets/cp-service-token
|
||||
mode: "0400"
|
||||
@@ -1,39 +0,0 @@
|
||||
#!/bin/bash
|
||||
# cp-service-token init — restage the ESO-mounted secret to an agent-owned
|
||||
# copy.
|
||||
#
|
||||
# Threat model: keep the ESO mount root-only (mode 0400) so the agent user
|
||||
# cannot directly `cat` /run/agent/secrets/cp-service-token/*. init.sh runs
|
||||
# as root (in uid-wrapper.sh, before the gosu drop) and writes a restaged
|
||||
# copy at /run/agent/cp-service-token/token with mode 0600 owned by the
|
||||
# agent user. This mirrors anthropic-cloud-paul-oauth/v1/init.sh's pattern.
|
||||
#
|
||||
# Bug discovered live 2026-09-03: without this restaging step, cp-cli
|
||||
# (invoked by the agent user, uid 1000) could not read the raw 0400
|
||||
# root-owned mount at all — "cp_cli invocation can't proceed... token is
|
||||
# root-owned mode 0400 so it's not readable by the agent user". The context
|
||||
# originally set CP_SERVICE_TOKEN_FILE directly at the raw ESO mount path,
|
||||
# which is the mistake this init.sh fixes.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
ESO_TOKEN="/run/agent/secrets/cp-service-token/value"
|
||||
|
||||
if [ ! -r "$ESO_TOKEN" ]; then
|
||||
echo "ERROR: $ESO_TOKEN not readable. Check ESO ExternalSecret acct-1d963673." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
AGENT_USER="${AGENT_USER:-agent}"
|
||||
|
||||
DEST_DIR="/run/agent/cp-service-token"
|
||||
mkdir -p "$DEST_DIR"
|
||||
|
||||
DEST_FILE="$DEST_DIR/token"
|
||||
cp "$ESO_TOKEN" "$DEST_FILE"
|
||||
|
||||
chown "$AGENT_USER:" "$DEST_DIR" "$DEST_FILE" 2>/dev/null || true
|
||||
chmod 0700 "$DEST_DIR"
|
||||
chmod 0600 "$DEST_FILE"
|
||||
|
||||
echo "cp-service-token: restaged into $DEST_FILE (0600 $AGENT_USER:$AGENT_USER)"
|
||||
17
harnesses/contexts/minimax-m31-flash/v1/harness.yaml
Normal file
17
harnesses/contexts/minimax-m31-flash/v1/harness.yaml
Normal file
@@ -0,0 +1,17 @@
|
||||
kind: context
|
||||
name: minimax-m31-flash
|
||||
version: 1
|
||||
description: "Pin MiniMax-M3.1-Flash-Preview (1M context) — layer AFTER minimax/v1 (auth + base URL come from there). Thinking depth is set per dispatch via --effort (models/*.yaml default_flags.reasoning_effort), not here."
|
||||
requires: []
|
||||
provides: []
|
||||
|
||||
# Replaces minimax-m3-max/v1 (kept for rollback). Differences:
|
||||
# - Flash always thinks and ignores MAX_THINKING_TOKENS/budget_tokens (effort is
|
||||
# the only depth control, server default max), so there is no init.sh.
|
||||
# - No [1m] suffix: Flash is 1M natively (verified 2026-10-13, modelUsage.contextWindow).
|
||||
# - CLAUDE_CODE_AUTO_COMPACT_WINDOW stays: an unknown model id otherwise gets
|
||||
# Claude Code's 200K default and compacts early.
|
||||
env:
|
||||
ANTHROPIC_MODEL: "MiniMax-M3.1-Flash-Preview"
|
||||
ANTHROPIC_SMALL_FAST_MODEL: "MiniMax-M3.1-Flash-Preview"
|
||||
CLAUDE_CODE_AUTO_COMPACT_WINDOW: "1000000"
|
||||
@@ -1,27 +0,0 @@
|
||||
# Session Log -- 2026-05-04
|
||||
|
||||
## Summary
|
||||
Implemented 5 planned fixes for BUG-5 (`.agent-output/` silent drops) and BUG-20 (MiniMax finalize no-push): pre_test.sh harness hook, output validation in finalize, push retry with debug tracing, spec-planning workflow output tags, and wrong-path detection with agent correction re-invoke. Fork cleanup PR merged, agent-monitor PR #44 merged.
|
||||
|
||||
## Decisions
|
||||
- Fork cleanup workflow (PR #1 on agent-runtimes-agents): weekly cron deletes `task-*` branches inactive for 7+ days; main reset to orphan commit if no commits for 7+ days. Supports dry-run via workflow_dispatch. **Merged** (2026-05-04 10:39 UTC by admin).
|
||||
- Gitea API token auth requires HTTP Basic Auth (`auth=("user","pass")`) not Bearer token — Bearer returns 401 even with valid access token. Use `httpx.get(..., auth=("user","pass"))` pattern.
|
||||
- Shallow clone (`--depth 1`) for task branches in fork cleanup to avoid bloating with full history.
|
||||
|
||||
## Gotchas Discovered
|
||||
- **[Gitea Gotchas]** Gitea API endpoint `/login` is `404 Not Found` — use `/users/<username>/access_tokens` to create tokens, or use HTTP Basic Auth directly against any `/api/v1/` endpoint with `auth=("user","pass")`.
|
||||
- **[Gitea Gotchas]** PR state `closed` with `merged: true` means the PR was merged (not just closed). Gitea distinguishes merged vs closed states.
|
||||
- **[Gitea Gotchas]** Fork PRs use the parent repo's Actions (has_actions=true on parent, `has_actions: false` on fork). Fork cleanup workflow runs on the fork's GHA because fork enables Actions independently.
|
||||
- **[tokens]** Token file at `~/.config/agent-runtimes/tokens.json` uses flat structure: `{"access_token": "...", "expires_at": N}`. Token was valid (not expired at session continuation time: now 1777891716 vs expiry 1777891804).
|
||||
|
||||
## Key Context
|
||||
- **agent-runtime-framework** `8bbf6cb`: pre_test.sh created (harness-level test folder revert), finalize.sh gains `set -x` tracing, push retry (`AGENT_PUSH_RETRIES`), output validation (`AGENT_EXPECTED_OUTPUT`), wrong-path detection + `.correction-prompt.txt` sentinel for entrypoint re-invoke. All 17 spec-planning workflow nodes tagged with `output: {path, min_bytes}`.
|
||||
- **agent-runtimes-agents** fork: `fork-cleanup` branch merged to main (PR #1 by ai_enablement, merged by admin 2026-05-04). Workflow file at `.gitea/workflows/cleanup.yaml`.
|
||||
- **agent-runtimes** `148d9d3`: PR #44 merged — `scripts/agent-monitor` gains `provider` and `model_full` columns.
|
||||
- **agent-runtimes** local uncommitted: M22 Phase 3 manifests work (`controlplane/api/manifests.py`, `controlplane/db/manifest_store.py`, migration `20260504_0017_m16_phase3_manifests.py`, `controlplane/manifests/`). Not related to this session's focus.
|
||||
- Two ops gates pending before M22 Phase 9 live cutover: CI build `agent-runtimes-init:1.0.0` image; `agent-session` Role deployed via `homelab/agent-runtimes-deploy` + ArgoCD sync.
|
||||
|
||||
## Process Notes
|
||||
- When verifying PR state via Gitea API: parse JSON directly (don't assume specific keys — use `json.dumps(..., indent=2)` to inspect full structure).
|
||||
- Fork cleanup workflow commits to `agent-runtimes-agents` fork main; parent `agent-runtimes` main unchanged. Fork's main tracked via `agent-runtimes-agents` remote in local `agent-runtime-framework` checkout.
|
||||
- Fork cleanup's `fetch-depth: 0` is needed for commit date inspection even with shallow clones elsewhere.
|
||||
32
memory/log/2026-10-13.144722.md
Normal file
32
memory/log/2026-10-13.144722.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# Session Log -- 2026-10-13
|
||||
|
||||
## Summary
|
||||
Moved the framework's MiniMax content to MiniMax-M3.1-Flash-Preview (new context, Flash models/registry, bare `minimax`/`minimax-m2` slugs removed; d7decc1), then added a `benchmarks/` directory with an Artificial Analysis snapshot, unverified M3.1 Flash figures, and a flagged placeholder estimate (418d634). Both pushed to `main`.
|
||||
|
||||
## Decisions
|
||||
- Decision: new context `harnesses/contexts/minimax-m31-flash/v1` (env pin `MiniMax-M3.1-Flash-Preview`, no init.sh) replaces `minimax-m3-max/v1` in scaffolding-repo, integration-direct, test-writing-repo, best-practices-minimax-repo; the old context stays for rollback -- Rationale: Flash ignores MAX_THINKING_TOKENS; effort comes from `--effort` via `default_flags.reasoning_effort`.
|
||||
- Decision: deleted `model-registry/minimax.yaml` (endpoint `minimax` never existed) and repointed the six `task-templates/minimax-*.yaml` and the three `*-review-loop@1` `model: minimax-m2` nodes to `minimax-m31-flash-max` -- Rationale: MiniMax silently maps unknown model names to M3.
|
||||
- Decision: benchmark data lives in `benchmarks/` beside the registry, not inside registry entries -- Rationale: `RegistryEntry` (pydantic default extra=ignore) silently drops unknown fields, and the loader never reads this dir.
|
||||
- Decision: cost factor recorded is "cost per intelligence point" = AA cost per task / AA Intelligence Index (derived; the site has no such column) -- Rationale: operator request.
|
||||
- Decision: M3.1 Flash placeholder = 0.9x M3 cost per task ($0.46) and M3's index scaled by the KingBench-3 ratio (29 x 2.12 = 61.5), flagged `ESTIMATE-TODO` -- Rationale: operator stop-gap; 61.5 exceeds Opus 5.5 max (58), so implausible. Planner FUTURE item 629829d3 (agent-runtimes) tracks replacing it.
|
||||
|
||||
## Gotchas Discovered
|
||||
- **[minimax]** Symptom: unknown model names (`minimax`, `minimax-m31-flash-max`) return HTTP 200 with `model: MiniMax-M3` -- Fix: send the provider id; translation now done by agent-runtimes ef822703 (deployed 0.0.5.1817). Check the echoed `model`.
|
||||
- **[artificial-analysis]** Symptom: WebFetch of artificialanalysis.ai/models returns only headline numbers (tables are JS-rendered) -- Fix: open the page in Chrome (claude-in-chrome), wait ~4 s, read `document.querySelector('table')` rows via javascript_tool (269 rows on /leaderboards/models; columns: model, context, creator, index, cost per task, tokens/s, latency).
|
||||
- **[artificial-analysis]** MiniMax-M3.1-Flash-Preview is not on the leaderboard (no row; /models/minimax-m3-1-* 404 as of 2026-10-13); only `MiniMax-M3` (index 29, $0.51/task) exists for MiniMax.
|
||||
- **[benchmarks]** Cost vs intelligence correlates strongly only within one model family across effort levels (Claude: r 0.93-0.99, cost x4-6 per +10 index points); across vendors it is weak (r 0.53) and Qwen3.8 27B is inverted (cost does not fall with lower effort), so cross-vendor cost estimates for M3.1 Flash are unreliable.
|
||||
- **[git]** The framework checkout was on the user's feature branch `feat/hatchmate-per-project-fork-key`; moving my uncommitted work to `main` needed `git stash push -- <paths>` (untracked new files stay in place), `checkout main`, ff-pull, `stash pop`.
|
||||
|
||||
## Open Questions
|
||||
- Which Qwen3.8 size do `airouter-qwen38*` models actually call? (AA has 27B, 2.4T A95B, Max, Flash-Next.)
|
||||
- Registry model ids `claude-sonnet-5` / `claude-opus-5` / `claude-haiku-4-5` predate the 5.5 releases AA lists -- verify before mapping AA rows to them.
|
||||
- Source for the KingBench 3 figures; scoring rule to map AA index / cost per intelligence point onto the registry's 0-10 dimensions.
|
||||
|
||||
## Key Context
|
||||
- Files: `benchmarks/artificial-analysis/{2026-10-13.csv,README.md}` (34 rows: Anthropic, Qwen, MiniMax-M3), `benchmarks/minimax-m3-1-flash-unverified.md`, `benchmarks/estimates.csv`.
|
||||
- Live: CRS checkout for this repo is at d7decc1 on the controlplane (verified); 418d634 adds only benchmarks (nothing reads it).
|
||||
- Registry scores/costs for `minimax-m31-flash-{max,medium}` are still placeholders (copies of the M3 entries; costs just under them).
|
||||
|
||||
## Process Notes
|
||||
- Before rebasing onto a feature-branch checkout, check `git branch --show-current`; land work on `main` explicitly.
|
||||
- Record derived metrics with their derivation (README) and mark stop-gap numbers with a TODO in the data file name/status column AND a durable planner item.
|
||||
@@ -5,5 +5,6 @@ roles:
|
||||
claude-sonnet-5: code-sonnet-tdd-repo
|
||||
claude-opus-5: code-opus-tdd-repo
|
||||
claude-haiku-4-5: code-haiku-tdd-repo
|
||||
minimax: code-minimax-tdd-repo
|
||||
minimax-m31-flash-max: code-minimax-tdd-repo
|
||||
minimax-m31-flash-medium: code-minimax-tdd-repo
|
||||
claude-fable-5: code-fable-tdd-repo
|
||||
|
||||
21
model-registry/minimax-m31-flash-max.yaml
Normal file
21
model-registry/minimax-m31-flash-max.yaml
Normal file
@@ -0,0 +1,21 @@
|
||||
# agent-runtime-framework — model-registry/ (CRS tag: model-registry)
|
||||
# Max-effort Flash; replaces minimax-m3-max in planning/draft-heavy and explicit-model nodes.
|
||||
# PLACEHOLDER scores/costs: scores are copied from the M3 entry this replaces and
|
||||
# costs are set just below it so cheapest-wins (tiers.yaml) selects Flash over
|
||||
# the retained M3 entries. Revisit with observed scores and published pricing.
|
||||
name: minimax-m31-flash-max
|
||||
endpoint: minimax-m31-flash-max
|
||||
provider: minimax
|
||||
family: minimax
|
||||
scores:
|
||||
complexity: 10
|
||||
test_pass_rate: 9
|
||||
creativity: 10
|
||||
spec_adherence: 9
|
||||
cost_efficiency: 5
|
||||
context_utilisation: 10
|
||||
cost_per_1k_tokens:
|
||||
input: 0.0025
|
||||
output: 0.0125
|
||||
source: benchmark
|
||||
harness_version: null
|
||||
21
model-registry/minimax-m31-flash-medium.yaml
Normal file
21
model-registry/minimax-m31-flash-medium.yaml
Normal file
@@ -0,0 +1,21 @@
|
||||
# agent-runtime-framework — model-registry/ (CRS tag: model-registry)
|
||||
# Medium-effort Flash; replaces minimax-m3 in the build/integrate/spec-test/coding tiers.
|
||||
# PLACEHOLDER scores/costs: scores are copied from the M3 entry this replaces and
|
||||
# costs are set just below it so cheapest-wins (tiers.yaml) selects Flash over
|
||||
# the retained M3 entries. Revisit with observed scores and published pricing.
|
||||
name: minimax-m31-flash-medium
|
||||
endpoint: minimax-m31-flash-medium
|
||||
provider: minimax
|
||||
family: minimax
|
||||
scores:
|
||||
complexity: 8
|
||||
test_pass_rate: 9
|
||||
creativity: 8
|
||||
spec_adherence: 9
|
||||
cost_efficiency: 6
|
||||
context_utilisation: 9
|
||||
cost_per_1k_tokens:
|
||||
input: 0.00125
|
||||
output: 0.00625
|
||||
source: benchmark
|
||||
harness_version: null
|
||||
@@ -1,15 +0,0 @@
|
||||
name: minimax
|
||||
endpoint: minimax
|
||||
provider: minimax
|
||||
scores:
|
||||
complexity: 10
|
||||
test_pass_rate: 9
|
||||
creativity: 10
|
||||
spec_adherence: 9
|
||||
cost_efficiency: 3
|
||||
context_utilisation: 10
|
||||
cost_per_1k_tokens:
|
||||
input: 0.0
|
||||
output: 0.0
|
||||
source: benchmark
|
||||
harness_version: null
|
||||
@@ -22,7 +22,7 @@ tiers:
|
||||
context_utilisation: 9
|
||||
qualifies_today:
|
||||
- deepseek-v4flash # benchmark-inflated 9s; hold to allowed_models ceiling
|
||||
- minimax
|
||||
- minimax-m31-flash-max
|
||||
- claude-opus-5
|
||||
note: >
|
||||
WF-SCH-14 allowed_models ceiling remains authoritative until M7 observed-score
|
||||
@@ -37,7 +37,8 @@ tiers:
|
||||
spec_adherence: 9
|
||||
test_pass_rate: 9
|
||||
qualifies_today:
|
||||
- minimax
|
||||
- minimax-m31-flash-max
|
||||
- minimax-m31-flash-medium
|
||||
- claude-sonnet-5
|
||||
- claude-opus-5
|
||||
|
||||
@@ -51,7 +52,8 @@ tiers:
|
||||
qualifies_today:
|
||||
- airouter-qwen3
|
||||
- airouter-deepseekv4flash
|
||||
- minimax
|
||||
- minimax-m31-flash-max
|
||||
- minimax-m31-flash-medium
|
||||
- claude-sonnet-5
|
||||
- claude-opus-5
|
||||
- claude-fable-5
|
||||
|
||||
21
models/minimax-m31-flash-max.yaml
Normal file
21
models/minimax-m31-flash-max.yaml
Normal file
@@ -0,0 +1,21 @@
|
||||
# agent-runtime-framework — models/ (CRS tag: models)
|
||||
# MiniMax-M3.1-Flash-Preview at reasoning_effort max. Always thinks (thinking
|
||||
# cannot be disabled; effort is the only depth control). The claude runner maps
|
||||
# default_flags.reasoning_effort to `--effort` (agent-runtimes ME-6a). Verified
|
||||
# 2026-10-13: the container MiniMax key reaches this model on both
|
||||
# api.minimax.io and api.minimaxi.chat; 1M context window.
|
||||
# Endpoint name MUST equal the registry name (MR-12).
|
||||
name: minimax-m31-flash-max
|
||||
provider: minimax
|
||||
family: minimax
|
||||
endpoint:
|
||||
type: cloud_api
|
||||
base_url: "https://api.minimax.io/anthropic"
|
||||
api_key_env: MINIMAX_API_KEY
|
||||
runner: claude
|
||||
default_flags:
|
||||
model: "MiniMax-M3.1-Flash-Preview"
|
||||
reasoning_effort: max
|
||||
provider_context_ref: "minimax/v1"
|
||||
provider_secret_ref: "MINIMAX_API_KEY"
|
||||
session_capable: false
|
||||
21
models/minimax-m31-flash-medium.yaml
Normal file
21
models/minimax-m31-flash-medium.yaml
Normal file
@@ -0,0 +1,21 @@
|
||||
# agent-runtime-framework — models/ (CRS tag: models)
|
||||
# MiniMax-M3.1-Flash-Preview at reasoning_effort medium. Always thinks (thinking
|
||||
# cannot be disabled; effort is the only depth control). The claude runner maps
|
||||
# default_flags.reasoning_effort to `--effort` (agent-runtimes ME-6a). Verified
|
||||
# 2026-10-13: the container MiniMax key reaches this model on both
|
||||
# api.minimax.io and api.minimaxi.chat; 1M context window.
|
||||
# Endpoint name MUST equal the registry name (MR-12).
|
||||
name: minimax-m31-flash-medium
|
||||
provider: minimax
|
||||
family: minimax
|
||||
endpoint:
|
||||
type: cloud_api
|
||||
base_url: "https://api.minimax.io/anthropic"
|
||||
api_key_env: MINIMAX_API_KEY
|
||||
runner: claude
|
||||
default_flags:
|
||||
model: "MiniMax-M3.1-Flash-Preview"
|
||||
reasoning_effort: medium
|
||||
provider_context_ref: "minimax/v1"
|
||||
provider_secret_ref: "MINIMAX_API_KEY"
|
||||
session_capable: false
|
||||
@@ -1,6 +1,6 @@
|
||||
name: minimax-code-repo
|
||||
description: "MiniMax code agent — repo clone, agent-repo branch persistence"
|
||||
model: minimax
|
||||
model: minimax-m31-flash-max
|
||||
harness: code-minimax-repo/v1
|
||||
requires_tags: [claude-code, coding-agent, git-access, project-repo]
|
||||
required_params: [repo_url, agent_repo_url]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
name: minimax-code-tdd
|
||||
description: "MiniMax code agent with TDD enforcement — tests are read-only and must pass before the task finishes"
|
||||
model: minimax
|
||||
model: minimax-m31-flash-max
|
||||
harness: code-minimax-tdd-repo/v1
|
||||
requires_tags: [coding-agent, git-access, project-repo, tdd-gate]
|
||||
required_params: [repo_url, agent_repo_url]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
name: minimax-planning
|
||||
description: "MiniMax planning agent — specs, reviews, architecture docs"
|
||||
model: minimax
|
||||
model: minimax-m31-flash-max
|
||||
harness: planning-minimax-repo/v1
|
||||
requires_tags: [claude-code, planning-agent, git-access, project-repo]
|
||||
required_params: [repo_url, agent_repo_url]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
name: minimax-security-review
|
||||
description: "MiniMax security review agent — security audit via Anthropic-compatible proxy"
|
||||
model: minimax
|
||||
model: minimax-m31-flash-max
|
||||
harness: security-review-minimax-repo/v1
|
||||
requires_tags: [claude-code, security-reviewer, best-practices, git-access, project-repo]
|
||||
required_params: [repo_url, agent_repo_url]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
name: minimax-tdd-impl
|
||||
description: "MiniMax TDD implementation agent — tests locked read-only, cannot be modified by agent"
|
||||
model: minimax
|
||||
model: minimax-m31-flash-max
|
||||
harness: minimax-tdd-impl/v1
|
||||
requires_tags: [claude-code, coding-agent, git-access, project-repo]
|
||||
required_params: [repo_url, agent_repo_url]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
name: minimax-test-writer
|
||||
description: "MiniMax test writing agent — comprehensive tests with edge cases"
|
||||
model: minimax
|
||||
model: minimax-m31-flash-max
|
||||
harness: test-writing-minimax-repo/v1
|
||||
requires_tags: [claude-code, test-writer, best-practices, git-access, project-repo]
|
||||
required_params: [repo_url, agent_repo_url]
|
||||
|
||||
@@ -1,103 +0,0 @@
|
||||
name: bug-verify-sonnet
|
||||
version: 1
|
||||
description: Read-only verification of an ai-resolve-tagged bug using Sonnet planning
|
||||
runtime: planning-sonnet-repo
|
||||
input:
|
||||
state: verify-pending
|
||||
tags_required: [ai-resolve]
|
||||
tags_forbidden: []
|
||||
required_artifacts: []
|
||||
output:
|
||||
state: 'verified|rejected'
|
||||
scope_budget:
|
||||
files_modified_max: 1
|
||||
lines_diff_max: 400
|
||||
wall_clock_seconds_max: 1800
|
||||
nodes:
|
||||
# NOTE (2026-09-09, work item 88ecd23e): the `verdict-tx` script node that
|
||||
# used to follow this one has been REMOVED. `kind: script` nodes are
|
||||
# harness-exempt (WF-COMP-6 / MN-38) — they run in their own empty
|
||||
# container with no credentialed way to clone the agent's branch or read
|
||||
# /workspace/verdict, so verdict-tx never actually worked (it always ran
|
||||
# against its own empty /workspace). Replaced with the "agent
|
||||
# self-transition" pattern (spec/typed-workflows.md): this node's own
|
||||
# `cp-service-token/v1` layer (via the `planning-sonnet-repo` composite)
|
||||
# gives it CP write credentials, so the agent PATCHes its own item's
|
||||
# flow_state directly as its final step, instead of relying on a second
|
||||
# node to read a hand-off file. finalize_run's output.state auto-advance
|
||||
# is intentionally disabled for this template (its `output.state` is the
|
||||
# pipe-separated "verified|rejected" — finalize.py treats any
|
||||
# pipe-separated output.state as schema-validation-only, never a literal
|
||||
# value to PATCH, so it never clobbers the value set below).
|
||||
- id: agent
|
||||
kind: agent
|
||||
template: planning-sonnet-repo
|
||||
model: sonnet
|
||||
runtime_env:
|
||||
AGENT_REPO_URL: "git@gitea.oreillyit.nz-ai-enablement:skynet/agent-runtimes-agents.git"
|
||||
AGENT_UPSTREAM_REPO_URL: "git@gitea.oreillyit.nz-ai-enablement:skynet/agent-runtimes.git"
|
||||
AGENT_UPSTREAM_BRANCH: "main"
|
||||
AGENT_BRANCH: "bug-verify/${item.uuid}"
|
||||
CP_URL: "https://agents.oreillyit.nz/api"
|
||||
prompt: |
|
||||
The agent-runtimes project repo is cloned at /workspace/project.
|
||||
Change to that directory before doing any work.
|
||||
|
||||
Verify the following bug report. This is a READ-ONLY investigation —
|
||||
do NOT modify any source code. Your only file changes are the
|
||||
verification artifact described below.
|
||||
|
||||
**Title**: ${item.title}
|
||||
|
||||
**Bug report**:
|
||||
${item.body}
|
||||
|
||||
Steps:
|
||||
1. cd /workspace/project
|
||||
2. Investigate the codebase to determine whether the reported bug is
|
||||
real, reproducible, and correctly diagnosed. Do not make code
|
||||
changes — this is verification, not a fix.
|
||||
3. Write your findings to
|
||||
spec/bugs/<slug>-verification.md (derive <slug> from this item's
|
||||
own slug) with exactly these sections, in order:
|
||||
- `## Reproduction` — concrete steps or evidence confirming (or
|
||||
failing to confirm) the reported symptom
|
||||
- `## Root-Cause Hypothesis` — the mechanism you believe causes it,
|
||||
with file/line references
|
||||
- `## Verdict` — exactly one of `verified` or `rejected`
|
||||
- `## Rationale` — why you reached that verdict
|
||||
4. Commit and push the verification artifact on this branch.
|
||||
5. Transition this work item's own flow_state to your verdict by
|
||||
PATCHing the control plane directly — you have a scoped CP
|
||||
write credential for exactly this (`CP_SERVICE_TOKEN_FILE`,
|
||||
`CP_URL`). Set `flow_state` to exactly `verified` or exactly
|
||||
`rejected` (matching the `## Verdict` you wrote above) — no other
|
||||
value is accepted. Run:
|
||||
|
||||
```
|
||||
python3 - <<'PY'
|
||||
import json, os, urllib.request, uuid
|
||||
|
||||
VERDICT = "verified" # or "rejected" — must match your ## Verdict
|
||||
ITEM_UUID = "${item.uuid}"
|
||||
|
||||
token = open(os.environ["CP_SERVICE_TOKEN_FILE"]).read().strip()
|
||||
url = os.environ["CP_URL"].rstrip("/") + f"/v1/work-items/{ITEM_UUID}"
|
||||
body = json.dumps({
|
||||
"flow_state": VERDICT,
|
||||
"description": f"bug-verify-sonnet@1: verdict={VERDICT}",
|
||||
}).encode()
|
||||
req = urllib.request.Request(url, data=body, method="PATCH", headers={
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/json",
|
||||
"Idempotency-Key": str(uuid.uuid4()),
|
||||
})
|
||||
with urllib.request.urlopen(req, timeout=30) as resp:
|
||||
print(resp.status, resp.read().decode())
|
||||
PY
|
||||
```
|
||||
|
||||
Replace `VERDICT` with your actual verdict before running. This is
|
||||
the ONLY code you may run outside the investigation itself — do
|
||||
not use this credential for anything else.
|
||||
6. Call task_complete only after the PATCH above returns 200.
|
||||
@@ -24,7 +24,6 @@ nodes:
|
||||
kind: script
|
||||
cmd: scripts/workflow/state-tx
|
||||
cp_write: true
|
||||
on_fail: continue
|
||||
runtime_env:
|
||||
CP_URL: "https://agents.oreillyit.nz/api"
|
||||
STATE_TX_NEW_STATE: test-skeleton-ready
|
||||
|
||||
@@ -46,7 +46,7 @@ nodes:
|
||||
- id: review-minimax
|
||||
kind: agent
|
||||
role: plan-review
|
||||
model: minimax-m2
|
||||
model: minimax-m31-flash-max
|
||||
runtime_env:
|
||||
AGENT_REPO_URL: "git@gitea.oreillyit.nz-ai-enablement:skynet/agent-runtimes-agents.git"
|
||||
AGENT_BRANCH: "task/${item.uuid}"
|
||||
|
||||
@@ -1,68 +0,0 @@
|
||||
name: scope-decompose-sonnet
|
||||
version: 1
|
||||
description: Scope and decompose a spec into tagged tasks using Sonnet planning
|
||||
runtime: spec-writing-sonnet-repo
|
||||
input:
|
||||
state: spec-reviewed-arch
|
||||
tags_forbidden: []
|
||||
required_artifacts: []
|
||||
output:
|
||||
state: scoped-tagged
|
||||
scope_budget:
|
||||
files_modified_max: 10
|
||||
lines_diff_max: 500
|
||||
wall_clock_seconds_max: 2400
|
||||
nodes:
|
||||
- id: agent
|
||||
kind: agent
|
||||
# Decomposition is spec-test-tier work (claude/stage-routing.md): Sonnet,
|
||||
# not airouter. 2026-07-04 E2E: Qwen3.6 empty-delivered this stage (task
|
||||
# "succeeded" touching only .gitignore, zero work items created) — the
|
||||
# template NAME never set the model; the node model: field does.
|
||||
template: spec-writing-sonnet-repo
|
||||
# cli defaults to "claude" — the anthropic-cloud OAuth composite drives
|
||||
# the claude CLI; "cli: agentic" is the OpenAI-compatible runner and
|
||||
# fails fast with "OPENAI_BASE_URL required" (E2E 2026-07-04).
|
||||
model: sonnet
|
||||
runtime_env:
|
||||
AGENT_REPO_URL: "git@gitea.oreillyit.nz-ai-enablement:skynet/agent-runtimes-agents.git"
|
||||
AGENT_UPSTREAM_REPO_URL: "git@gitea.oreillyit.nz-ai-enablement:skynet/agent-runtimes.git"
|
||||
AGENT_UPSTREAM_BRANCH: "main"
|
||||
AGENT_BRANCH: "spec/${item.uuid}"
|
||||
CP_URL: "https://agents.oreillyit.nz/api"
|
||||
# AR-38: this stage's deliverable is work items POSTed to the CP, not a
|
||||
# repo diff — the empty-deliverable audit would fail every success.
|
||||
AGENT_EMPTY_DELIVERABLE_CHECK: "false"
|
||||
prompt: |
|
||||
The agent-runtimes project repo is cloned at /workspace/project.
|
||||
Change to that directory before doing any work.
|
||||
|
||||
Find the spec file in /workspace/project/spec/ (the spec was
|
||||
written in an earlier pipeline stage — look for a file whose name matches
|
||||
the concept title '${item.title}' as a slug, e.g. spec/<slug>.md).
|
||||
Read that spec carefully.
|
||||
|
||||
Write a decomposition plan to /workspace/.agent-output/decompose.json with
|
||||
exactly this structure:
|
||||
{
|
||||
"project_id": "${item.project_id}",
|
||||
"space_id": "${item.space_id}",
|
||||
"parent_item_uuid": "${item.uuid}",
|
||||
"tasks": [
|
||||
{
|
||||
"title": "...",
|
||||
"description": "...",
|
||||
"flow_state": "needs-tests",
|
||||
"spec_ids": ["WT-EVAL-1", "..."],
|
||||
"test_files": ["tests/triggers/test_schema.py", "..."],
|
||||
"tags": ["airouter-eligible"]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
Include "airouter-eligible" in tags only for single-file, self-contained
|
||||
tasks that Qwen3.6 can handle alone. Omit tags for complex multi-file tasks.
|
||||
|
||||
After writing decompose.json, run this command to create the work items:
|
||||
|
||||
python3 -m lib.cp_cli planner items decompose ${item.uuid} --file /workspace/.agent-output/decompose.json
|
||||
@@ -54,7 +54,7 @@ nodes:
|
||||
- id: review-minimax
|
||||
kind: agent
|
||||
role: spec-review
|
||||
model: minimax-m2
|
||||
model: minimax-m31-flash-max
|
||||
runtime_env:
|
||||
AGENT_REPO_URL: "git@gitea.oreillyit.nz-ai-enablement:skynet/agent-runtimes-agents.git"
|
||||
AGENT_BRANCH: "spec/${item.uuid}"
|
||||
|
||||
@@ -29,12 +29,10 @@ nodes:
|
||||
kind: script
|
||||
cmd: scripts/workflow/preflight-test-fail-mode
|
||||
cp_write: false
|
||||
on_fail: continue
|
||||
- id: state-tx
|
||||
kind: script
|
||||
cmd: scripts/workflow/state-tx
|
||||
cp_write: true
|
||||
on_fail: continue
|
||||
runtime_env:
|
||||
CP_URL: "https://agents.oreillyit.nz/api"
|
||||
STATE_TX_NEW_STATE: test-validated
|
||||
|
||||
@@ -21,7 +21,7 @@ nodes:
|
||||
role: test-write-review # M36 v0.6 (bug 69370eda fix): role + model replaces
|
||||
# the ungrounded `template: minimax-test-writer`
|
||||
# (a composite that never existed in the catalog).
|
||||
model: minimax-m2
|
||||
model: minimax-m31-flash-max
|
||||
runtime_env:
|
||||
# WF-COMP-15: stage agents share one per-item work branch so their work
|
||||
# accumulates across iterations.
|
||||
|
||||
0
tests/__init__.py
Normal file
0
tests/__init__.py
Normal file
56
tests/invariants/test_cp_docs_token_layer.py
Normal file
56
tests/invariants/test_cp_docs_token_layer.py
Normal file
@@ -0,0 +1,56 @@
|
||||
"""cp-docs-token/v1 context: env-only step-docs layer (PLN-CLI-16, D-40, AU-59).
|
||||
|
||||
The per-step wfcd_ token and CA file are delivered by the dispatcher. The layer
|
||||
sets the env contract and ships one context doc; no ESO mount, no init script,
|
||||
and no reference to the cp-service-token layer.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
_LAYER = (
|
||||
Path(__file__).resolve().parents[2] / "harnesses" / "contexts" / "cp-docs-token" / "v1"
|
||||
)
|
||||
|
||||
|
||||
def _doc() -> dict:
|
||||
return yaml.safe_load((_LAYER / "harness.yaml").read_text())
|
||||
|
||||
|
||||
def test_cp_docs_token_layer_parses_with_env_keys():
|
||||
doc = _doc()
|
||||
assert doc["kind"] == "context"
|
||||
assert doc["name"] == "cp-docs-token"
|
||||
assert doc["version"] == 1
|
||||
assert doc["requires"] == []
|
||||
assert doc["provides"] == ["cp-docs-token"]
|
||||
assert doc["env"] == {
|
||||
"CP_DOCS_TOKEN_FILE": "/run/agent/cp-docs/token",
|
||||
"CP_DOCS_CA_FILE": "/run/agent/cp-docs/ca.crt",
|
||||
}
|
||||
|
||||
|
||||
def test_cp_docs_token_layer_declares_no_secrets_and_no_init():
|
||||
doc = _doc()
|
||||
assert not doc.get("secrets_required")
|
||||
assert not doc.get("scripts")
|
||||
assert not (_LAYER / "init.sh").exists()
|
||||
|
||||
|
||||
def test_cp_docs_token_layer_context_file_mapping():
|
||||
doc = _doc()
|
||||
assert doc["context_files"] == [
|
||||
{
|
||||
"source": "./CLAUDE.md",
|
||||
"target": "/opt/harness/context/cp-docs-token/CLAUDE.md",
|
||||
}
|
||||
]
|
||||
assert (_LAYER / "CLAUDE.md").is_file()
|
||||
|
||||
|
||||
def test_cp_docs_token_layer_never_mentions_cp_service_token():
|
||||
for path in _LAYER.rglob("*"):
|
||||
if path.is_file():
|
||||
assert "cp-service-token" not in path.read_text(), path
|
||||
32
tests/invariants/test_no_legacy_script_composite.py
Normal file
32
tests/invariants/test_no_legacy_script_composite.py
Normal file
@@ -0,0 +1,32 @@
|
||||
"""Track R / R3b: the legacy AU-51 bearer composite and layer stay deleted.
|
||||
|
||||
``workflow-script/v1`` (the legacy script composite) and the
|
||||
``cp-service-token/v1`` context were removed once the control plane stopped
|
||||
falling back to them (agent-runtimes 0.0.5.1890). Nothing may re-introduce
|
||||
either, and no live composite may layer the bearer context.
|
||||
"""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
_ROOT = Path(__file__).resolve().parents[2]
|
||||
_HARNESSES = _ROOT / "harnesses"
|
||||
|
||||
|
||||
def test_legacy_script_composite_directory_is_gone():
|
||||
assert not (_HARNESSES / "composites" / "workflow-script").exists()
|
||||
|
||||
|
||||
def test_cp_service_token_context_directory_is_gone():
|
||||
assert not (_HARNESSES / "contexts" / "cp-service-token").exists()
|
||||
|
||||
|
||||
def test_no_composite_layers_cp_service_token():
|
||||
offenders = []
|
||||
for path in sorted((_HARNESSES / "composites").glob("*/v*/harness.yaml")):
|
||||
doc = yaml.safe_load(path.read_text()) or {}
|
||||
for layer in doc.get("layers") or []:
|
||||
if str(layer.get("context", "")).startswith("cp-service-token"):
|
||||
offenders.append(str(path.relative_to(_ROOT)))
|
||||
assert offenders == []
|
||||
@@ -93,9 +93,7 @@ def test_cp_url_templates_discovered():
|
||||
assert {
|
||||
"test-validate@1.yaml",
|
||||
"frozen-fixture-emit@1.yaml",
|
||||
"scope-decompose-sonnet@1.yaml",
|
||||
"scope-decompose-sonnet@2.yaml",
|
||||
"bug-verify-sonnet@1.yaml",
|
||||
"bug-verify-sonnet@2.yaml",
|
||||
} <= names
|
||||
|
||||
|
||||
@@ -1,35 +0,0 @@
|
||||
"""workflow-script/v1 composite: exact layer order and credential env export.
|
||||
|
||||
Script nodes get their CP write credential only through this composite
|
||||
(the legacy WF-COMP-26 mount is removed), so the layering is load-bearing.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
_ROOT = Path(__file__).resolve().parents[2] / "harnesses"
|
||||
|
||||
|
||||
def _load(kind_dir: str, name: str) -> dict:
|
||||
return yaml.safe_load((_ROOT / kind_dir / name / "v1" / "harness.yaml").read_text())
|
||||
|
||||
|
||||
def test_workflow_script_layers_shared_variables_cp_service_token_repo_readonly():
|
||||
doc = _load("composites", "workflow-script")
|
||||
assert doc["layers"] == [
|
||||
{"context": "shared-variables/v1"},
|
||||
{"context": "cp-service-token/v1"},
|
||||
{"context": "repo-readonly/v1"},
|
||||
]
|
||||
|
||||
|
||||
def test_workflow_script_env_exports_cp_service_token_file():
|
||||
doc = _load("composites", "workflow-script")
|
||||
env: dict = dict(doc.get("env") or {})
|
||||
for layer in doc["layers"]:
|
||||
(kind, ref), = layer.items()
|
||||
name = ref.split("/")[0]
|
||||
env.update(_load(kind + "s", name).get("env") or {})
|
||||
assert env["CP_SERVICE_TOKEN_FILE"] == "/run/agent/cp-service-token/token"
|
||||
@@ -1,227 +0,0 @@
|
||||
"""Workflow-script harness composition invariant (spec/typed-workflows.md WF-COMP-31
|
||||
+ spec/workflow-variables.md WV-19 + spec/harness.md HD-14).
|
||||
|
||||
The two new framework layers — ``harnesses/contexts/shared-variables/v1/`` and
|
||||
``harnesses/composites/workflow-script/v1/`` — must exist on the tree, must parse,
|
||||
and the composite must resolve every layer it references to a real file on disk.
|
||||
This test guards against the most common regression: a missing or renamed layer
|
||||
that would only surface at the dispatcher's ``compose_harness`` call, where it
|
||||
costs an entire task's worth of wasted dispatch round-trip.
|
||||
|
||||
Spec: /home/paul/dev/claude/projects/agent-runtimes/spec/workflow-variables.md
|
||||
(WV-19, WV-20, WF-COMP-31), spec/harness.md (HD-14).
|
||||
|
||||
Not committed by the authoring agent per its instructions — written here so it
|
||||
can run locally against the current tree.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
import yaml
|
||||
|
||||
_HARNESSES_ROOT = Path(__file__).resolve().parents[2] / "harnesses"
|
||||
|
||||
# Composite layer entries use the singular kind as the dict key. The kind maps
|
||||
# to a top-level directory by adding "s".
|
||||
_KIND_TO_DIR = {
|
||||
"context": "contexts",
|
||||
"capability": "capabilities",
|
||||
"composite": "composites",
|
||||
}
|
||||
|
||||
|
||||
def _load_layer(kind_dir: str, name: str) -> dict:
|
||||
p = _HARNESSES_ROOT / kind_dir / name / "v1" / "harness.yaml"
|
||||
return yaml.safe_load(p.read_text(encoding="utf-8"))
|
||||
|
||||
|
||||
def test_shared_variables_layer_exists_and_parses():
|
||||
"""``harnesses/contexts/shared-variables/v1/harness.yaml`` must exist,
|
||||
parse as YAML, and carry ``provides: [shared-variables]`` (the HD-14 tag).
|
||||
|
||||
Spec: spec/workflow-variables.md WV-19; spec/harness.md HD-14.
|
||||
"""
|
||||
parsed = _load_layer("contexts", "shared-variables")
|
||||
assert parsed["kind"] == "context"
|
||||
assert parsed["name"] == "shared-variables"
|
||||
assert parsed["version"] == 1
|
||||
assert parsed["provides"] == ["shared-variables"], (
|
||||
"WV-19: provides MUST contain the 'shared-variables' tag — HD-14's "
|
||||
"extra_layers dedup key, and the tag the dispatcher's payload-marker "
|
||||
"injection matches against."
|
||||
)
|
||||
# WV-20: this layer's OWN env: declaration sets the two WORKFLOW_VARIABLES_*
|
||||
# vars (NOT the entrypoint, NOT the container backend).
|
||||
env = parsed["env"]
|
||||
assert env["WORKFLOW_VARIABLES_ROOT"] == "/variables"
|
||||
assert "WORKFLOW_VARIABLES_SELF" in env, (
|
||||
"WV-20: WORKFLOW_VARIABLES_SELF must be set by the layer's env: "
|
||||
"(templated per-node at harness-composition time on the dispatcher)"
|
||||
)
|
||||
# Agent-facing context: a CLAUDE.md must be mounted under /opt/harness/context/.
|
||||
files = parsed.get("context_files", [])
|
||||
targets = [f["target"] for f in files]
|
||||
assert any(t.startswith("/opt/harness/context/shared-variables/") for t in targets), (
|
||||
"WV-19: agent-facing context must be mounted under /opt/harness/context/"
|
||||
"(the HC-7 namespacing point) so the agent sees it at startup."
|
||||
)
|
||||
|
||||
|
||||
def test_shared_variables_claude_md_present_and_substantial():
|
||||
"""The agent-facing CLAUDE.md must be a non-trivial markdown document
|
||||
(HC-7 says it is read by the agent, not a placeholder).
|
||||
|
||||
Spec: spec/workflow-variables.md WV-19; spec/harness.md HC-7.
|
||||
"""
|
||||
md = _HARNESSES_ROOT / "contexts" / "shared-variables" / "v1" / "CLAUDE.md"
|
||||
assert md.exists(), "WV-19: shared-variables/v1 must ship a CLAUDE.md"
|
||||
text = md.read_text(encoding="utf-8")
|
||||
assert text.count("\n") >= 30, (
|
||||
f"WV-19: the CLAUDE.md must explain the /variables/ contract to the "
|
||||
f"agent — got {text.count(chr(10))} lines, expected >=30."
|
||||
)
|
||||
# Spot-check the four high-value pieces of content the agent MUST see:
|
||||
assert "/variables/" in text, "must explain the filesystem root"
|
||||
assert "WORKFLOW_VARIABLES_SELF" in text, "must document the env vars"
|
||||
assert "wfvar" in text, "must point at the CLI helper (WV-21)"
|
||||
assert "reserved" in text.lower(), "must list the reserved-name families (WV-6)"
|
||||
|
||||
|
||||
def test_workflow_script_composite_resolves():
|
||||
"""``harnesses/composites/workflow-script/v1/harness.yaml`` must declare
|
||||
exactly the two layers the spec names (WF-COMP-31) and every referenced
|
||||
file must exist on disk.
|
||||
|
||||
Spec: spec/typed-workflows.md WF-COMP-31 ("Internal composite layered from
|
||||
``shared-variables/v1`` + ``cp-service-token/v1``").
|
||||
"""
|
||||
parsed = _load_layer("composites", "workflow-script")
|
||||
assert parsed["kind"] == "composite"
|
||||
assert parsed["name"] == "workflow-script"
|
||||
assert parsed["version"] == 1
|
||||
assert isinstance(parsed["layers"], list) and len(parsed["layers"]) >= 2, (
|
||||
"WF-COMP-31: the composite must layer at least shared-variables/v1 + "
|
||||
"cp-service-token/v1; missing one breaks the credential env seam."
|
||||
)
|
||||
|
||||
refs = []
|
||||
for layer in parsed["layers"]:
|
||||
assert isinstance(layer, dict) and len(layer) == 1, (
|
||||
f"each layer entry must be a single-key dict (kind: ref), got {layer}"
|
||||
)
|
||||
kind_name, ref = next(iter(layer.items()))
|
||||
assert kind_name in _KIND_TO_DIR, f"unknown layer kind {kind_name!r}"
|
||||
assert "/" in ref, f"layer ref must be '<name>/v<N>', got {ref!r}"
|
||||
name, ver = ref.split("/")
|
||||
assert ver.startswith("v") and ver[1:].isdigit(), (
|
||||
f"layer ref version must match v<N>, got {ver!r}"
|
||||
)
|
||||
kind_dir = _KIND_TO_DIR[kind_name]
|
||||
layer_path = _HARNESSES_ROOT / kind_dir / name / ver / "harness.yaml"
|
||||
assert layer_path.exists(), (
|
||||
f"WF-COMP-31: composite references a layer that does not exist on "
|
||||
f"the tree: {layer_path.relative_to(_HARNESSES_ROOT)}"
|
||||
)
|
||||
layer_data = yaml.safe_load(layer_path.read_text(encoding="utf-8"))
|
||||
assert layer_data["name"] == name, (
|
||||
f"name mismatch in {layer_path}: file says {layer_data['name']!r}, "
|
||||
f"ref says {name!r}"
|
||||
)
|
||||
refs.append((kind_name, name, ver, layer_data))
|
||||
|
||||
# WF-COMP-31: the two SPECIFIED layers must both be present.
|
||||
ref_names = {(kind, name) for kind, name, _, _ in refs}
|
||||
assert ("context", "shared-variables") in ref_names, (
|
||||
"WF-COMP-31: workflow-script/v1 MUST layer shared-variables/v1 "
|
||||
"(WV-19 — the agent-facing /variables/ contract)"
|
||||
)
|
||||
assert ("context", "cp-service-token") in ref_names, (
|
||||
"WF-COMP-31 / AU-51 credential env seam: workflow-script/v1 MUST "
|
||||
"layer cp-service-token/v1 so CP_SERVICE_TOKEN_FILE is in the "
|
||||
"entrypoint process env before shell_runner runs."
|
||||
)
|
||||
|
||||
|
||||
def test_workflow_script_composite_brings_no_model_or_endpoint_layer():
|
||||
"""``kind: script`` nodes run via the shell runner — the composite MUST
|
||||
NOT bring agent-only layers (no model, endpoint, CLAUDE.md context that
|
||||
drives the agent runner, etc.). This is a structural regression gate: a
|
||||
future refactor that adds e.g. ``anthropic-cloud-paul-oauth/v1`` here
|
||||
would silently widen the script-node attack surface.
|
||||
|
||||
Spec: spec/typed-workflows.md WF-COMP-31 ("the flag-on kind: script
|
||||
harness ... MUST NOT bring agent-only layers").
|
||||
"""
|
||||
parsed = _load_layer("composites", "workflow-script")
|
||||
forbidden_substrings = (
|
||||
# cloud-auth contexts the agent runner needs
|
||||
"anthropic-cloud", "minimax", "qwen-code", "deepseek-code",
|
||||
"alibaba-model-studio", "fable-low-effort", "z-ai",
|
||||
# the model-bearing capability / context layers
|
||||
"code-methodology", "spec-writing", "planning", "integration",
|
||||
# harness composites that bundle a model
|
||||
"code-", "planning-", "spec-writing-", "best-practices-",
|
||||
"sonnet-", "opus-", "minimax-", "haiku-", "airouter-", "fable-",
|
||||
)
|
||||
layer_names = [
|
||||
next(iter(layer.values())).split("/")[0]
|
||||
for layer in parsed["layers"]
|
||||
]
|
||||
bad = [
|
||||
n for n in layer_names
|
||||
if any(sub in n for sub in forbidden_substrings)
|
||||
]
|
||||
assert not bad, (
|
||||
f"WF-COMP-31: workflow-script/v1 MUST NOT bring agent-only layers; "
|
||||
f"found: {bad}"
|
||||
)
|
||||
|
||||
|
||||
def test_cp_service_token_layer_already_sets_cp_service_token_file_env():
|
||||
"""WF-COMP-31 'AU-51 credential env seam': cp-service-token/v1 must set
|
||||
``CP_SERVICE_TOKEN_FILE`` in its env: declaration (the layer's init.sh
|
||||
restages the ESO-mounted secret to an agent-readable 0600 copy at the
|
||||
path CP_SERVICE_TOKEN_FILE points at). Verified here at the YAML level
|
||||
rather than at runtime — the spec's claim is structural.
|
||||
|
||||
Spec: spec/typed-workflows.md WF-COMP-31; spec/harness.md AU-51/AU-51b
|
||||
(bug b3a96acc).
|
||||
"""
|
||||
parsed = _load_layer("contexts", "cp-service-token")
|
||||
env = parsed["env"]
|
||||
assert "CP_SERVICE_TOKEN_FILE" in env, (
|
||||
"WF-COMP-31 'AU-51 credential env seam': cp-service-token/v1 must "
|
||||
"declare CP_SERVICE_TOKEN_FILE in its env: block; without it the "
|
||||
"shell_runner subprocess does not inherit the AU-51 credential."
|
||||
)
|
||||
fpath = env["CP_SERVICE_TOKEN_FILE"]
|
||||
assert fpath.endswith("/token"), (
|
||||
f"CP_SERVICE_TOKEN_FILE must point at the restaged 0600 file, not "
|
||||
f"the raw 0400 ESO mount — got {fpath!r}"
|
||||
)
|
||||
|
||||
|
||||
def test_workflow_script_composite_forge_credentials_are_read_only():
|
||||
"""Any layer in workflow-script/v1 that carries a forge credential MUST
|
||||
declare ``write_targets: []`` (HC-11): a script node may read the project
|
||||
repo but never push. The write-capable gitea-ssh/v1 layer must not be
|
||||
present (bug 45a48685).
|
||||
|
||||
Spec: spec/harness.md HC-11, spec/typed-workflows.md WF-COMP-34.
|
||||
"""
|
||||
parsed = _load_layer("composites", "workflow-script")
|
||||
from tests.invariants.forge_credentials import _layer_has_forge_credentials
|
||||
|
||||
for layer in parsed["layers"]:
|
||||
kind_name, ref = next(iter(layer.items()))
|
||||
name = ref.split("/")[0]
|
||||
assert name != "gitea-ssh", "workflow-script/v1 must not layer the write-capable gitea-ssh/v1 (bug 45a48685)"
|
||||
kind_dir = _KIND_TO_DIR[kind_name]
|
||||
sub = _load_layer(kind_dir, name)
|
||||
if _layer_has_forge_credentials(sub):
|
||||
assert sub.get("write_targets") == [], (
|
||||
f"workflow-script/v1's layer {name} carries a forge credential "
|
||||
f"but does not declare write_targets: [] (HC-11 read-only)."
|
||||
)
|
||||
Reference in New Issue
Block a user