The validator pattern-matches *TOKEN* as credential-shaped and rejected
both contexts, failing every task on the new harnesses. The env var name
is fixed by Claude Code, so deliver the thinking budget via the
settings.json env map in init.sh instead (jq-merge preserves apiKeyHelper
from the minimax/oauth layer).
Claude-Session: https://claude.ai/code/session_019tJk7P8tZzJ24PvtgoGhLN
Integration runs were burning the personal Anthropic 5-hour window
(429 storm + window drain, 2026-09-02). New env-only context
minimax-m3-max/v1 pins ANTHROPIC_MODEL=MiniMax-M3[1m], maxes the
thinking budget, and sets the 1M auto-compact window; layered on
minimax/v1 for base URL + apiKeyHelper auth.
Claude-Session: https://claude.ai/code/session_019tJk7P8tZzJ24PvtgoGhLN
The 237KB library was mounted at /workspace/best-practices/ in every
consuming ConfigMap but nothing in the prompt mentioned it (directory
targets never enter the context manifest). New ~90-word CLAUDE.md file
target enters the system prompt: points at INDEX.md, instructs reading
only task-relevant topics.
Claude-Session: https://claude.ac/code/session_012arAwUuBQrpnMzJ36mEVBk
- tdd-gate: pass criterion is the task's TEST_FILES scope (matches
run-ci-tests.sh M19 P-14), not the full suite — contradicted
qwen-code-methodology rule 5 in the same system prompt
- tdd-protect: tests/ path described relative to project working dir
(AGENT_WORKING_DIR is /workspace/project in repo composites)
- qwen-code-methodology: task_complete tool DOES exist — rule 8 said
the opposite; pytest examples use -q
- airouter-code-tdd template: TT-39 runtime_overrides
(max_result_bytes 24KB, max_history_turns 16)
Claude-Session: https://claude.ai/code/session_012arAwUuBQrpnMzJ36mEVBk
MiniMax moved their Anthropic-compatible API from api.minimax.io to
api.minimaxi.chat. Update ANTHROPIC_BASE_URL in the minimax/v1 context
harness and corresponding comments in init.sh and the TODO note.
Model routing is unaffected: the API defaults unrecognised model names
(e.g. "minimax" from the registry slug) to MiniMax-M3 already.
Claude-Session: https://claude.ai/code/session_01XtWRazqRbMnMRa4iV3zbVx
New context: hugo-content-workspace/v1
- init.sh: clones hugo-{customer}-content at HUGO_CONTENT_BRANCH and
hugo-{customer}-integration at main via gitea-ssh-accelerators
- finalize.sh: detects changes, commits content repo first then
integration repo; skips silently if no changes
- CLAUDE.md: instructs the agent to work in /workspace/content/
New composite: hugo-content-airouter/v1
- Combines airouter/v1 + gitea-ssh-accelerators/v1 + hugo-content-workspace/v1
- Label-gated to airouter dispatchers (ESO secret required)
Used by cms-proxy /ai/{customer}/draft endpoint.
Brings the framework CRS repo up to date with all content that was
living in agent-runtimes (local-dev fallback) but hadn't been promoted.
New composites: feature-delivery-loop, integration-direct, scaffolding-repo,
sonnet-impl-narrow, sonnet-manager, test-writing-repo
New contexts: integration/v1, scaffolding/v1, sonnet-manager/v1, z-ai/v1,
airouter/v1/bin (anthropic-compat-wrapper.sh), cp-harness/v1/init.sh
New task-templates: sonnet-integrator.yaml, workflow/* (17 typed workflow
task templates for the Epic 1 pipeline)
Updated: agent-repo/v1/finalize.sh — adds AR-38/F97 empty-deliverable audit
(SKIP_BRANCH_PUSH support, boilerplate-path filtering, ci_metadata.json flag)
Also adds MEMORY.md index and memory/ topic files for the framework repo.
Copies code-sonnet-direct/v1 (composite) and direct-push/v1 (context)
from agent-runtimes local harnesses to the CRS-served framework repo.
These are required by the sonnet-integrator task template used in the
ralph coding loop. Without them, all integrator dispatches fail with
"harness not found in any concepts repo: code-sonnet-direct/v1".
Probes 7-9 (2026-05-08) all "succeeded" but with empty diffs because
AR-14a fell back to fork main: "WARNING: /workspace/reference/main/.git
not found; using fork main (may be stale)". The reference clone python
loop printed "Cloning git@..." but never "Cloned main successfully" —
no error visible either.
Most likely cause: full-history clone (post-3087ad7) is hitting some
silent failure mode (auth, fs perms, OOM, timeout). With capture_output=True
the git clone's own output was hidden.
Add explicit logging:
- python3 -u (unbuffered stdout)
- pre-loop "reference_branches loop: N entries"
- post-clone "clone returncode=N"
- always-print stdout (last 1KB) and stderr (last 1KB) from subprocess
- explicit chmod return code check
- post-clone .git existence assertion
Next probe will tell us EXACTLY what's failing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Probe 7 (2026-05-08) finally surfaced the actual push error:
! [remote rejected] HEAD -> task-9dc266b5 (shallow update not allowed)
Cause: REFERENCE_BRANCHES handler clones with --depth 1, AR-14a then
fetches upstream-ref --depth 1. The agent's task branch is a single
commit on top of a single shallow commit — no ancestry visible. Gitea
rejects shallow pushes server-side.
Fix:
- Reference clone drops --depth 1 (full history)
- AR-14a fetch upstream-ref drops --depth 1 (full fetch from local-path)
- Cost: a few extra MB per task on tmpfs/PVC. Acceptable.
This unblocks AR-14a's upstream-seeding for the dogfood pipeline. The
agent's branch now has the full upstream history visible to gitea.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Probe 6 (2026-05-08) showed git push failing twice with exit 1 and
zero visible output — the previous form `if cmd 2>&1; then` redirected
git's stderr to stdout where the entrypoint's stderr-only log capture
missed it. CP-side log showed only the bash `set -x` trace, not the
actual git error message (e.g., "Permission denied (publickey)" or
"remote: pre-receive hook rejected").
Refactor the push retry loop:
- Capture output to PUSH_OUT via $() with `2>&1`
- Wrap in set +e/set -e to detect non-zero without aborting
- echo PUSH_OUT to stderr (where set -x trace also goes) so the
entrypoint's stderr capture sees it
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The 2026-05-08 attempt-2 dogfood batch had 8/8 tasks "succeed" with
zero branches pushed. Root cause: my AR-21 diff-verification block was
running under set -euo pipefail without explicit error handling. A
single non-zero exit anywhere in the `git diff | tr | sed` pipeline
killed finalize.sh before the metadata write or push ran.
Specific risk: `git diff <REF_HEAD>..HEAD` returns non-zero when the
SHA is unreachable (e.g., shallow clone with init.sh fork-fallback
where upstream-ref wasn't fetched). pipefail then kills the pipeline,
set -e kills the script.
Fix: wrap the entire AR-21 block in `set +eo pipefail` (with explicit
`set -eo pipefail` restore at the end). Also:
- Use `${arr[@]:-}` instead of `${arr[@]}` for set -u safety on empty
arrays
- Add `|| true` to git command substitutions (belt-and-braces)
- Use `printf` instead of `echo` for the comma-wrap (more portable)
Verified locally: when `/workspace/reference/main/.git` is absent the
block correctly skips with the existing fallback; when present and
upstream-ref is reachable, the block runs and reports DIFF_VERIFIED.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three load-bearing fixes for the airouter dogfood pipeline, derived from
the 2026-05-08 batch-3 dogfood postmortem (gotchas-airouter.md items 27-30):
1. agent-repo/v1/init.sh — seed fresh task branches from
/workspace/reference/main/ (the upstream clone) rather than the agent
repo's stale main. This was THE killer for batch 3: the
agent-runtimes-agents fork has been frozen at 2026-05-04 since the
"Fork cleanup" PR, so every agent started from old state, missing
recent test files and the M16/M22 scripts to delete. The fork remains
the push remote (so finalize.sh works); only the working-tree seed
moves to the upstream reference. Falls back to fork main when the
reference clone isn't available (preserves legacy behavior). Tagged
AR-14a.
2. requires_labels on contexts/composites — airouter context + both
airouter composites declare requires_labels: [airouter] so the
dispatcher's _collect_supported_harnesses (with the matching agent-
runtimes change) advertises them only on dispatchers carrying the
airouter label. Stops the main dispatcher from claiming airouter-
labeled tasks and dying at init time. Composites that wrap label-
restricted contexts MUST redeclare their own requires_labels — no
auto-traversal of layers (kept simple).
3. agent-repo/v1/finalize.sh — AR-21 diff-against-upstream verification.
New env-var protocol:
- AGENT_EXPECTED_CHANGED_FILES (comma-separated paths that MUST
appear in `git diff <ref/main>..HEAD`)
- AGENT_FORBIDDEN_CHANGED_FILES (paths that MUST NOT appear)
finalize.sh fails the task (exit 1) if either invariant is violated;
the branch is still pushed for forensics so the operator can inspect.
Catches BOTH the false-success mode (item 30 — agent reports succeeded
but never changed the target file) AND the destructive-Write mode
(item 21 — task 4a2f2988 stripped 9 unrelated functions). Also writes
diff_verified, diff_mismatch, diff_changed_files into ci_metadata.json.
CRS pulls all three on next CP poll — no agent-runtimes image rebuild
needed for the framework parts. The matching dispatcher poller filter
ships in agent-runtimes (separate commit).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
After the 2026-05-08 M16 Wave A1 dogfood task `4a2f2988` produced a
correct narrow change but accidentally deleted 9 unrelated functions in
the same file (Write rather than Edit), the existing Rule 1 in
qwen-code-methodology/v1/CLAUDE.md is clearly insufficient — the agent
saw it and overrode it anyway. Concrete real-world example added inline
so the next agent reading this CLAUDE.md sees the consequence: "the
narrow test passed but the application broke everywhere else, the
branch was rejected, the agent thought it had succeeded."
CRS picks up automatically on next CP poll.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The agent-runtimes repo is moving to a CRS-only harness model: dispatcher
images stop baking in /app/harnesses/ and discover them via CRS at runtime.
cp-harness was the only harness that lived only in the agent-runtimes
checkout (created in M16 Phase 1 D4, commit c1df4cc) — copied here so
nothing is lost in the cutover.
cp-harness has no current consumers (no composite or template references
it); it is infrastructure for the future M16 self-orchestration api_key
submission flow.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The airouter ESO ExternalSecret materialises a single key `api_key`
(matching the provider schema in agent-runtimes M22 Phase 8e cutover,
acct-59b7fb0b). The harness init script was checking for `auth_token` +
`base_url` (an Anthropic-compat shape that never existed in real Vault
state) and failing on every dispatch:
ERROR: /run/agent/secrets/airouter/auth_token not readable.
Check ESO ExternalSecret for airouter.
Surfaced as the second blocker for the M16 Wave A1 dogfood (the first
was the airouter dispatcher missing CRS sync; that fix went into
agent-runtimes-deploy@0f11cd1).
Same shape of bug as the minimax + gitea-ssh init scripts that landed
during the same M22 phase — those were fixed at the time, airouter was
not.
Changes:
- init.sh: verify the single `api_key` file (root-only, 0400 ESO mount).
Stage to /var/agent-secrets/airouter/api_key with mode 0600 agent-owned
(mirrors minimax pattern). H-SECRET-4 compliant — no exports.
- harness.yaml: add OPENAI_API_KEY_FILE pointing at the staged path.
Agentic runner reads the file at request time per
entrypoint/runners/agentic.py:146 (OPENAI_API_KEY_FILE precedence).
- Delete dead bin/anthropic-compat-wrapper.sh — confirmed unused per
agent-runtimes/memory/log/2026-05-07.214249.md (post-M22-Phase-9 cleanup
found these per-provider wrappers were never invoked; runner only
prepends /opt/agent/claude-wrapper.sh).
CRS picks this up automatically on next CP poll; no agent-runtimes image
rebuild needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes harnesses/contexts/z-ai/v1/. Mirrors the agent-runtimes
companion PR — the z-ai harness was never cut over to ESO and
there's no active subscription. Revival path documented in
agent-runtimes planning/future/providers F49.
Removed:
- harnesses/contexts/z-ai/v1/{harness.yaml, init.sh}
Updated:
- CLAUDE.md — drops z-ai from the 'no secrets in this repo'
context list. Retirement note added.
- gitea-admin/v1/init.sh — comment ref to z-ai removed.
- gitea-https/v1/init.sh — comment ref to z-ai removed.
No composites layer z-ai (verified via grep across composites/);
no model registry entries reference it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Phase 8 cutover (M22) renamed the anthropic harness from anthropic-cloud
→ anthropic-cloud-paul-oauth in agent-runtimes, but the equivalent
rename in this framework's composites and the deletion of the stale
SOPS-era anthropic-cloud/v1 dir were never done. Every cloud-Anthropic
composite (haiku/sonnet/opus code, planning, security-review, spec-
writing, test-writing, best-practices) layered the dead context and
silently never resolved a credential.
- Update 14 composites to layer anthropic-cloud-paul-oauth/v1
- Delete the orphaned harnesses/contexts/anthropic-cloud/v1/ directory
(used pre-Phase-9 SOPS file delivery; now obsolete)
Closes BUG-22b. Pairs with the live PATCH /v1/accounts/{id} rename
shipped under BUG-22a.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same bug as upstream agent-runtimes: scoped form never matched
because the dispatcher does literal scope.id compare and
owner_principal_id stores the user UUID, not 'paul'.
Pairs with PATCH /v1/accounts/{id} renaming the live account_label
to 'anthropic-cloud-paul-oauth' so the bare ref resolves uniquely.
Closes BUG-22a.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors agent-runtimes audit + fix. All three M22 Phase 9 wrapper-script
auth paths were broken since the cutover: each one ran as the agent UID
trying to `cat` a 0400 root-only ESO mount. Same root cause we hit on
minimax/z-ai earlier today.
Changes:
1. anthropic-cloud-paul-oauth/v1 (NEW in framework)
- Mirrors agent-runtimes — was previously only present there.
- init.sh stages oauth_token into ~/.claude/.credentials.json (Claude
Code's native subscription-OAuth schema). No env, no apiKeyHelper,
no wrapper. Restores the equivalent of what harness_init.py used to
do for the legacy SOPS path.
- The legacy `bin/anthropic-wrapper.sh` was dead code (never wired).
2. gitea-https/v1
- init.sh stages the token to $HOME/.config/git/gitea-https-token
(0600 agent:agent) and points the per-host git credential helper at
the staged copy. Previously the helper `cat`d the ESO mount path
and silently failed at every git invocation.
3. gitea-admin/v1
- init.sh stages the token to $HOME/.config/gitea-admin/token
(0600 agent:agent). Wrapper updated to read from the staged copy.
- Removes stale `requires: anthropic-cloud/v1` (the only anthropic
harness in agent-runtimes is anthropic-cloud-paul-oauth/v1).
Pattern matches gitea-ssh / minimax / z-ai: ESO mount stays root-only,
init.sh runs as root and `install -m 0600 -o agent -g agent`s a single
explicit copy. Per-secret enumeration; future ESO Secret keys remain
inaccessible by default.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors agent-runtimes commit 6f20b51. CRS serves these harness files to
dispatchers, so this repo must match.
Switches from "mount mode 0444 so the agent user can `cat` the ESO file"
to the gitea-ssh staging pattern: keep the ESO mount root-only (0400),
init.sh as root `install`s a per-secret 0600 agent-owned copy, and
apiKeyHelper points at the staged copy. Stronger blast-radius guarantee
— if the ESO Secret later grows additional keys, they remain root-only
unless the harness explicitly stages them.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors agent-runtimes commit 166c19b. CRS serves these harness files to
dispatchers, so this repo must match.
Two fixes from the failing smoke test:
1. init.sh resolves the agent user's home via getent (init.sh runs as
root, but claude runs as the agent user — different $HOME).
2. secrets_required mode "0400" → "0444" so the agent user can read the
ESO-mounted secret via apiKeyHelper. The file is in pod-local tmpfs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors the minimax fix to z-ai. Replaces bin/anthropic-compat-wrapper.sh
(which did `exec env ANTHROPIC_AUTH_TOKEN="$(cat ...)" claude "$@"`,
exposing the secret in the claude subprocess' /proc/<pid>/environ) with
init.sh that writes ~/.claude/settings.json with:
apiKeyHelper: "cat /run/agent/secrets/z-ai/auth_token"
Claude Code routes apiKeyHelper output to `Authorization: Bearer <value>`
when ANTHROPIC_BASE_URL is non-anthropic.com — exactly what the Z.ai
proxy at api.z.ai/api/anthropic requires.
The legacy K8s Secret may still ship a `base_url` file; it is
intentionally ignored by init.sh (the base URL is not a credential and
lives in harness.yaml).
Mirrors agent-runtimes commit (z-ai apiKeyHelper). CRS serves these
harness files to dispatchers, so this repo must match.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors agent-runtimes commit 1b83c81. CRS serves these harness files
to dispatchers, so this repo must match for the fix to take effect.
The minimax harness now writes ~/.claude/settings.json with an
apiKeyHelper that `cat`s the ESO-mounted /run/agent/secrets/minimax/
api_key file. Claude Code routes apiKeyHelper output to
`Authorization: Bearer <value>` for non-anthropic.com base URLs, which
is what MiniMax's /anthropic proxy requires. The secret is never read
into env, never written into a config file, and ESO rotation is
auto-recovered via Claude Code's per-session helper invocation
(also re-runs after a 401).
- harness.yaml: drop ANTHROPIC_AUTH_TOKEN_FILE env var
- init.sh: write (or jq-merge) settings.json with apiKeyHelper, chmod 0600
- bin/anthropic-compat-wrapper.sh: deleted
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors agent-runtimes — adds ANTHROPIC_AUTH_TOKEN_FILE pointer so the
Claude runner sets apiKeyHelper. Without it Claude Code CLI exits with
'Not logged in'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
gitea-ssh has 3 accounts (ai_enablement, accelerators, homelab) and
gitea-https has 2 (primary, admin) — bare provider_type refs hit
MultipleAccountsError per SR-DISP-8. Switch to scoped name.space:platform
form. Mirrors the agent-runtimes commit; the dispatcher-side parser
also gains underscore support for account names like 'ai_enablement'.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The CRS-served harnesses still carried `secrets_files: [{encrypted: true}]`
which now hard-fails on H-SECRET-1 ("SOPS-encrypted secrets_files entries
are no longer permitted") in the dispatcher's harness validator. Sync the
8 provider harnesses with the agent-runtimes copies: same `secrets_required`
shape, same `init.sh` (ESO-mounted file paths), same `bin/` wrappers.
Use bare `account_ref: "<provider>"` (not `<provider>.cp:cp` — that
scope-kind isn't valid per SR-DISP-1-FIELD).
Provider key names follow the per-provider schema as emitted by the CP
provisioner: minimax/airouter/z-ai → api_key; gitea-https/gitea-admin →
{token,base_url,username}; gitea-ssh* → {host,private_key}.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the tdd-gate harness context (TEST_PASS_REQUIRED=true + CLAUDE.md
methodology) and five provider-specific TDD composites that layer
tdd-protect + tdd-gate + python-dev + tdd-file-lock on top of the
existing code-* layer stacks. Matching task templates for sonnet, opus,
haiku, minimax, and airouter. allowed_failing_tests support noted in
warnings pending M19 pre-test hook implementation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Mount best-practices context at /workspace/best-practices/ (was
/opt/harness/context/best-practices/) for consistent agent access
- Fix /workspace/working/ → /workspace/project/ in all CLAUDE.md files
(planning, spec-writing, security-review, code-methodology,
qwen-code-methodology, test-writing)
- Update best-practices path references in all CLAUDE.md files to
/workspace/best-practices/
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
