skynet/agent-runtime-framework
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(agent-repo,airouter): seed from upstream + label-gate + diff verify

Browse Source 
Three load-bearing fixes for the airouter dogfood pipeline, derived from
the 2026-05-08 batch-3 dogfood postmortem (gotchas-airouter.md items 27-30):

1. agent-repo/v1/init.sh — seed fresh task branches from
   /workspace/reference/main/ (the upstream clone) rather than the agent
   repo's stale main. This was THE killer for batch 3: the
   agent-runtimes-agents fork has been frozen at 2026-05-04 since the
   "Fork cleanup" PR, so every agent started from old state, missing
   recent test files and the M16/M22 scripts to delete. The fork remains
   the push remote (so finalize.sh works); only the working-tree seed
   moves to the upstream reference. Falls back to fork main when the
   reference clone isn't available (preserves legacy behavior). Tagged
   AR-14a.

2. requires_labels on contexts/composites — airouter context + both
   airouter composites declare requires_labels: [airouter] so the
   dispatcher's _collect_supported_harnesses (with the matching agent-
   runtimes change) advertises them only on dispatchers carrying the
   airouter label. Stops the main dispatcher from claiming airouter-
   labeled tasks and dying at init time. Composites that wrap label-
   restricted contexts MUST redeclare their own requires_labels — no
   auto-traversal of layers (kept simple).

3. agent-repo/v1/finalize.sh — AR-21 diff-against-upstream verification.
   New env-var protocol:
     - AGENT_EXPECTED_CHANGED_FILES (comma-separated paths that MUST
       appear in `git diff <ref/main>..HEAD`)
     - AGENT_FORBIDDEN_CHANGED_FILES (paths that MUST NOT appear)
   finalize.sh fails the task (exit 1) if either invariant is violated;
   the branch is still pushed for forensics so the operator can inspect.
   Catches BOTH the false-success mode (item 30 — agent reports succeeded
   but never changed the target file) AND the destructive-Write mode
   (item 21 — task 4a2f2988 stripped 9 unrelated functions). Also writes
   diff_verified, diff_mismatch, diff_changed_files into ci_metadata.json.

CRS pulls all three on next CP poll — no agent-runtimes image rebuild
needed for the framework parts. The matching dispatcher poller filter
ships in agent-runtimes (separate commit).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Paul O'Reilly
2026-05-08 15:45:41 +12:00
parent 45ec4af0e9
commit 07309567d5
5 changed files with 137 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files

77
harnesses/contexts/agent-repo/v1/finalize.sh
View File

@@ -260,6 +260,68 @@ if [ -n "${AGENT_EXPECTED_OUTPUT:-}" ]; then
fi
fi
# AR-21 (2026-05-08): Diff-against-upstream verification.
# AGENT_EXPECTED_CHANGED_FILES: comma-separated list of paths that MUST appear
# in the working-tree diff vs upstream main. Catches the failure mode where
# an agent reports "succeeded" but produced no actual change to the target
# file (real incident: 2026-05-08 dogfood batch, gotchas item 30 — three
# "succeeded" tasks merged nothing actionable).
# AGENT_FORBIDDEN_CHANGED_FILES: comma-separated list of paths that MUST NOT
# appear in the diff. Catches the inverse: an agent silently destroying or
# refactoring files outside the task scope (real incident: 2026-05-08 task
# 4a2f2988 — agent stripped 9 unrelated functions; gotchas item 17/21).
DIFF_VERIFIED=true
DIFF_MISMATCH=""
DIFF_SUMMARY=""
if [ -d /workspace/reference/main/.git ]; then
# All files actually changed in the agent's commit (vs upstream HEAD).
REF_HEAD=$(git -C /workspace/reference/main rev-parse HEAD 2>/dev/null || echo "")
if [ -n "$REF_HEAD" ]; then
# Files modified/added/deleted by this commit.
DIFF_SUMMARY=$(git diff --name-only "$REF_HEAD"..HEAD 2>/dev/null | tr '\n' ',' | sed 's/,$//')
# Required files must each appear in DIFF_SUMMARY.
if [ -n "${AGENT_EXPECTED_CHANGED_FILES:-}" ]; then
echo "Verifying required changed files: $AGENT_EXPECTED_CHANGED_FILES"
IFS=',' read -ra _REQUIRED <<< "$AGENT_EXPECTED_CHANGED_FILES"
for required in "${_REQUIRED[@]}"; do
required="${required#"${required%%[![:space:]]*}"}"
required="${required%"${required##*[![:space:]]}"}"
[ -z "$required" ] && continue
if ! echo ",$DIFF_SUMMARY," | grep -qF ",$required,"; then
echo "ERROR: Required change to '$required' missing from agent's diff"
DIFF_VERIFIED=false
DIFF_MISMATCH="$DIFF_MISMATCH missing:$required"
fi
done
fi
# Forbidden files must NOT appear in DIFF_SUMMARY.
if [ -n "${AGENT_FORBIDDEN_CHANGED_FILES:-}" ]; then
echo "Verifying forbidden files unchanged: $AGENT_FORBIDDEN_CHANGED_FILES"
IFS=',' read -ra _FORBIDDEN <<< "$AGENT_FORBIDDEN_CHANGED_FILES"
for forbidden in "${_FORBIDDEN[@]}"; do
forbidden="${forbidden#"${forbidden%%[![:space:]]*}"}"
forbidden="${forbidden%"${forbidden##*[![:space:]]}"}"
[ -z "$forbidden" ] && continue
if echo ",$DIFF_SUMMARY," | grep -qF ",$forbidden,"; then
echo "ERROR: Forbidden file '$forbidden' was modified by agent"
DIFF_VERIFIED=false
DIFF_MISMATCH="$DIFF_MISMATCH forbidden:$forbidden"
fi
done
fi
if [ "$DIFF_VERIFIED" = "true" ] && [ -n "${AGENT_EXPECTED_CHANGED_FILES:-}${AGENT_FORBIDDEN_CHANGED_FILES:-}" ]; then
echo "Diff verification passed (changed: $DIFF_SUMMARY)"
fi
else
echo "WARNING: /workspace/reference/main has no commits; skipping diff verification"
fi
else
echo "Note: /workspace/reference/main not present; skipping AR-21 diff verification"
fi
# AR-19: Push with retry — attempt up to PUSH_RETRIES+1 times (default 2: initial + 1 retry)
echo "Pushing branch $BRANCH to $REPO_URL..."
PUSHED=false
@@ -311,6 +373,12 @@ meta['diff_kb'] = float('$DIFF_KB') if '$DIFF_KB' else 0.0
meta['output_validated'] = $( [ '$OUTPUT_VALIDATED' = 'true' ] && echo 'True' || echo 'False' )
if '$OUTPUT_MISSING':
meta['output_missing'] = '$OUTPUT_MISSING'
# AR-21: diff verification metadata
meta['diff_verified'] = $( [ '$DIFF_VERIFIED' = 'true' ] && echo 'True' || echo 'False' )
if '$DIFF_MISMATCH'.strip():
meta['diff_mismatch'] = '$DIFF_MISMATCH'.strip()
if '$DIFF_SUMMARY':
meta['diff_changed_files'] = [f for f in '$DIFF_SUMMARY'.split(',') if f]
with open(out, 'w') as f:
json.dump(meta, f)
print('Wrote ci_metadata.json')
@@ -320,4 +388,13 @@ if [ "$PUSHED" = "false" ]; then
exit 1
fi
# AR-21: fail the task if diff verification didn't pass — even if push succeeded.
# Branch is preserved on the agent repo for forensics, but the task ends as
# failed so the operator + CP know it shouldn't be merged.
if [ "$DIFF_VERIFIED" = "false" ]; then
echo "ERROR: Diff verification failed:$DIFF_MISMATCH" >&2
echo "Branch $BRANCH is pushed for forensics, but the task is being marked failed." >&2
exit 1
fi
echo "=== agent-repo/v1 finalize.sh complete ==="

52
harnesses/contexts/agent-repo/v1/init.sh
View File

@@ -38,17 +38,57 @@ for ref in refs:
"
fi
# Clone agent repo working branch (AR-14)
# Clone agent repo working branch (AR-14).
#
# AR-14a (2026-05-08): seed fresh task branches from /workspace/reference/main/
# rather than the agent-repo fork's main, so a stale fork (e.g. periodic
# "Fork cleanup" PRs that reset main) doesn't poison every fresh task with
# old project state. Existing AGENT_BRANCH cherry-picks remain unchanged
# (continuing prior work). Pre-existing operator workaround in
# memory/gotchas-airouter.md item 27.
if [ -n "${AGENT_REPO_URL:-}" ] && [ -n "${AGENT_BRANCH:-}" ]; then
echo "Cloning agent working repo: $AGENT_REPO_URL (branch: $AGENT_BRANCH)"
if git clone --depth 1 --branch "$AGENT_BRANCH" "$AGENT_REPO_URL" /workspace/project 2>/dev/null; then
echo "Cloned existing branch $AGENT_BRANCH"
echo "Cloned existing branch $AGENT_BRANCH (continuing prior work)"
else
echo "Branch $AGENT_BRANCH does not exist, creating fresh clone..."
# Clone default branch, then checkout new branch
if git clone --depth 1 "$AGENT_REPO_URL" /workspace/project; then
echo "Branch $AGENT_BRANCH does not exist — seeding fresh branch from upstream reference"
# Full clone (not --depth 1) so we get a working remote for finalize.sh push.
if git clone "$AGENT_REPO_URL" /workspace/project; then
cd /workspace/project
git checkout -b "$AGENT_BRANCH"
REF_REPO=/workspace/reference/main
if [ -d "$REF_REPO/.git" ]; then
# Seed working tree from the upstream reference clone — AR-14a.
# Reference is read-only (chmod a-w), but git can still read it
# as a local-path remote for fetch + reset.
#
# Use a temporary remote name so we don't collide with 'origin'.
git remote add upstream-ref "$REF_REPO"
git fetch upstream-ref --depth 1 2>&1 | head -3 || {
echo "WARNING: failed to fetch from upstream reference; falling back to fork main" >&2
git remote remove upstream-ref 2>/dev/null
git checkout -b "$AGENT_BRANCH"
}
if git rev-parse upstream-ref/HEAD >/dev/null 2>&1; then
UPSTREAM_REF="upstream-ref/HEAD"
elif git rev-parse upstream-ref/main >/dev/null 2>&1; then
UPSTREAM_REF="upstream-ref/main"
else
UPSTREAM_REF=""
fi
if [ -n "$UPSTREAM_REF" ]; then
git checkout -b "$AGENT_BRANCH" "$UPSTREAM_REF"
git remote remove upstream-ref
echo "Seeded $AGENT_BRANCH from $REF_REPO ($(git log --oneline -1)) — AR-14a"
else
git remote remove upstream-ref 2>/dev/null
git checkout -b "$AGENT_BRANCH"
echo "WARNING: upstream-ref had no resolvable HEAD; using fork main (may be stale)" >&2
fi
else
# No reference clone available — fall back to fork main.
git checkout -b "$AGENT_BRANCH"
echo "WARNING: $REF_REPO/.git not found; using fork main (may be stale)" >&2
fi
else
echo "ERROR: Failed to clone agent repo $AGENT_REPO_URL" >&2
exit 1