debug(agent-repo): expose reference_branches clone failure mode

Probes 7-9 (2026-05-08) all "succeeded" but with empty diffs because
AR-14a fell back to fork main: "WARNING: /workspace/reference/main/.git
not found; using fork main (may be stale)". The reference clone python
loop printed "Cloning git@..." but never "Cloned main successfully" —
no error visible either.

Most likely cause: full-history clone (post-3087ad7) is hitting some
silent failure mode (auth, fs perms, OOM, timeout). With capture_output=True
the git clone's own output was hidden.

Add explicit logging:
- python3 -u (unbuffered stdout)
- pre-loop "reference_branches loop: N entries"
- post-clone "clone returncode=N"
- always-print stdout (last 1KB) and stderr (last 1KB) from subprocess
- explicit chmod return code check
- post-clone .git existence assertion

Next probe will tell us EXACTLY what's failing.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Paul O'Reilly
2026-05-08 17:24:10 +12:00
parent 3087ad7de1
commit 415118434c

View File

@@ -13,9 +13,10 @@ mkdir -p /workspace/.agent-output
# Clone reference branches (AR-12, AR-13)
if [ -n "${REFERENCE_BRANCHES:-}" ] && [ "${REFERENCE_BRANCHES:-}" != "[]" ]; then
echo "Cloning reference branches..."
echo "$REFERENCE_BRANCHES" | python3 -c "
echo "$REFERENCE_BRANCHES" | python3 -u -c "
import json, sys, subprocess, os
refs = json.load(sys.stdin)
print(f'reference_branches loop: {len(refs)} entries')
for ref in refs:
name = ref.get('name', '')
repo_url = ref.get('repo_url', '')
@@ -24,20 +25,30 @@ for ref in refs:
print(f'Cloning {repo_url} ({branch}) -> {dest}')
# Full clone (no --depth) so AR-14a-seeded agent branches have visible
# ancestry when pushed back to the agent-repo. Gitea rejects shallow
# pushes with "shallow update not allowed". Real incident: 2026-05-08
# probe 7. The cost is a few extra MB on a tmpfs/PVC; acceptable.
# pushes with shallow-update-not-allowed. Real incident: 2026-05-08
# probe 7.
result = subprocess.run(
['git', 'clone', '--branch', branch,
'-c', 'core.symlinks=false', repo_url, dest],
capture_output=True, text=True
)
print(f'clone returncode={result.returncode}')
if result.stdout:
print(f'clone stdout (last 1KB): {result.stdout[-1000:]}')
if result.stderr:
print(f'clone stderr (last 1KB): {result.stderr[-1000:]}')
if result.returncode != 0:
print(f'ERROR: Failed to clone {repo_url}: {result.stderr}', file=sys.stderr)
print(f'ERROR: Failed to clone {repo_url}', file=sys.stderr)
sys.exit(1)
# Strip any symlinks (security: prevent /proc/1/environ exfiltration)
subprocess.run(['find', dest, '-type', 'l', '-exec', 'rm', '{}', ';'])
# Make reference read-only
subprocess.run(['chmod', '-R', 'a-w', dest])
chmod_r = subprocess.run(['chmod', '-R', 'a-w', dest], capture_output=True, text=True)
if chmod_r.returncode != 0:
print(f'WARNING: chmod returned {chmod_r.returncode}: {chmod_r.stderr[:300]}')
if not os.path.isdir(os.path.join(dest, '.git')):
print(f'ERROR: clone exit 0 but {dest}/.git missing', file=sys.stderr)
sys.exit(1)
print(f'Cloned {name} successfully')
"
fi