mm-worktree-task: --gate re-runs the test suite itself, never trusts the agent's claim
Across ~12 supervised MiniMax units, more than half reported a green suite where a manual re-run found red (see memory/gotchas-minimax.md). Add --gate "<command>" (with --gate-timeout, default 900s): after the detached sub-session exits, mm-worktree-task itself re-runs the command in the worktree against the sub-session's LAST commit, in its own reapable process group (setsid + group kill, so a hung gate and everything it forked are bounded and cleaned up), and appends an unambiguous PASSED/FAILED verdict — plus an explicit MISMATCH callout when the agent's own report claimed success — to the result file. A missing command, non-zero exit, timeout, or dirty worktree are all failures, never a silent skip. mm-worktree-task's own exit code is unchanged: the launch already returns before the sub-session (and gate) run, detached. Claude-Session: https://claude.ai/code/session_014KNyfHmHpMbdMrHZXDRkAp
This commit is contained in:
@@ -26,6 +26,8 @@ mm-worktree-task <item-uuid> [OPTIONS]
|
||||
| `--docs all\|recent\|none` | `recent` | Which item documents to bundle. `recent` = documents updated in the last 14 days; falls back to `all` if the CLI does not expose update dates. The full list of document names is always included in the prompt regardless of which are bundled. |
|
||||
| `--tools "<list>"` | — | Extra `--allowedTools` entries merged into the default allowlist (comma-separated). |
|
||||
| `--timeout <secs>` | `1800` | Max seconds the sub-session may run (`timeout <secs>` wrapper). |
|
||||
| `--gate "<command>"` | — | Supervisor gate: after the sub-session exits, `mm-worktree-task` itself re-runs this command in the worktree against the sub-session's LAST commit and appends a `## Supervisor gate` section to the result file (see Behaviour §8). Never trust the agent's own "tests pass" claim — this re-run is the check. |
|
||||
| `--gate-timeout <secs>` | `900` | Max seconds `--gate`'s command may run before it is killed and recorded as a timed-out (failed) gate. |
|
||||
| `--context <name>` | resolved (see §0) | `cp-cli --context` used for every planner call. |
|
||||
| `--upload` | off | After the sub-session exits, upload its report as a document on the item and patch `flow_state` to `impl-complete`. Implemented only in combination with `--collect` (see Behaviour §6) since launch is detached. |
|
||||
| `--base <ref>` | `main` | Branch or commit the worktree is created from, and the ref `--collect` logs/diffs against. Needed when work lives on a long-running branch, or when one agent's branch must build on another's (e.g. an implementation agent starting from a test agent's `mm/<slug>` branch). |
|
||||
@@ -81,6 +83,14 @@ mm-worktree-task <item-uuid> [OPTIONS]
|
||||
```
|
||||
wrapped in a backgrounded subshell that, on exit, appends `<slug> rc=<rc> elapsed=<s>s` to `<report-dir>/launch.log`. The parent process returns immediately.
|
||||
Default allowlist: `Read,Write,Edit,Glob,Grep,Bash(pytest *),Bash(python3 *),Bash(ruff *),Bash(grep *),Bash(ls *),Bash(git add *),Bash(git commit *),Bash(git status*),Bash(git diff*),Bash(git rev-parse*),Bash(git log*)`.
|
||||
- **Supervisor gate (`--gate "<command>"`, optional).** Runs inside the same backgrounded subshell, immediately after the sub-session's `ask-minimax-run` process exits and before the `launch.log` line is appended — so it always evaluates the worktree exactly as the sub-session left it, never a snapshot from mid-run:
|
||||
1. Record `git -C <worktree> rev-parse HEAD` as the gate sha — this is the sub-session's LAST commit, not whatever commit existed when the sub-session claimed success.
|
||||
2. Check whether the worktree is dirty (`git status --porcelain` non-empty). A dirty worktree makes the recorded sha not fully represent what was tested, so it is treated as a **failed** gate regardless of the command's own exit code.
|
||||
3. Run the command as `setsid bash -c "<command>"` in the worktree (its own process group, `pgid == pid`), capturing combined stdout+stderr and its exit code, with a watchdog that after `--gate-timeout` (default `900`) seconds sends `SIGTERM` to the whole group (`kill -TERM -- -<pid>`), waits 5s, then `SIGKILL`s the group — a command that forks children (a test runner's own workers, say) is fully reaped, not just its immediate wrapper. A timed-out gate is recorded as **TIMED OUT** and failed regardless of the eventual exit code. A command that does not exist exits non-zero (typically `127`) and is failed the same way — there is no separate "skipped" state.
|
||||
4. Append `## Supervisor gate (re-run by mm-worktree-task)` to `<report-dir>/<slug>-result.md` (created if the sub-session never wrote one, noting that explicitly) with: the HEAD sha, the exact command, the clean/dirty flag, the exit code, an explicit **PASSED**/**FAILED** verdict, and the tail (last 60 lines) of captured output.
|
||||
5. If the verdict is FAILED and the text of `<result-file>` *as the sub-session left it* (before this section was appended) reads as a claim that tests passed, the appended section says so explicitly (`**MISMATCH:** the agent's own report ... claims a passing test run; this independent re-run ... FAILED`) — the agent's claim is never the last word.
|
||||
6. `mm-worktree-task`'s own exit code is unaffected by the gate outcome: the launch has already returned 0 before the sub-session (and gate) run at all, detached. The gate's PASSED/FAILED verdict is only ever visible in the result file / `--collect` output — never in a process exit code.
|
||||
With no `--gate`, none of the above runs and the result file is exactly what the sub-session wrote, unchanged from pre-`--gate` behaviour.
|
||||
9. Print: report dir, prompt file path, stdout file path, launch.log path, and a one-line `poll with: tail -f <stdout file>` hint.
|
||||
10. `--collect`: instead of steps 6-9, for the given `<slug>`'s existing worktree, print `git -C <worktree> log --oneline <base>..HEAD`, `git -C <worktree> diff --stat <base>...HEAD`, and the contents of `<report-dir>/<slug>-result.md` if present. If `--upload` is also given and the report file exists:
|
||||
- `python3 -m lib.cp_cli planner items documents put <uuid> mm-<slug>-<YYYY-MM-DD>.md --file <report>`
|
||||
@@ -91,7 +101,7 @@ mm-worktree-task <item-uuid> [OPTIONS]
|
||||
|
||||
With `--dryrun`/`-n`:
|
||||
- No worktree, branch, bundle file, prompt file, or report-dir directory is created; no `ask-minimax-run` process is started.
|
||||
- Prints, in order: resolved slug/uuid/worktree path/branch, the bundle manifest (section names + document names, bundled vs. available-only), the full assembled prompt text, and the exact `ask-minimax-run` command line that would run (including the resolved allowlist and timeout).
|
||||
- Prints, in order: resolved slug/uuid/worktree path/branch, the bundle manifest (section names + document names, bundled vs. available-only), the full assembled prompt text, and the exact `ask-minimax-run` command line that would run (including the resolved allowlist and timeout), plus the resolved `--gate` command and `--gate-timeout` (or `<none>` if not given). The gate itself is never executed under `--dryrun`.
|
||||
- Exit 0.
|
||||
|
||||
## Edge Cases
|
||||
@@ -105,6 +115,8 @@ With `--dryrun`/`-n`:
|
||||
| `--upload` without `--collect` | Error, exit 1 |
|
||||
| `--docs recent` but CLI gives no `updated_at` | Fall back to `--docs all` behaviour, note this in bundle `## Documents` section |
|
||||
| cp_cli sibling-list command unsupported | Skip `## Siblings` section silently (best-effort) |
|
||||
| `--gate` command missing, non-zero, or times out | Recorded as `Verdict: **FAILED**` in the `## Supervisor gate` section — never a silent skip |
|
||||
| `--gate` runs against a dirty worktree | Recorded as `Verdict: **FAILED**` regardless of the command's own exit code (the sha would not represent everything present) |
|
||||
|
||||
## Security
|
||||
|
||||
@@ -130,4 +142,9 @@ mm-worktree-task 1f17f61e-317e-4e10-bcd6-bee6a3264379 --slug ci-gate --collect
|
||||
|
||||
# Collect and upload the report + mark impl-complete
|
||||
mm-worktree-task 1f17f61e-317e-4e10-bcd6-bee6a3264379 --slug ci-gate --collect --upload
|
||||
|
||||
# Launch with a supervisor gate: re-run the real suite yourself after the
|
||||
# sub-session's last commit, bounded to 10 minutes
|
||||
mm-worktree-task 1f17f61e-317e-4e10-bcd6-bee6a3264379 --slug ci-gate \
|
||||
--gate "python3 -m pytest -q" --gate-timeout 600
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user