merge: mm/fw-script (workflow-script/v1 composite)

harnesses/composites/workflow-script/v1/harness.yaml

@@ -0,0 +1,24 @@
+kind: composite
+name: workflow-script
+version: 1
+description: "Internal composite for the flag-on kind: script path (typed-workflows WF-COMP-31 / workflow-variables WV-19). Brings the shared-variables filesystem context AND the AU-51 CP write credential (cp-service-token/v1 sets CP_SERVICE_TOKEN_FILE in the entrypoint process env before shell_runner runs). NOT a model-bearing composite — the kind: script compiled shape is cli=\"shell\", params.cmd=<cmd>; this composite provides the harness layers that path needs and nothing else."
+
+# Order matters only for HC-6 env last-writer-wins / HC-8 git identity. The two
+# layers here have disjoint env vars (shared-variables sets WORKFLOW_VARIABLES_*
+# paths, cp-service-token sets CP_SERVICE_TOKEN_FILE) and no overlapping git
+# identity, so the listed order has no observable merge effect — both layers
+# reach the script-node container either way.
+layers:
+  # spec/workflow-variables.md WV-19 / spec/harness.md HD-14 — agent-facing
+  # /variables/ context (CLAUDE.md mounted at /opt/harness/context/shared-variables/CLAUDE.md)
+  # plus WORKFLOW_VARIABLES_ROOT=/variables and templated WORKFLOW_VARIABLES_SELF=<step_path>.
+  # Provides the [shared-variables] tag used by the dispatcher's extra_layers injection.
+  - context: shared-variables/v1
+  # spec/typed-workflows.md WF-COMP-31 "AU-51 credential env seam" — this layer's
+  # init.sh restages the ESO-mounted CP-service-token secret (root-owned mode 0400)
+  # to an agent-owned 0600 copy at /run/agent/cp-service-token/token and exports
+  # CP_SERVICE_TOKEN_FILE pointing at it. Verified: env.CP_SERVICE_TOKEN_FILE is set
+  # on the entrypoint process environment before shell_runner runs (entrypoint reads
+  # os.environ via env=os.environ.copy() in subprocess.run at entrypoint/runners/shell.py:73,
+  # so the same credential channel an agent node gets reaches a script node unchanged).
+  - context: cp-service-token/v1

harnesses/contexts/shared-variables/v1/CLAUDE.md

@@ -0,0 +1,146 @@
+# Shared Variables Context
+
+Your container is running with the **shared variables filesystem** mounted at `/variables/`.
+This is the only way you read a step's inputs and return its outputs to the control
+plane — there is no API to call, no credential to hold, no env var to substitute.
+You write plain files under your own step folder; the CP picks them up.
+
+Normative source: `spec/workflow-variables.md` (WV-1..6a). This document summarises
+the contract; when in doubt, the spec wins.
+
+## What is where
+
+```
+/variables/
+  workflow/                       # run inputs, frozen at dispatch time (WV-8)
+    run-id
+    template
+    item.uuid
+    item.title
+    item.label
+    item.flow-state
+    artifacts.<key>               # per WorkflowInput.required_artifacts
+  <step>/                         # this step's OWN folder — write outputs here
+  <loop-id>_<n>/<step>/           # loop body node in iteration n (n is 1-based)
+  .self                           # path to your own step folder (read-only)
+  .done                           # completion marker, written by the entrypoint
+```
+
+You only write under your OWN folder. The pack sidecar walks that folder after you
+exit and returns its contents to the CP. Earlier loop iterations' results and your
+direct `depends_on` dependencies' outputs are also under `/variables/`, pre-filled
+when your container started — you only READ those.
+
+## Two env vars are set for you
+
+- `WORKFLOW_VARIABLES_ROOT` — always `/variables`.
+- `WORKFLOW_VARIABLES_SELF` — the path to your own step folder
+  (e.g. `/variables/judge` for a top-level node named `judge`, or
+  `/variables/review-loop_2/judge` for loop iteration 2's `judge` node).
+
+Neither ever contains a value. Both are paths only. Treat both as read-only inputs.
+
+## Key naming (the contract you'll be checked against)
+
+- A variable's key is the joined, lowercased path from `/variables/` to the leaf
+  file, with `.` as the separator.
+  `/variables/judge/VERDICT` ↔ `judge.verdict`.
+  `/variables/review-loop_2/judge/findings/HIGH` ↔ `review-loop_2.judge.findings.high`.
+- **Leaf files are UPPERCASE** on disk by convention; folders stay lowercase.
+  Lookup via the helpers and the CP API is case-insensitive.
+- Each path segment, after lowercasing, must match `^[a-z0-9][a-z0-9_-]{0,63}$`.
+  Maximum depth is 8 segments. Dotfiles, symlinks, FIFOs, sockets, and devices are
+  skipped with a warning (never read or followed).
+- Two leaf files at the same level whose names lowercase to the same key → both
+  rejected at pack time as `invalid_output_key`.
+- **Reserved names** that cannot be used as step or loop ids (case-insensitive):
+  `workflow`, `while`, `for`, `foreach`, `loop`, `until`, `do`, `repeat`, `if`,
+  `else`, `elif`, `elseif`, `then`, `switch`, `case`, `default`, `when`, `break`,
+  `continue`, `return`, `goto`, `yield`, `try`, `catch`, `except`, `finally`,
+  `raise`, `throw`, `and`, `or`, `not`, `xor`, `in`, `is`, `true`, `false`,
+  `null`, `none`, `nil`, `yes`, `no`, `on`, `off`. The full list lives in
+  `controlplane/typed_workflows/reserved_ids.py` and may grow by spec amendment.
+- Step/loop ids are restricted to lowercase letters, digits, `-` and `_`, starting
+  with a letter; no trailing `_<digits>` (reserved for loop-iteration suffixes).
+
+## What you can see (visibility)
+
+- `workflow.*` — always. These are the run inputs, frozen for the life of the run
+  (later item changes are not reflected here; pass them on through step outputs).
+- The outputs of each of your **direct** `depends_on` nodes (the same set the
+  manifest DAG wires you to) — not transitive.
+- Inside a loop body, **every prior iteration's** outputs for **every** step in
+  the loop — not just the immediately preceding one. So iteration 2 sees what
+  iterations 1, 2, … of every loop body node wrote. Iteration 1 sees only its
+  direct dependencies.
+- Nothing else. Your own folder is always created, even if empty.
+
+## Declared outputs
+
+If your node's template declares `outputs: [<key>, ...]`, those are the keys you
+are expected to write. Any file actually present under your step folder is
+accepted regardless — `outputs:` is for policy and tooling, not an allowlist.
+A missing declared output produces a warning by default (configurable per-project
+to fail the step with `variables_missing_output`).
+
+## Values
+
+- **Strings only.** JSON is supported as a string value — you parse it yourself;
+  the CP never interprets a value's structure.
+- Per-value cap is configurable (`max_value_bytes`, default 1 MiB). Per-step cap
+  (`max_step_bytes`) and per-step file count (`max_step_files`) also apply.
+- Values are data — never substituted into prompts, commands, or environment
+  variables by the CP. Don't put values in shell; read them through the helpers
+  or the CLI.
+
+## Helpers (use these, not shell)
+
+A Python package is pre-installed in your image at `/opt/agent/variables/` and a
+CLI at `/usr/local/bin/wfvar`. Both are stdlib-only.
+
+```python
+from variables import load, save, keys
+
+value = load("verdict")                    # load from your own folder
+save("verdict", "CLEAN")                   # write into your own folder
+keys()                                     # list keys under your own folder
+keys(prefix="judge.")                      # narrow by prefix (read-only access)
+```
+
+```bash
+wfvar get VERDICT                 # print the value, NO trailing newline
+wfvar set VERDICT CLEAN           # write into your own folder
+wfvar list                        # list your own keys
+wfvar path                        # print your own step folder (same as $WORKFLOW_VARIABLES_SELF)
+wfvar get judge.findings.high     # read another step's output (visibility permitting)
+```
+
+Exit codes: `0` ok, `1` not found (key is valid but absent), `2` invalid key or
+usage, `3` value exceeds per-value limit, `4` write attempted outside your own
+step. Do not wrap `wfvar get` with extra shell that strips newlines — it prints
+exactly the stored bytes.
+
+**Don't write plan-like content with `wfvar set <key> <body>`** — argv breaks on
+shell quoting and truncates at ARG_MAX. For multi-line / quoting-sensitive bodies
+(e.g. the decompose plan files at `/variables/<node>/items/<local_id>`), use your
+file-write tool or a heredoc with a terminator that cannot occur in the body,
+writing directly into `/variables/<node>/...`.
+
+## Safety
+
+The pack sidecar reads your step folder with strict traversal protections
+(`O_NOFOLLOW`, `S_ISREG` checks, size caps, 4 KiB marker cap). It does not follow
+symlinks you might drop in. A symlink at any level is silently skipped with a
+warning — never assume it will be read.
+
+## What you must NOT do
+
+- Do not read or write outside your own step folder under `/variables/`.
+- Do not call out to the CP API directly — the helpers and the pack sidecar
+  handle all variable transport.
+- Do not assume `/variables/` exists in an unstamped container — it doesn't.
+- Do not write to `/variables/.done` — the entrypoint owns that marker; it is
+  written for you after `run_finalize_phase` returns, whether you exited 0, non-zero,
+  or via SIGTERM.
+- Do not store values in env vars, command lines, or prompts — values are data,
+  and data doesn't go there.

harnesses/contexts/shared-variables/v1/harness.yaml

@@ -0,0 +1,20 @@
+kind: context
+name: shared-variables
+version: 1
+description: "Workflow variables filesystem contract — /variables/ layout, key naming, helpers and CLI for CP-held step inputs/outputs (spec/workflow-variables.md WV-19; injected per-task via HD-14 extra_layers)"
+requires: []
+provides: [shared-variables]
+
+# WV-20: both env vars are set by THIS harness layer (not the entrypoint, not
+# the container backend). WORKFLOW_VARIABLES_ROOT is static; WORKFLOW_VARIABLES_SELF
+# is templated by the composer at harness-composition time on the dispatcher,
+# substituting the node's resolved step_path (the SAME value WV-15 derives at mint
+# time). A task with no shared-variables/v1 layer (unstamped) has neither env var
+# set, and /variables does not exist for it at all.
+env:
+  WORKFLOW_VARIABLES_ROOT: /variables
+  WORKFLOW_VARIABLES_SELF: "{{ step_path }}"
+
+context_files:
+  - source: ./CLAUDE.md
+    target: /opt/harness/context/shared-variables/CLAUDE.md

tests/invariants/test_workflow_script_composite_resolves.py

@@ -0,0 +1,229 @@
+"""Workflow-script harness composition invariant (spec/typed-workflows.md WF-COMP-31
++ spec/workflow-variables.md WV-19 + spec/harness.md HD-14).
+
+The two new framework layers — ``harnesses/contexts/shared-variables/v1/`` and
+``harnesses/composites/workflow-script/v1/`` — must exist on the tree, must parse,
+and the composite must resolve every layer it references to a real file on disk.
+This test guards against the most common regression: a missing or renamed layer
+that would only surface at the dispatcher's ``compose_harness`` call, where it
+costs an entire task's worth of wasted dispatch round-trip.
+
+Spec: /home/paul/dev/claude/projects/agent-runtimes/spec/workflow-variables.md
+      (WV-19, WV-20, WF-COMP-31), spec/harness.md (HD-14).
+
+Not committed by the authoring agent per its instructions — written here so it
+can run locally against the current tree.
+"""
+from __future__ import annotations
+
+from pathlib import Path
+
+import pytest
+import yaml
+
+_HARNESSES_ROOT = Path(__file__).resolve().parents[2] / "harnesses"
+
+# Composite layer entries use the singular kind as the dict key. The kind maps
+# to a top-level directory by adding "s".
+_KIND_TO_DIR = {
+    "context": "contexts",
+    "capability": "capabilities",
+    "composite": "composites",
+}
+
+
+def _load_layer(kind_dir: str, name: str) -> dict:
+    p = _HARNESSES_ROOT / kind_dir / name / "v1" / "harness.yaml"
+    return yaml.safe_load(p.read_text(encoding="utf-8"))
+
+
+def test_shared_variables_layer_exists_and_parses():
+    """``harnesses/contexts/shared-variables/v1/harness.yaml`` must exist,
+    parse as YAML, and carry ``provides: [shared-variables]`` (the HD-14 tag).
+
+    Spec: spec/workflow-variables.md WV-19; spec/harness.md HD-14.
+    """
+    parsed = _load_layer("contexts", "shared-variables")
+    assert parsed["kind"] == "context"
+    assert parsed["name"] == "shared-variables"
+    assert parsed["version"] == 1
+    assert parsed["provides"] == ["shared-variables"], (
+        "WV-19: provides MUST contain the 'shared-variables' tag — HD-14's "
+        "extra_layers dedup key, and the tag the dispatcher's payload-marker "
+        "injection matches against."
+    )
+    # WV-20: this layer's OWN env: declaration sets the two WORKFLOW_VARIABLES_*
+    # vars (NOT the entrypoint, NOT the container backend).
+    env = parsed["env"]
+    assert env["WORKFLOW_VARIABLES_ROOT"] == "/variables"
+    assert "WORKFLOW_VARIABLES_SELF" in env, (
+        "WV-20: WORKFLOW_VARIABLES_SELF must be set by the layer's env: "
+        "(templated per-node at harness-composition time on the dispatcher)"
+    )
+    # Agent-facing context: a CLAUDE.md must be mounted under /opt/harness/context/.
+    files = parsed.get("context_files", [])
+    targets = [f["target"] for f in files]
+    assert any(t.startswith("/opt/harness/context/shared-variables/") for t in targets), (
+        "WV-19: agent-facing context must be mounted under /opt/harness/context/"
+        "(the HC-7 namespacing point) so the agent sees it at startup."
+    )
+
+
+def test_shared_variables_claude_md_present_and_substantial():
+    """The agent-facing CLAUDE.md must be a non-trivial markdown document
+    (HC-7 says it is read by the agent, not a placeholder).
+
+    Spec: spec/workflow-variables.md WV-19; spec/harness.md HC-7.
+    """
+    md = _HARNESSES_ROOT / "contexts" / "shared-variables" / "v1" / "CLAUDE.md"
+    assert md.exists(), "WV-19: shared-variables/v1 must ship a CLAUDE.md"
+    text = md.read_text(encoding="utf-8")
+    assert text.count("\n") >= 30, (
+        f"WV-19: the CLAUDE.md must explain the /variables/ contract to the "
+        f"agent — got {text.count(chr(10))} lines, expected >=30."
+    )
+    # Spot-check the four high-value pieces of content the agent MUST see:
+    assert "/variables/" in text, "must explain the filesystem root"
+    assert "WORKFLOW_VARIABLES_SELF" in text, "must document the env vars"
+    assert "wfvar" in text, "must point at the CLI helper (WV-21)"
+    assert "reserved" in text.lower(), "must list the reserved-name families (WV-6)"
+
+
+def test_workflow_script_composite_resolves():
+    """``harnesses/composites/workflow-script/v1/harness.yaml`` must declare
+    exactly the two layers the spec names (WF-COMP-31) and every referenced
+    file must exist on disk.
+
+    Spec: spec/typed-workflows.md WF-COMP-31 ("Internal composite layered from
+    ``shared-variables/v1`` + ``cp-service-token/v1``").
+    """
+    parsed = _load_layer("composites", "workflow-script")
+    assert parsed["kind"] == "composite"
+    assert parsed["name"] == "workflow-script"
+    assert parsed["version"] == 1
+    assert isinstance(parsed["layers"], list) and len(parsed["layers"]) >= 2, (
+        "WF-COMP-31: the composite must layer at least shared-variables/v1 + "
+        "cp-service-token/v1; missing one breaks the credential env seam."
+    )
+
+    refs = []
+    for layer in parsed["layers"]:
+        assert isinstance(layer, dict) and len(layer) == 1, (
+            f"each layer entry must be a single-key dict (kind: ref), got {layer}"
+        )
+        kind_name, ref = next(iter(layer.items()))
+        assert kind_name in _KIND_TO_DIR, f"unknown layer kind {kind_name!r}"
+        assert "/" in ref, f"layer ref must be '<name>/v<N>', got {ref!r}"
+        name, ver = ref.split("/")
+        assert ver.startswith("v") and ver[1:].isdigit(), (
+            f"layer ref version must match v<N>, got {ver!r}"
+        )
+        kind_dir = _KIND_TO_DIR[kind_name]
+        layer_path = _HARNESSES_ROOT / kind_dir / name / ver / "harness.yaml"
+        assert layer_path.exists(), (
+            f"WF-COMP-31: composite references a layer that does not exist on "
+            f"the tree: {layer_path.relative_to(_HARNESSES_ROOT)}"
+        )
+        layer_data = yaml.safe_load(layer_path.read_text(encoding="utf-8"))
+        assert layer_data["name"] == name, (
+            f"name mismatch in {layer_path}: file says {layer_data['name']!r}, "
+            f"ref says {name!r}"
+        )
+        refs.append((kind_name, name, ver, layer_data))
+
+    # WF-COMP-31: the two SPECIFIED layers must both be present.
+    ref_names = {(kind, name) for kind, name, _, _ in refs}
+    assert ("context", "shared-variables") in ref_names, (
+        "WF-COMP-31: workflow-script/v1 MUST layer shared-variables/v1 "
+        "(WV-19 — the agent-facing /variables/ contract)"
+    )
+    assert ("context", "cp-service-token") in ref_names, (
+        "WF-COMP-31 / AU-51 credential env seam: workflow-script/v1 MUST "
+        "layer cp-service-token/v1 so CP_SERVICE_TOKEN_FILE is in the "
+        "entrypoint process env before shell_runner runs."
+    )
+
+
+def test_workflow_script_composite_brings_no_model_or_endpoint_layer():
+    """``kind: script`` nodes run via the shell runner — the composite MUST
+    NOT bring agent-only layers (no model, endpoint, CLAUDE.md context that
+    drives the agent runner, etc.). This is a structural regression gate: a
+    future refactor that adds e.g. ``anthropic-cloud-paul-oauth/v1`` here
+    would silently widen the script-node attack surface.
+
+    Spec: spec/typed-workflows.md WF-COMP-31 ("the flag-on kind: script
+    harness ... MUST NOT bring agent-only layers").
+    """
+    parsed = _load_layer("composites", "workflow-script")
+    forbidden_substrings = (
+        # cloud-auth contexts the agent runner needs
+        "anthropic-cloud", "minimax", "qwen-code", "deepseek-code",
+        "alibaba-model-studio", "fable-low-effort", "z-ai",
+        # the model-bearing capability / context layers
+        "code-methodology", "spec-writing", "planning", "integration",
+        # harness composites that bundle a model
+        "code-", "planning-", "spec-writing-", "best-practices-",
+        "sonnet-", "opus-", "minimax-", "haiku-", "airouter-", "fable-",
+    )
+    layer_names = [
+        next(iter(layer.values())).split("/")[0]
+        for layer in parsed["layers"]
+    ]
+    bad = [
+        n for n in layer_names
+        if any(sub in n for sub in forbidden_substrings)
+    ]
+    assert not bad, (
+        f"WF-COMP-31: workflow-script/v1 MUST NOT bring agent-only layers; "
+        f"found: {bad}"
+    )
+
+
+def test_cp_service_token_layer_already_sets_cp_service_token_file_env():
+    """WF-COMP-31 'AU-51 credential env seam': cp-service-token/v1 must set
+    ``CP_SERVICE_TOKEN_FILE`` in its env: declaration (the layer's init.sh
+    restages the ESO-mounted secret to an agent-readable 0600 copy at the
+    path CP_SERVICE_TOKEN_FILE points at). Verified here at the YAML level
+    rather than at runtime — the spec's claim is structural.
+
+    Spec: spec/typed-workflows.md WF-COMP-31; spec/harness.md AU-51/AU-51b
+    (bug b3a96acc).
+    """
+    parsed = _load_layer("contexts", "cp-service-token")
+    env = parsed["env"]
+    assert "CP_SERVICE_TOKEN_FILE" in env, (
+        "WF-COMP-31 'AU-51 credential env seam': cp-service-token/v1 must "
+        "declare CP_SERVICE_TOKEN_FILE in its env: block; without it the "
+        "shell_runner subprocess does not inherit the AU-51 credential."
+    )
+    fpath = env["CP_SERVICE_TOKEN_FILE"]
+    assert fpath.endswith("/token"), (
+        f"CP_SERVICE_TOKEN_FILE must point at the restaged 0600 file, not "
+        f"the raw 0400 ESO mount — got {fpath!r}"
+    )
+
+
+def test_workflow_script_composite_no_forge_credentials_no_write_targets_needed():
+    """workflow-script/v1 has no forge credentials of its own and so does
+    NOT need to declare ``write_targets`` (HC-11). This is the inverse of
+    the forge-credentials invariant: a non-credential composite is fine
+    without that field.
+
+    Spec: spec/harness.md HC-11 (only layers with forge credentials MUST
+    declare write_targets).
+    """
+    parsed = _load_layer("composites", "workflow-script")
+    # Walk every layer entry, ensure none introduce a forge credential.
+    from tests.invariants.forge_credentials import _layer_has_forge_credentials
+
+    for layer in parsed["layers"]:
+        kind_name, ref = next(iter(layer.items()))
+        name = ref.split("/")[0]
+        kind_dir = _KIND_TO_DIR[kind_name]
+        sub = _load_layer(kind_dir, name)
+        assert not _layer_has_forge_credentials(sub), (
+            f"workflow-script/v1's layer {name} introduced a forge credential; "
+            f"HC-11 would then require write_targets to be declared on the "
+            f"composite, and a script-node composite should never carry "
+            f"forge credentials (WF-COMP-31)."
+        )