Files
Paul O'Reilly dda01ebeca formatters/sh: SETE-2 must scan the whole file, not line by line
SETE-2 matched `sh -c "..."` with a per-line regex requiring both quotes on one
line. The shape that motivated the check spans lines:

    sh -c "
        hugo mod get ...
        hugo mod tidy
    "

so the real sync-deps bug — the one this check exists for — was never flagged.
Verified against the actual pre-fix sync-deps: now reports at line 358,
"4 commands and no 'set -e' inside".

Scans the file text with re.DOTALL and reports the line of the match start.
Falls back to counting non-blank lines when the body is newline- rather than
;-separated, which is what the multi-line form looks like.

Adds both regression cases (multi-line without set -e must flag; multi-line
with set -e must not). Verified they go red when the whole-file scan is
disabled. Neither of the two fixed real scripts is flagged.

Note for whoever extends this suite: fixtures written with a heredoc inside it
did not behave; the printf form does. Both new tests use printf.

Claude-Session: https://claude.ai/code/session_014KNyfHmHpMbdMrHZXDRkAp
2026-09-23 19:49:33 +12:00

137 lines
4.6 KiB
Bash
Executable File

#!/usr/bin/env bash
# Formatter: Shell — shfmt + shellcheck + SETE/LEAK checks
# Contract: $1 = absolute file path, format in place, lint, exit 0 if clean / 1 if errors
#
# SETE-1: shebanged script with no `set -e` / `set -euo pipefail` anywhere
# SETE-2: sh/bash -c string with 2+ ;- or newline-separated commands and no set -e inside
# LEAK-1: command substitution reading a secret-named file interpolated into
# echo/printf/die/error/fail argument
#
# Missing shfmt AND shellcheck: emit one warning to stderr, still exit 0
# (a missing tool must not block an edit). Any SETE/LEAK finding: print
# "path:line: ID message" to stderr, exit non-zero.
set -uo pipefail
FILE="$1"
[ -z "$FILE" ] && exit 0
[ -f "$FILE" ] || exit 0
HAD_ERRORS=0
HAS_SHFMT=0
HAS_SHELLCHECK=0
# Format with shfmt if available
if command -v shfmt >/dev/null 2>&1; then
HAS_SHFMT=1
shfmt -w "$FILE" 2>/dev/null
fi
# Lint with shellcheck if available
if command -v shellcheck >/dev/null 2>&1; then
HAS_SHELLCHECK=1
if ! ERRORS=$(shellcheck -f gcc "$FILE" 2>&1); then
echo "$ERRORS" >&2
HAD_ERRORS=1
fi
fi
# --- Custom checks: SETE-1, SETE-2, LEAK-1 ---
# Always run, regardless of shfmt/shellcheck presence. Reads $FILE, emits
# one "path:line: ID message" line per finding to stdout, exits non-zero on
# any finding.
if ! CUSTOM_OUT=$(
python3 - "$FILE" <<'PYEOF'
import re
import sys
filepath = sys.argv[1]
findings = []
try:
with open(filepath, 'r', encoding='utf-8', errors='replace') as f:
content = f.read()
lines = content.splitlines()
except OSError:
sys.exit(0)
# SETE-1: shebanged script with no `set -e` / `set -euo pipefail` anywhere
first = lines[0] if lines else ''
if first.startswith('#!'):
has_sete = False
for line in lines:
if re.search(r'(^|[^A-Za-z0-9_])set\s+-[A-Za-z]*[eE]', line):
has_sete = True
break
if not has_sete:
findings.append((1, "SETE-1 executable shell script without 'set -e'; failed earlier commands will not abort"))
SECRET_RE = re.compile(r'TOKEN|SECRET|KEY|PASSWORD|_JSON_FILE|CREDENTIAL', re.IGNORECASE)
SINK_RE = re.compile(r'\b(echo|printf|die|error|fail)(?=\s|\()')
FILE_READ_RE = re.compile(
r'(\$\(|`)\s*(cat|<)\s*"\$\{?([A-Za-z_][A-Za-z0-9_]*)\}?"\s*(\)|`)'
)
def count_top_level_commands(s):
"""Count ;- or newline-separated top-level commands in a shell string,
ignoring separators inside single or double quotes."""
in_single = False
in_double = False
separators = 0
for c in s:
if not in_double and c == "'":
in_single = not in_single
elif not in_single and c == '"':
in_double = not in_double
elif c in ';\n' and not in_single and not in_double:
separators += 1
return separators + 1
for i, line in enumerate(lines, 1):
# LEAK-1: command substitution reading a secret-named file, inside echo/printf/die/error/fail
sink_match = SINK_RE.search(line)
if sink_match:
cmd = sink_match.group(1)
for cmdsub in FILE_READ_RE.finditer(line):
var = cmdsub.group(3)
if SECRET_RE.search(var):
findings.append((i, "LEAK-1 secret variable '$" + var + "' read into string passed to '" + cmd + "'; could print credential document"))
# SETE-2 runs over the WHOLE source, not line by line: the shape that motivated
# this check spans lines —
# sh -c "
# hugo mod get ...
# hugo mod tidy
# "
# — so a per-line regex requiring both quotes on one line never matched it.
for m in re.finditer(r'\b(?:sh|bash)\s+-c\s+(["\'])(.*?)\1', content, re.DOTALL):
body = m.group(2)
if re.search(r'(^|[^A-Za-z0-9_])set\s+-[A-Za-z]*[eE]', body):
continue
n = count_top_level_commands(body)
if n < 2:
n = len([ln for ln in body.splitlines() if ln.strip() and not ln.strip().startswith('#')])
if n >= 2:
line_no = content[:m.start()].count('\n') + 1
findings.append((line_no, "SETE-2 sh/bash -c with " + str(n) + " commands and no 'set -e' inside; exit status reflects only the last command"))
for line_no, msg in findings:
print(filepath + ":" + str(line_no) + ": " + msg)
sys.exit(1 if findings else 0)
PYEOF
); then
echo "$CUSTOM_OUT" >&2
HAD_ERRORS=1
fi
# If neither shfmt nor shellcheck is installed, warn ONCE so the silent pass
# is no longer silent. A missing tool must not block an edit (would be a worse
# bug than the one this warning surfaces).
if [ "$HAS_SHFMT" -eq 0 ] && [ "$HAS_SHELLCHECK" -eq 0 ]; then
echo "formatters/sh: shfmt and shellcheck not installed; install via 'apt install shfmt shellcheck' (or equivalent). Custom checks still ran." >&2
fi
exit $HAD_ERRORS