best-practices/llm-code-security.md

# Security of LLM-Generated Code

Practical guide to security vulnerabilities commonly introduced by LLMs (Claude, GPT-4, Copilot) when generating Python, shell scripts, Kubernetes manifests, and Helm charts. Based on published research from 2024-2026.

## Key Statistics

- 25-75% of AI-generated code contains security vulnerabilities depending on language, model, and prompting (Endor Labs, multiple academic studies)
- 29.5% of Copilot-generated Python snippets and 24.2% of JavaScript snippets contain security weaknesses across 43 CWE categories (ACM study, 2024)
- 19.7% of LLM-suggested packages are hallucinations -- non-existent package names (slopsquatting study, 576,000 code samples across 16 models)
- 80% of AI-suggested dependencies contain known risks (Endor Labs 2025 State of Dependency Management Report)
- Repositories with Copilot active show 6.4% secret leakage rate, 40% higher than the 4.6% baseline across public repos

---

## 1. OWASP Top 10 in LLM-Generated Code

Missing input sanitization is the single most common security flaw in LLM-generated code across all languages and models. The most prevalent CWE categories are:

| CWE | Name | Frequency |
|-----|------|-----------|
| CWE-89 | SQL Injection | Very High |
| CWE-79 | Cross-Site Scripting (XSS) | Very High |
| CWE-78 | OS Command Injection | High |
| CWE-22 | Path Traversal | High |
| CWE-20 | Improper Input Validation | Very High |
| CWE-259/798 | Hard-coded Credentials | High |
| CWE-330 | Insufficiently Random Values | High |
| CWE-94 | Code Injection | High |
| CWE-120/787 | Buffer Overflow | Medium (C/C++) |
| CWE-918 | SSRF | Medium |

### What LLMs get wrong

LLMs generate code that "works" for the happy path but omits defensive coding. They reproduce patterns from training data, which is full of tutorials and Stack Overflow snippets that skip security for brevity. The model optimises for functional correctness, not security.

### SQL Injection

**Vulnerable pattern (Python):**
```python
# LLM-generated: string interpolation in SQL
def get_user(username):
    query = f"SELECT * FROM users WHERE username = '{username}'"
    cursor.execute(query)
    return cursor.fetchone()
```

**Secure alternative:**
```python
def get_user(username):
    cursor.execute("SELECT * FROM users WHERE username = %s", (username,))
    return cursor.fetchone()
```

### Command Injection

**Vulnerable pattern (Python):**
```python
import subprocess
def ping_host(hostname):
    result = subprocess.run(f"ping -c 1 {hostname}", shell=True, capture_output=True)
    return result.stdout
```

**Secure alternative:**
```python
import subprocess
import shlex
def ping_host(hostname):
    # Validate hostname format first
    if not re.match(r'^[a-zA-Z0-9._-]+$', hostname):
        raise ValueError("Invalid hostname")
    result = subprocess.run(["ping", "-c", "1", hostname], capture_output=True)
    return result.stdout
```

### Command Injection (Shell Scripts)

**Vulnerable pattern:**
```bash
#!/bin/bash
# LLM-generated: unquoted variable in command
filename=$1
cat $filename | grep "pattern"
```

**Secure alternative:**
```bash
#!/bin/bash
filename="$1"
# Validate the path is within expected directory
realpath_file="$(realpath -- "$filename")"
if [[ "$realpath_file" != /expected/dir/* ]]; then
    echo "Error: path outside allowed directory" >&2
    exit 1
fi
grep "pattern" -- "$filename"
```

### Path Traversal

**Vulnerable pattern (Python):**
```python
@app.route('/files/<path:filename>')
def serve_file(filename):
    return send_file(os.path.join('/data', filename))
```

**Secure alternative:**
```python
@app.route('/files/<path:filename>')
def serve_file(filename):
    # send_from_directory validates the path stays within the directory
    return send_from_directory('/data', filename)
```

### How to catch it in review

- Search for string formatting in SQL: `f"SELECT`, `f"INSERT`, `f"UPDATE`, `f"DELETE`, `"SELECT.*" %`, `"SELECT.*" +`
- Search for `shell=True` in subprocess calls
- Search for `os.path.join` with user-controlled input without path validation
- Search for unquoted `$variables` in shell scripts
- Use SAST tools: Bandit (Python), ShellCheck (bash), semgrep with security rulesets

---

## 2. Secrets and Credentials

### What LLMs get wrong

LLMs frequently hardcode secrets directly into generated code. This happens because training data is full of tutorials with placeholder credentials that look real, and the model replicates the pattern. CWE-259 (Hard-coded Password) and CWE-798 (Hard-coded Credentials) are among the most common LLM-generated vulnerabilities.

Copilot specifically has been shown to leak secrets from its training context -- researchers built algorithms that generate prompts designed to extract secrets by inducing Copilot to disclose original credentials from training data.

### Vulnerable patterns

**Hardcoded API key (Python):**
```python
API_KEY = "sk-proj-abc123def456..."
client = openai.OpenAI(api_key=API_KEY)
```

**Hardcoded database credentials (Python):**
```python
conn = psycopg2.connect(
    host="db.example.com",
    user="admin",
    password="supersecret123",
    database="production"
)
```

**Hardcoded token in shell script:**
```bash
curl -H "Authorization: Bearer ghp_abc123def456" https://api.github.com/repos
```

**Secrets in Kubernetes manifests:**
```yaml
env:
  - name: DATABASE_PASSWORD
    value: "plaintext-password-here"  # Not a Secret reference
```

### Secure alternatives

**Python -- environment variables or file-based secrets:**
```python
import os
API_KEY = os.environ["OPENAI_API_KEY"]
# Or read from a mounted secret file
with open("/run/secrets/api_key") as f:
    API_KEY = f.read().strip()
```

**Shell -- read from file, never as CLI argument:**
```bash
# Read token from file (not visible in ps output)
TOKEN="$(cat /path/to/secret/file)"
curl -H "Authorization: Bearer ${TOKEN}" https://api.github.com/repos
```

**Kubernetes -- reference a Secret object:**
```yaml
env:
  - name: DATABASE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: db-credentials
        key: password
```

### How to catch it in review

- Run `detect-secrets scan` or `gitleaks` on every commit (pre-commit hook)
- Search for patterns: `password =`, `api_key =`, `token =`, `secret =` with string literal values
- Search for `Bearer ` followed by a literal string in shell scripts
- In Kubernetes manifests, search for `value:` under `env:` entries (should be `valueFrom:` for sensitive values)
- Check that `.env` files are in `.gitignore`

---

## 3. Dependency Risks

### What LLMs get wrong

LLMs hallucinate package names at alarming rates. A study of 576,000 code samples across 16 LLMs found 19.7% of suggested packages were hallucinations. Open-source models hallucinate at 21.7%, commercial models at 5.2%. Critically, 43% of hallucinated package names appeared consistently across repeated prompts, making them predictable targets.

This enables **slopsquatting**: attackers register packages matching commonly hallucinated names and inject malicious code. 38% of hallucinated names were similar to real package names (not random strings), making them plausible-looking.

Beyond hallucination, LLMs also suggest:
- **Outdated versions** with known CVEs (training data lag)
- **Deprecated packages** that have been superseded
- **Packages with known vulnerabilities** -- 80% of AI-suggested dependencies contain known risks

### Vulnerable patterns

**Hallucinated package (Python):**
```python
# LLM suggests a package that doesn't exist (or was registered by an attacker)
from flask_security_utils import sanitize_input  # Not a real package
```

**Pinned to vulnerable version:**
```
# requirements.txt generated by LLM
requests==2.25.1  # Known CVE in older versions
pyjwt==1.7.1      # Known vulnerabilities
```

**Overly broad dependency (shell):**
```bash
pip install cryptography  # Without version pin -- could get a compromised version
```

### Secure alternatives

- **Always verify packages exist** on PyPI/npm/etc. before using LLM-suggested imports
- **Pin versions and verify them:**
  ```
  requests==2.32.3  # Verified from PyPI, no known CVEs
  ```
- **Use lockfiles** (`pip freeze`, `poetry.lock`, `package-lock.json`) and audit them
- **Run dependency scanners:** `pip-audit`, `npm audit`, `trivy fs .`

### How to catch it in review

- Run `pip install --dry-run` or equivalent to verify packages resolve before committing
- Use `pip-audit` / `npm audit` / `trivy` in CI to catch known vulnerabilities
- Compare LLM-suggested package names against registry search results
- Be suspicious of packages with very few downloads or recent creation dates
- Search for version pins and verify them against current stable releases

---

## 4. Over-Permissive Defaults

### What LLMs get wrong

LLMs default to the most permissive configuration because it "works" with the least friction. Training data is full of tutorials and quick-start guides that use wide-open settings. The model has no concept of a deployment environment or threat model.

### Vulnerable patterns

**Binding to all interfaces (Python):**
```python
app.run(host="0.0.0.0", port=8080, debug=True)  # Exposed to network + debug mode
```

**Wide-open CORS (Python/Flask):**
```python
CORS(app, origins="*", supports_credentials=True)
```

**Permissive file permissions (shell):**
```bash
chmod 777 /app/data
chmod 666 /etc/config/credentials.yaml
```

**Disabled TLS verification (Python):**
```python
requests.get(url, verify=False)
```

**Kubernetes Service exposed externally by default:**
```yaml
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  type: LoadBalancer  # Exposed to the network
  ports:
    - port: 80
```

### Secure alternatives

**Bind to localhost unless external access is needed:**
```python
app.run(host="127.0.0.1", port=8080, debug=False)
```

**Explicit CORS origins:**
```python
CORS(app, origins=["https://app.example.com"], supports_credentials=True)
```

**Restrictive file permissions:**
```bash
chmod 750 /app/data        # Owner rwx, group rx, others none
chmod 640 /etc/config/credentials.yaml  # Owner rw, group r, others none
```

**TLS verification enabled (always):**
```python
requests.get(url, verify=True)  # Default, but be explicit
# If using internal CA:
requests.get(url, verify="/etc/ssl/certs/internal-ca.pem")
```

**ClusterIP by default, expose deliberately:**
```yaml
apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  type: ClusterIP  # Internal only, use Ingress for external access
  ports:
    - port: 80
```

### How to catch it in review

- Search for `0.0.0.0`, `host="0.0.0.0"`, `debug=True` in application code
- Search for `origins="*"` or `Access-Control-Allow-Origin: *` in CORS config
- Search for `chmod 777`, `chmod 666`, or any world-readable/writable permissions
- Search for `verify=False` in HTTP client calls
- Search for `type: LoadBalancer` or `type: NodePort` in Kubernetes manifests without explicit justification
- Search for `GRANT ALL` in database setup scripts

---

## 5. Infrastructure-as-Code Risks

### What LLMs get wrong

LLMs generate Kubernetes manifests and Helm charts that are functionally correct but security-negligent. They omit security contexts, resource limits, network policies, and run containers as root by default. Research (GenKubeSec, KubeGuard) found that LLMs can "confidently recommend configurations that introduce new vulnerabilities" including suggesting "allow all" rules just to satisfy constraints.

### Vulnerable patterns

**Privileged container (Kubernetes):**
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
        - name: my-app
          image: my-app:latest  # No digest, mutable tag
          # No securityContext at all -- runs as root
          # No resource limits -- can consume entire node
          # No readOnlyRootFilesystem
```

**Overly broad RBAC:**
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: my-app
subjects:
  - kind: ServiceAccount
    name: my-app
roleRef:
  kind: ClusterRole
  name: cluster-admin  # Full cluster access
```

**No NetworkPolicy (default allows all traffic):**
```yaml
# LLMs typically omit NetworkPolicy entirely
# Without it, any pod can talk to any other pod
```

**Helm values without security defaults:**
```yaml
# values.yaml generated by LLM
replicaCount: 1
image:
  repository: my-app
  tag: latest  # Mutable, unpinned
service:
  type: LoadBalancer  # Externally exposed
# No securityContext, no resources, no networkPolicy
```

### Secure alternatives

**Hardened container:**
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: my-app
          image: my-app@sha256:abc123...  # Pinned by digest
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 500m
              memory: 256Mi
```

**Least-privilege RBAC:**
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role  # Namespaced, not ClusterRole
metadata:
  name: my-app
  namespace: my-namespace
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list"]  # Only what's needed
```

**Default-deny NetworkPolicy:**
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: my-app
spec:
  podSelector:
    matchLabels:
      app: my-app
  policyTypes: ["Ingress", "Egress"]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - port: 8080
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: database
      ports:
        - port: 5432
```

### How to catch it in review

- Run `kubesec scan`, `kube-linter`, or `trivy config` against manifests
- Search for `privileged: true`, `allowPrivilegeEscalation: true` (should almost never appear)
- Search for `cluster-admin` in RBAC bindings
- Check that every Deployment/StatefulSet has `resources:` limits and `securityContext:`
- Check that every namespace has at least one NetworkPolicy
- Search for `image:.*:latest` -- tags should be pinned to specific versions or digests
- Check for `automountServiceAccountToken: false` on pods that don't need the K8s API
- In Helm charts, verify `values.yaml` includes security defaults, not just functional defaults

---

## 6. Input Validation Gaps

### What LLMs get wrong

LLMs generate code that handles the happy path but skips validation of types, lengths, formats, and ranges. They omit validation unless explicitly prompted, because training data (tutorials, examples) does the same. The model has no awareness of the threat model or what inputs are user-controlled.

### Vulnerable patterns

**No type/length validation (Python API):**
```python
@app.route('/api/users', methods=['POST'])
def create_user():
    data = request.get_json()
    username = data['username']  # No validation at all
    email = data['email']        # No format check
    age = data['age']            # No type or range check
    db.execute("INSERT INTO users (username, email, age) VALUES (%s, %s, %s)",
               (username, email, age))
```

**No path validation (shell):**
```bash
#!/bin/bash
# LLM-generated backup script
BACKUP_DIR="$1"
cp -r /important/data "$BACKUP_DIR"  # No validation of $1
```

### Secure alternatives

**Validated API input (Python):**
```python
from pydantic import BaseModel, EmailStr, Field

class CreateUserRequest(BaseModel):
    username: str = Field(min_length=3, max_length=50, pattern=r'^[a-zA-Z0-9_]+$')
    email: EmailStr
    age: int = Field(ge=0, le=150)

@app.route('/api/users', methods=['POST'])
def create_user():
    data = CreateUserRequest(**request.get_json())  # Validates or raises 422
    db.execute("INSERT INTO users (username, email, age) VALUES (%s, %s, %s)",
               (data.username, data.email, data.age))
```

**Validated shell input:**
```bash
#!/bin/bash
BACKUP_DIR="$1"
if [[ -z "$BACKUP_DIR" ]]; then
    echo "Error: backup directory required" >&2
    exit 1
fi
if [[ ! -d "$BACKUP_DIR" ]]; then
    echo "Error: '$BACKUP_DIR' is not a directory" >&2
    exit 1
fi
# Resolve and validate path
REAL_DIR="$(realpath -- "$BACKUP_DIR")"
if [[ "$REAL_DIR" != /allowed/backup/* ]]; then
    echo "Error: backup directory must be under /allowed/backup/" >&2
    exit 1
fi
cp -r /important/data "$REAL_DIR"
```

### How to catch it in review

- Check that all API endpoints use schema validation (Pydantic, marshmallow, JSON Schema, Joi)
- Search for `request.get_json()`, `request.args`, `request.form` usage without subsequent validation
- In shell scripts, check that all positional parameters (`$1`, `$2`, etc.) are validated before use
- Look for direct use of user input in file operations, database queries, or system commands
- Verify that numeric inputs have range checks and string inputs have length/format checks

---

## 7. Error Handling That Leaks Information

### What LLMs get wrong

LLMs generate code with verbose error handling that exposes internal details -- stack traces, file paths, database schemas, SQL queries, internal hostnames. This happens because training data includes development-mode error handling, and the model doesn't distinguish between dev and production contexts.

### Vulnerable patterns

**Leaking stack traces (Python/Flask):**
```python
@app.errorhandler(Exception)
def handle_error(e):
    return jsonify({
        "error": str(e),
        "traceback": traceback.format_exc(),  # Full stack trace
        "query": last_query,                   # SQL query that failed
    }), 500
```

**Leaking database details:**
```python
try:
    cursor.execute(query)
except psycopg2.Error as e:
    return f"Database error: {e}"  # Includes table names, column names, query
```

**Leaking file paths (shell):**
```bash
echo "Error: failed to read config from /etc/myapp/secrets/database.yaml"
echo "Stack: $(python3 -c 'import traceback; traceback.print_exc()')"
```

### Secure alternatives

**Generic error response with internal logging:**
```python
import logging
logger = logging.getLogger(__name__)

@app.errorhandler(Exception)
def handle_error(e):
    logger.exception("Unhandled exception")  # Full details go to logs
    return jsonify({"error": "Internal server error"}), 500  # Generic to client
```

**Safe database error handling:**
```python
try:
    cursor.execute(query, params)
except psycopg2.Error as e:
    logger.exception("Database query failed")
    return jsonify({"error": "A database error occurred"}), 500
```

### How to catch it in review

- Search for `traceback.format_exc()` or `traceback.print_exc()` in response-building code
- Search for `str(e)` or `repr(e)` in API responses (should go to logs, not clients)
- Check that `DEBUG = False` / `debug=False` in production config
- Verify error handlers return generic messages and log details internally
- Search for internal paths (`/etc/`, `/home/`, `/var/`) in user-facing error strings

---

## 8. Cryptography Mistakes

### What LLMs get wrong

LLMs reproduce cryptographic anti-patterns from training data. CWE-780 (Use of RSA without OAEP) is the most observed weakness in Java. Common failures include using ECB mode (which leaks patterns), predictable IVs, deprecated algorithms (MD5, SHA-1 for security purposes), and rolling custom crypto. Cryptography misconfiguration appears in approximately 22-24% of security vulnerabilities across leading LLM models.

### Vulnerable patterns

**ECB mode (Python):**
```python
from Crypto.Cipher import AES
cipher = AES.new(key, AES.MODE_ECB)  # ECB leaks patterns in ciphertext
ciphertext = cipher.encrypt(plaintext)
```

**Hardcoded IV:**
```python
iv = b'\x00' * 16  # Predictable IV defeats the purpose of CBC/GCM
cipher = AES.new(key, AES.MODE_CBC, iv=iv)
```

**MD5 for password hashing:**
```python
import hashlib
password_hash = hashlib.md5(password.encode()).hexdigest()  # Broken for security
```

**Weak random for tokens:**
```python
import random
token = ''.join(random.choices(string.ascii_letters, k=32))  # Not cryptographically secure
```

### Secure alternatives

**AES-GCM with random IV:**
```python
from Crypto.Cipher import AES
from Crypto.Random import get_random_bytes

key = get_random_bytes(32)  # AES-256
nonce = get_random_bytes(12)  # Random nonce for GCM
cipher = AES.new(key, AES.MODE_GCM, nonce=nonce)
ciphertext, tag = cipher.encrypt_and_digest(plaintext)
# Store nonce + tag + ciphertext together
```

**Proper password hashing:**
```python
import bcrypt
# Hashing
password_hash = bcrypt.hashpw(password.encode(), bcrypt.gensalt(rounds=12))
# Verification
bcrypt.checkpw(password.encode(), stored_hash)
```

**Cryptographically secure random:**
```python
import secrets
token = secrets.token_urlsafe(32)  # Cryptographically secure
```

### How to catch it in review

- Search for `MODE_ECB` -- should almost never be used
- Search for `md5`, `sha1` used for passwords or security tokens (fine for checksums, not for security)
- Search for `random.` (stdlib) used for tokens, keys, or security values -- should be `secrets.`
- Search for hardcoded IVs: `iv = b'`, `iv = bytes(`, `nonce = b'\x00`
- Search for `hashlib` used directly for password storage -- should be `bcrypt`, `argon2`, or `scrypt`
- Use `bandit` which has specific checks for weak crypto (B303, B304, B305)

---

## 9. Research Findings (2024-2026)

### ACM / TOSEM: Security Weaknesses of Copilot-Generated Code in GitHub Projects
Analyzed real-world Copilot-generated code on GitHub. Found 29.5% of Python and 24.2% of JavaScript snippets contained security weaknesses across 43 CWE categories. Top weaknesses: CWE-330 (insufficiently random values), CWE-94 (code injection), CWE-79 (XSS).

### Large-Scale GitHub Analysis (October 2025)
Analyzed 7,703 files from 4 AI tools across public GitHub repos. Found 4,241 CWE instances across 77 distinct vulnerability types. ChatGPT-generated code comprised 91.5% of the sample, Copilot 7.5%.

### Slopsquatting Research (2025)
576,000 code samples across 16 LLMs: 19.7% of suggested packages were hallucinations (205,474 unique fake names). Open-source models hallucinated at 21.7%, commercial at 5.2%. 43% of hallucinated names appeared consistently (predictable, attackable).

### Endor Labs: State of Dependency Management (2025)
80% of AI-suggested dependencies contain known risks. 44-49% of dependencies imported by coding agents contained known security vulnerabilities.

### Copilot Code Review Study (2025)
GitHub Copilot's code review feature frequently fails to detect critical vulnerabilities (SQL injection, XSS, insecure deserialization). Primarily flags low-severity issues like coding style.

### Sonar: Coding Personalities of Leading LLMs (2025)
Multi-model analysis finding that cryptography misconfiguration appears in 22-24% of vulnerabilities across leading models. Missing input sanitization is the most common flaw category.

### GenKubeSec / KubeGuard (2024-2025)
Research on LLM-generated Kubernetes configurations found models confidently recommend insecure configurations and may suggest "allow all" rules to satisfy functional requirements.

### OWASP Top 10 for LLM Applications (2025 Update)
Updated to reflect agentic AI risks. Key additions: System Prompt Leakage, Excessive Agency. Improper Output Handling (treating LLM output as trusted) remains a top-5 risk. Core message: treat all LLM output as untrusted data.

### Security Degradation in Iterative Generation (2025)
Code security degrades with iterative LLM refinement -- each round of "fix this" prompting can introduce new vulnerabilities while fixing the original one.

---

## 10. Practical Review Checklist

Use this checklist when reviewing LLM-generated code:

### Python
- [ ] No string formatting in SQL queries (use parameterised queries)
- [ ] No `shell=True` in subprocess calls
- [ ] No `verify=False` in HTTP requests
- [ ] No `random.` for security values (use `secrets.`)
- [ ] No `hashlib.md5/sha1` for passwords (use `bcrypt`/`argon2`)
- [ ] No hardcoded credentials (search for `password =`, `api_key =`, `token =`)
- [ ] Input validation on all API endpoints (Pydantic, marshmallow)
- [ ] Error handlers return generic messages, log details internally
- [ ] `debug=False` in production config
- [ ] All dependencies exist on PyPI and are pinned to audited versions
- [ ] `host="127.0.0.1"` unless external binding is explicitly required

### Shell Scripts
- [ ] All variables quoted (`"$var"` not `$var`)
- [ ] User-provided paths validated with `realpath` and boundary checks
- [ ] No secrets as command-line arguments (use files or env vars)
- [ ] No `chmod 777` or `chmod 666`
- [ ] ShellCheck passes with no warnings

### Kubernetes Manifests
- [ ] `securityContext` present with `runAsNonRoot: true`, `readOnlyRootFilesystem: true`, `allowPrivilegeEscalation: false`
- [ ] `capabilities.drop: ["ALL"]`
- [ ] `resources.requests` and `resources.limits` defined
- [ ] No `privileged: true`
- [ ] No `cluster-admin` RBAC bindings
- [ ] `automountServiceAccountToken: false` where K8s API access is not needed
- [ ] Images pinned to digest or specific version (not `:latest`)
- [ ] Services use `ClusterIP` by default (not `LoadBalancer`/`NodePort` without justification)
- [ ] NetworkPolicy exists for the namespace/workload

### Helm Charts
- [ ] `values.yaml` includes secure defaults for securityContext, resources, service type
- [ ] Templates don't embed secrets in plaintext
- [ ] Chart version and appVersion pinned
- [ ] `helm template` renders valid, secure manifests with default values
- [ ] `values.schema.json` validates required security fields

---

## 11. Gated Model Downloads Require Out-of-Band License Acceptance

### What goes wrong

HuggingFace (and similar model hubs) return **403 Forbidden** for "gated" models even when the HTTP request carries a valid user token. The hub enforces that the token's user has manually accepted the license agreement on the web UI for that *specific* model. Automation cannot bypass this — there is no API to accept the license.

This breaks reproducible-build scripts, container bake pipelines, and agent workflows that pull third-party ML models: the first run on a fresh account/token fails with an opaque 403 and no hint that human action is required.

### Pattern

Bake a pre-flight check into every model-download script:

```python
def preflight_gated_model(model_id: str, token: str) -> None:
    r = requests.head(f"https://huggingface.co/{model_id}/resolve/main/config.json",
                      headers={"Authorization": f"Bearer {token}"},
                      allow_redirects=True)
    if r.status_code == 403:
        url = f"https://huggingface.co/{model_id}"
        raise SystemExit(
            f"Model {model_id} is gated. Open {url} in a browser, sign in as "
            f"the token owner, accept the license, then rerun this script."
        )
    r.raise_for_status()
```

### Rules

- **Surface a human-readable error on 403** — do not retry, do not fall back to a different model silently
- **Include the exact URL** to visit and the exact action required ("accept the license")
- **Check every gated model** at pipeline start, not lazily at download time, so the human step is front-loaded
- **Document which models are gated** in the project's README — license acceptance is per-user, so every new operator needs to do it once

Applies to any project pulling third-party ML models from HuggingFace, Meta's Llama portal, Stability AI's hub, or similar.

---

## 12. Operational Vulnerabilities in AI-Generated Code (Beyond Traditional SAST)

Traditional SAST tools (Bandit, Semgrep, SonarQube, Snyk) focus on known vulnerability patterns — injection, XSS, hardcoded secrets. But analysis of AI-generated code at scale (538,860 findings across 3,518 scans, SentinaLayer 2026) reveals that the **top vulnerability categories are structural and operational**, not traditional:

| Category | % of P0-P2 Findings | What SAST Misses |
|---|---|---|
| CI/CD Integrity Gaps | 31% | Gate ordering, missing dependency chains, workflow step sequencing |
| Backend Reliability | 27% | Missing idempotency keys, premature health checks, retry logic gaps |
| Security Overlay | 24% | Supply chain trust gaps, unverified checksums, unsigned artifacts |
| Supply Chain Provenance | 11% | Missing signature verification, unpinned base images in multi-stage builds |
| Data Layer Integrity | 7% | Unsafe write paths, missing concurrent-access guards, race conditions |

**The single most common P0-P2 finding: missing idempotency keys in webhook handlers (18% of all critical/high/medium findings).** This is not a vulnerability that any SAST tool checks for.

### Why AI agents produce these

AI coding agents optimise for **functional correctness** — does the code produce the right output for the happy path? They consistently miss:

1. **Retry safety.** Webhook handlers, API endpoints, and event processors that work correctly on first invocation but corrupt data on retry. Agents don't model what happens when the same request arrives twice.

2. **Ordering dependencies.** CI/CD pipelines where steps depend on prior steps' artifacts but the dependency isn't explicit. Works when steps happen to run in order, breaks under parallelism or partial failure.

3. **Health check timing.** Services that report healthy before their dependencies are ready. The agent sees "return 200 from /health" and implements it, without considering that the database connection pool hasn't warmed yet.

4. **Checksum and signature verification.** Agents download artifacts, pull images, and install packages without verifying integrity. The code works, but the supply chain is unverified.

5. **Concurrent write safety.** File operations, database writes, and cache updates that work under single-threaded testing but corrupt under concurrent access. Agents don't think about lock ordering or write-after-read races.

### Deterministic checks you can add to CI

These can be implemented as fast pre-scan rules (regex + AST) that run before expensive test suites:

**Idempotency:**
- Webhook handlers without idempotency key extraction/dedup
- POST endpoints that create resources without checking for existing duplicates
- Event processors without at-least-once safety (no dedup by event ID)

**CI/CD Integrity:**
- GitHub/Gitea Actions steps that reference artifacts from prior steps without explicit `needs:`
- Dockerfile `COPY --from=` referencing stages without explicit ordering
- Helm hooks without `hook-weight` when ordering matters

**Supply Chain:**
- `curl | bash` or `wget -O- | sh` without checksum verification
- Container images pulled by tag without digest pinning
- Package installation without lockfile or hash verification
- `go install` / `pip install` from URLs without integrity checks

**Health Checks:**
- HTTP health endpoints that return 200 unconditionally (no dependency readiness check)
- Liveness probes identical to readiness probes (should be different — liveness checks "am I stuck?", readiness checks "am I ready to serve?")
- `initialDelaySeconds: 0` on readiness probes for services with startup dependencies

**Concurrent Access:**
- File writes without advisory locking (`fcntl.flock` / `flock`)
- Database read-then-write sequences without transactions or SELECT FOR UPDATE
- Cache operations without atomic compare-and-swap

### Review checklist (operational)

- [ ] Every webhook handler extracts and deduplicates by an idempotency key
- [ ] Every POST endpoint that creates resources checks for pre-existing duplicates
- [ ] CI/CD steps declare explicit dependencies on prior steps' outputs
- [ ] Downloaded artifacts are verified by checksum or signature before use
- [ ] Container base images are pinned by digest, not just tag
- [ ] Health check endpoints verify dependency readiness, not just "process is alive"
- [ ] Readiness and liveness probes serve different purposes
- [ ] File and database writes under concurrent access use appropriate locking
- [ ] Event processors handle at-least-once delivery (idempotent or deduplicating)
- [ ] Multi-stage Docker builds have explicit stage ordering and artifact dependencies

---

## Sources

- [Security Weaknesses of Copilot-Generated Code in GitHub Projects (ACM TOSEM)](https://dl.acm.org/doi/10.1145/3716848)
- [Security Vulnerabilities in AI-Generated Code: A Large-Scale Analysis (arXiv, Oct 2025)](https://arxiv.org/abs/2510.26103)
- [The Most Common Security Vulnerabilities in AI-Generated Code (Endor Labs)](https://www.endorlabs.com/learn/the-most-common-security-vulnerabilities-in-ai-generated-code)
- [Endor Labs 2025 State of Dependency Management Report](https://www.prnewswire.com/news-releases/endor-labs-launches-2025-state-of-dependency-management-report-finds-80-of-ai-suggested-dependencies-contain-risks-302603438.html)
- [LLMs' AI-Generated Code Remains Wildly Insecure (Dark Reading)](https://www.darkreading.com/application-security/llms-ai-generated-code-wildly-insecure)
- [Popular LLMs Found to Produce Vulnerable Code by Default (Infosecurity Magazine)](https://www.infosecurity-magazine.com/news/llms-vulnerable-code-default/)
- [Slopsquatting: How AI Hallucinations Are Fueling Supply Chain Attacks (Socket.dev)](https://socket.dev/blog/slopsquatting-how-ai-hallucinations-are-fueling-a-new-class-of-supply-chain-attacks)
- [Slopsquatting meets Dependency Confusion (Andrew Nesbitt)](https://nesbitt.io/2025/12/10/slopsquatting-meets-dependency-confusion.html)
- [AI-Generated Code Packages Can Lead to Slopsquatting Threat (DevOps.com)](https://devops.com/ai-generated-code-packages-can-lead-to-slopsquatting-threat-2/)
- [OWASP Top 10 for LLM Applications 2025](https://owasp.org/www-project-top-10-for-large-language-model-applications/)
- [OWASP LLM Top 10: How it Applies to Code Generation (Sonar)](https://www.sonarsource.com/resources/library/owasp-llm-code-generation/)
- [The Coding Personalities of Leading LLMs (SonarSource)](https://www.sonarsource.com/the-coding-personalities-of-leading-llms.pdf)
- [GenKubeSec: LLM-Based Kubernetes Misconfiguration Detection](https://arxiv.org/html/2405.19954v1)
- [KubeGuard: LLM-Assisted Kubernetes Hardening](https://arxiv.org/abs/2509.04191)
- [Security Degradation in Iterative AI Code Generation (arXiv)](https://arxiv.org/pdf/2506.11022)
- [GitHub Copilot's Code Review: Can AI Spot Security Flaws? (arXiv)](https://arxiv.org/html/2509.13650v1)
- [Security Risks of Vibe Coding and LLM Assistants (Kaspersky)](https://www.kaspersky.com/blog/vibe-coding-2025-risks/54584/)
- [The Risks of Hardcoding Secrets in Code Generated by LLMs (Cycode)](https://cycode.com/blog/the-risks-of-hardcoding-secrets-in-code-generated-by-language-learning-models/)
- [Security Flaws in DeepSeek-Generated Code (CrowdStrike)](https://www.crowdstrike.com/en-us/blog/crowdstrike-researchers-identify-hidden-vulnerabilities-ai-coded-software/)