best-practices/llm-code-security.md

Security of LLM-Generated Code

Practical guide to security vulnerabilities commonly introduced by LLMs (Claude, GPT-4, Copilot) when generating Python, shell scripts, Kubernetes manifests, and Helm charts. Based on published research from 2024-2026.

Key Statistics

• 25-75% of AI-generated code contains security vulnerabilities depending on language, model, and prompting (Endor Labs, multiple academic studies)
• 29.5% of Copilot-generated Python snippets and 24.2% of JavaScript snippets contain security weaknesses across 43 CWE categories (ACM study, 2024)
• 19.7% of LLM-suggested packages are hallucinations -- non-existent package names (slopsquatting study, 576,000 code samples across 16 models)
• 80% of AI-suggested dependencies contain known risks (Endor Labs 2025 State of Dependency Management Report)
• Repositories with Copilot active show 6.4% secret leakage rate, 40% higher than the 4.6% baseline across public repos

---

1. OWASP Top 10 in LLM-Generated Code

Missing input sanitization is the single most common security flaw in LLM-generated code across all languages and models. The most prevalent CWE categories are:

CWE	Name	Frequency
CWE-89	SQL Injection	Very High
CWE-79	Cross-Site Scripting (XSS)	Very High
CWE-78	OS Command Injection	High
CWE-22	Path Traversal	High
CWE-20	Improper Input Validation	Very High
CWE-259/798	Hard-coded Credentials	High
CWE-330	Insufficiently Random Values	High
CWE-94	Code Injection	High
CWE-120/787	Buffer Overflow	Medium (C/C++)
CWE-918	SSRF	Medium

What LLMs get wrong

LLMs generate code that "works" for the happy path but omits defensive coding. They reproduce patterns from training data, which is full of tutorials and Stack Overflow snippets that skip security for brevity. The model optimises for functional correctness, not security.

SQL Injection

Vulnerable pattern (Python):

# LLM-generated: string interpolation in SQL
def get_user(username):
    query = f"SELECT * FROM users WHERE username = '{username}'"
    cursor.execute(query)
    return cursor.fetchone()

Secure alternative:

def get_user(username):
    cursor.execute("SELECT * FROM users WHERE username = %s", (username,))
    return cursor.fetchone()

Command Injection

Vulnerable pattern (Python):

import subprocess
def ping_host(hostname):
    result = subprocess.run(f"ping -c 1 {hostname}", shell=True, capture_output=True)
    return result.stdout

Secure alternative:

import subprocess
import shlex
def ping_host(hostname):
    # Validate hostname format first
    if not re.match(r'^[a-zA-Z0-9._-]+$', hostname):
        raise ValueError("Invalid hostname")
    result = subprocess.run(["ping", "-c", "1", hostname], capture_output=True)
    return result.stdout

Command Injection (Shell Scripts)

Vulnerable pattern:

#!/bin/bash
# LLM-generated: unquoted variable in command
filename=$1
cat $filename | grep "pattern"

Secure alternative:

#!/bin/bash
filename="$1"
# Validate the path is within expected directory
realpath_file="$(realpath -- "$filename")"
if [[ "$realpath_file" != /expected/dir/* ]]; then
    echo "Error: path outside allowed directory" >&2
    exit 1
fi
grep "pattern" -- "$filename"

Path Traversal

Vulnerable pattern (Python):

@app.route('/files/<path:filename>')
def serve_file(filename):
    return send_file(os.path.join('/data', filename))

Secure alternative:

@app.route('/files/<path:filename>')
def serve_file(filename):
    # send_from_directory validates the path stays within the directory
    return send_from_directory('/data', filename)

How to catch it in review

• Search for string formatting in SQL: f"SELECT, f"INSERT, f"UPDATE, f"DELETE, "SELECT.*" %, "SELECT.*" +
• Search for shell=True in subprocess calls
• Search for os.path.join with user-controlled input without path validation
• Search for unquoted $variables in shell scripts
• Use SAST tools: Bandit (Python), ShellCheck (bash), semgrep with security rulesets

---

2. Secrets and Credentials

What LLMs get wrong

LLMs frequently hardcode secrets directly into generated code. This happens because training data is full of tutorials with placeholder credentials that look real, and the model replicates the pattern. CWE-259 (Hard-coded Password) and CWE-798 (Hard-coded Credentials) are among the most common LLM-generated vulnerabilities.

Copilot specifically has been shown to leak secrets from its training context -- researchers built algorithms that generate prompts designed to extract secrets by inducing Copilot to disclose original credentials from training data.

Vulnerable patterns

Hardcoded API key (Python):

API_KEY = "sk-proj-abc123def456..."
client = openai.OpenAI(api_key=API_KEY)

Hardcoded database credentials (Python):

conn = psycopg2.connect(
    host="db.example.com",
    user="admin",
    password="supersecret123",
    database="production"
)

Hardcoded token in shell script:

curl -H "Authorization: Bearer ghp_abc123def456" https://api.github.com/repos

Secrets in Kubernetes manifests:

env:
  - name: DATABASE_PASSWORD
    value: "plaintext-password-here"  # Not a Secret reference

Secure alternatives

Python -- environment variables or file-based secrets:

import os
API_KEY = os.environ["OPENAI_API_KEY"]
# Or read from a mounted secret file
with open("/run/secrets/api_key") as f:
    API_KEY = f.read().strip()

Shell -- read from file, never as CLI argument:

# Read token from file (not visible in ps output)
TOKEN="$(cat /path/to/secret/file)"
curl -H "Authorization: Bearer ${TOKEN}" https://api.github.com/repos

Kubernetes -- reference a Secret object:

env:
  - name: DATABASE_PASSWORD
    valueFrom:
      secretKeyRef:
        name: db-credentials
        key: password

How to catch it in review

• Run detect-secrets scan or gitleaks on every commit (pre-commit hook)
• Search for patterns: password =, api_key =, token =, secret = with string literal values
• Search for Bearer followed by a literal string in shell scripts
• In Kubernetes manifests, search for value: under env: entries (should be valueFrom: for sensitive values)
• Check that .env files are in .gitignore

---

3. Dependency Risks

What LLMs get wrong

LLMs hallucinate package names at alarming rates. A study of 576,000 code samples across 16 LLMs found 19.7% of suggested packages were hallucinations. Open-source models hallucinate at 21.7%, commercial models at 5.2%. Critically, 43% of hallucinated package names appeared consistently across repeated prompts, making them predictable targets.

This enables slopsquatting: attackers register packages matching commonly hallucinated names and inject malicious code. 38% of hallucinated names were similar to real package names (not random strings), making them plausible-looking.

Beyond hallucination, LLMs also suggest:

• Outdated versions with known CVEs (training data lag)
• Deprecated packages that have been superseded
• Packages with known vulnerabilities -- 80% of AI-suggested dependencies contain known risks

Vulnerable patterns

Hallucinated package (Python):

# LLM suggests a package that doesn't exist (or was registered by an attacker)
from flask_security_utils import sanitize_input  # Not a real package

Pinned to vulnerable version:

# requirements.txt generated by LLM
requests==2.25.1  # Known CVE in older versions
pyjwt==1.7.1      # Known vulnerabilities

Overly broad dependency (shell):

pip install cryptography  # Without version pin -- could get a compromised version

Secure alternatives

• Always verify packages exist on PyPI/npm/etc. before using LLM-suggested imports
• Pin versions and verify them:

  requests==2.32.3  # Verified from PyPI, no known CVEs
  

• Use lockfiles (pip freeze, poetry.lock, package-lock.json) and audit them
• Run dependency scanners: pip-audit, npm audit, trivy fs .

How to catch it in review

• Run pip install --dry-run or equivalent to verify packages resolve before committing
• Use pip-audit / npm audit / trivy in CI to catch known vulnerabilities
• Compare LLM-suggested package names against registry search results
• Be suspicious of packages with very few downloads or recent creation dates
• Search for version pins and verify them against current stable releases

---

4. Over-Permissive Defaults

What LLMs get wrong

LLMs default to the most permissive configuration because it "works" with the least friction. Training data is full of tutorials and quick-start guides that use wide-open settings. The model has no concept of a deployment environment or threat model.

Vulnerable patterns

Binding to all interfaces (Python):

app.run(host="0.0.0.0", port=8080, debug=True)  # Exposed to network + debug mode

Wide-open CORS (Python/Flask):

CORS(app, origins="*", supports_credentials=True)

Permissive file permissions (shell):

chmod 777 /app/data
chmod 666 /etc/config/credentials.yaml

Disabled TLS verification (Python):

requests.get(url, verify=False)

Kubernetes Service exposed externally by default:

apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  type: LoadBalancer  # Exposed to the network
  ports:
    - port: 80

Secure alternatives

Bind to localhost unless external access is needed:

app.run(host="127.0.0.1", port=8080, debug=False)

Explicit CORS origins:

CORS(app, origins=["https://app.example.com"], supports_credentials=True)

Restrictive file permissions:

chmod 750 /app/data        # Owner rwx, group rx, others none
chmod 640 /etc/config/credentials.yaml  # Owner rw, group r, others none

TLS verification enabled (always):

requests.get(url, verify=True)  # Default, but be explicit
# If using internal CA:
requests.get(url, verify="/etc/ssl/certs/internal-ca.pem")

ClusterIP by default, expose deliberately:

apiVersion: v1
kind: Service
metadata:
  name: my-app
spec:
  type: ClusterIP  # Internal only, use Ingress for external access
  ports:
    - port: 80

How to catch it in review

• Search for 0.0.0.0, host="0.0.0.0", debug=True in application code
• Search for origins="*" or Access-Control-Allow-Origin: * in CORS config
• Search for chmod 777, chmod 666, or any world-readable/writable permissions
• Search for verify=False in HTTP client calls
• Search for type: LoadBalancer or type: NodePort in Kubernetes manifests without explicit justification
• Search for GRANT ALL in database setup scripts

---

5. Infrastructure-as-Code Risks

What LLMs get wrong

LLMs generate Kubernetes manifests and Helm charts that are functionally correct but security-negligent. They omit security contexts, resource limits, network policies, and run containers as root by default. Research (GenKubeSec, KubeGuard) found that LLMs can "confidently recommend configurations that introduce new vulnerabilities" including suggesting "allow all" rules just to satisfy constraints.

Vulnerable patterns

Privileged container (Kubernetes):

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      containers:
        - name: my-app
          image: my-app:latest  # No digest, mutable tag
          # No securityContext at all -- runs as root
          # No resource limits -- can consume entire node
          # No readOnlyRootFilesystem

Overly broad RBAC:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: my-app
subjects:
  - kind: ServiceAccount
    name: my-app
roleRef:
  kind: ClusterRole
  name: cluster-admin  # Full cluster access

No NetworkPolicy (default allows all traffic):

# LLMs typically omit NetworkPolicy entirely
# Without it, any pod can talk to any other pod

Helm values without security defaults:

# values.yaml generated by LLM
replicaCount: 1
image:
  repository: my-app
  tag: latest  # Mutable, unpinned
service:
  type: LoadBalancer  # Externally exposed
# No securityContext, no resources, no networkPolicy

Secure alternatives

Hardened container:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
spec:
  template:
    spec:
      automountServiceAccountToken: false
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        runAsGroup: 1000
        fsGroup: 1000
        seccompProfile:
          type: RuntimeDefault
      containers:
        - name: my-app
          image: my-app@sha256:abc123...  # Pinned by digest
          securityContext:
            allowPrivilegeEscalation: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 500m
              memory: 256Mi

Least-privilege RBAC:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role  # Namespaced, not ClusterRole
metadata:
  name: my-app
  namespace: my-namespace
rules:
  - apiGroups: [""]
    resources: ["configmaps"]
    verbs: ["get", "list"]  # Only what's needed

Default-deny NetworkPolicy:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: my-app
spec:
  podSelector:
    matchLabels:
      app: my-app
  policyTypes: ["Ingress", "Egress"]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - port: 8080
  egress:
    - to:
        - podSelector:
            matchLabels:
              app: database
      ports:
        - port: 5432

How to catch it in review

• Run kubesec scan, kube-linter, or trivy config against manifests
• Search for privileged: true, allowPrivilegeEscalation: true (should almost never appear)
• Search for cluster-admin in RBAC bindings
• Check that every Deployment/StatefulSet has resources: limits and securityContext:
• Check that every namespace has at least one NetworkPolicy
• Search for image:.*:latest -- tags should be pinned to specific versions or digests
• Check for automountServiceAccountToken: false on pods that don't need the K8s API
• In Helm charts, verify values.yaml includes security defaults, not just functional defaults

---

6. Input Validation Gaps

What LLMs get wrong

LLMs generate code that handles the happy path but skips validation of types, lengths, formats, and ranges. They omit validation unless explicitly prompted, because training data (tutorials, examples) does the same. The model has no awareness of the threat model or what inputs are user-controlled.

Vulnerable patterns

No type/length validation (Python API):

@app.route('/api/users', methods=['POST'])
def create_user():
    data = request.get_json()
    username = data['username']  # No validation at all
    email = data['email']        # No format check
    age = data['age']            # No type or range check
    db.execute("INSERT INTO users (username, email, age) VALUES (%s, %s, %s)",
               (username, email, age))

No path validation (shell):

#!/bin/bash
# LLM-generated backup script
BACKUP_DIR="$1"
cp -r /important/data "$BACKUP_DIR"  # No validation of $1

Secure alternatives

Validated API input (Python):

from pydantic import BaseModel, EmailStr, Field

class CreateUserRequest(BaseModel):
    username: str = Field(min_length=3, max_length=50, pattern=r'^[a-zA-Z0-9_]+$')
    email: EmailStr
    age: int = Field(ge=0, le=150)

@app.route('/api/users', methods=['POST'])
def create_user():
    data = CreateUserRequest(**request.get_json())  # Validates or raises 422
    db.execute("INSERT INTO users (username, email, age) VALUES (%s, %s, %s)",
               (data.username, data.email, data.age))

Validated shell input:

#!/bin/bash
BACKUP_DIR="$1"
if [[ -z "$BACKUP_DIR" ]]; then
    echo "Error: backup directory required" >&2
    exit 1
fi
if [[ ! -d "$BACKUP_DIR" ]]; then
    echo "Error: '$BACKUP_DIR' is not a directory" >&2
    exit 1
fi
# Resolve and validate path
REAL_DIR="$(realpath -- "$BACKUP_DIR")"
if [[ "$REAL_DIR" != /allowed/backup/* ]]; then
    echo "Error: backup directory must be under /allowed/backup/" >&2
    exit 1
fi
cp -r /important/data "$REAL_DIR"

How to catch it in review

• Check that all API endpoints use schema validation (Pydantic, marshmallow, JSON Schema, Joi)
• Search for request.get_json(), request.args, request.form usage without subsequent validation
• In shell scripts, check that all positional parameters ($1, $2, etc.) are validated before use
• Look for direct use of user input in file operations, database queries, or system commands
• Verify that numeric inputs have range checks and string inputs have length/format checks

---

7. Error Handling That Leaks Information

What LLMs get wrong

LLMs generate code with verbose error handling that exposes internal details -- stack traces, file paths, database schemas, SQL queries, internal hostnames. This happens because training data includes development-mode error handling, and the model doesn't distinguish between dev and production contexts.

Vulnerable patterns

Leaking stack traces (Python/Flask):

@app.errorhandler(Exception)
def handle_error(e):
    return jsonify({
        "error": str(e),
        "traceback": traceback.format_exc(),  # Full stack trace
        "query": last_query,                   # SQL query that failed
    }), 500

Leaking database details:

try:
    cursor.execute(query)
except psycopg2.Error as e:
    return f"Database error: {e}"  # Includes table names, column names, query

Leaking file paths (shell):

echo "Error: failed to read config from /etc/myapp/secrets/database.yaml"
echo "Stack: $(python3 -c 'import traceback; traceback.print_exc()')"

Secure alternatives

Generic error response with internal logging:

import logging
logger = logging.getLogger(__name__)

@app.errorhandler(Exception)
def handle_error(e):
    logger.exception("Unhandled exception")  # Full details go to logs
    return jsonify({"error": "Internal server error"}), 500  # Generic to client

Safe database error handling:

try:
    cursor.execute(query, params)
except psycopg2.Error as e:
    logger.exception("Database query failed")
    return jsonify({"error": "A database error occurred"}), 500

How to catch it in review

• Search for traceback.format_exc() or traceback.print_exc() in response-building code
• Search for str(e) or repr(e) in API responses (should go to logs, not clients)
• Check that DEBUG = False / debug=False in production config
• Verify error handlers return generic messages and log details internally
• Search for internal paths (/etc/, /home/, /var/) in user-facing error strings

---

8. Cryptography Mistakes

What LLMs get wrong

LLMs reproduce cryptographic anti-patterns from training data. CWE-780 (Use of RSA without OAEP) is the most observed weakness in Java. Common failures include using ECB mode (which leaks patterns), predictable IVs, deprecated algorithms (MD5, SHA-1 for security purposes), and rolling custom crypto. Cryptography misconfiguration appears in approximately 22-24% of security vulnerabilities across leading LLM models.

Vulnerable patterns

ECB mode (Python):

from Crypto.Cipher import AES
cipher = AES.new(key, AES.MODE_ECB)  # ECB leaks patterns in ciphertext
ciphertext = cipher.encrypt(plaintext)

Hardcoded IV:

iv = b'\x00' * 16  # Predictable IV defeats the purpose of CBC/GCM
cipher = AES.new(key, AES.MODE_CBC, iv=iv)

MD5 for password hashing:

import hashlib
password_hash = hashlib.md5(password.encode()).hexdigest()  # Broken for security

Weak random for tokens:

import random
token = ''.join(random.choices(string.ascii_letters, k=32))  # Not cryptographically secure

Secure alternatives

AES-GCM with random IV:

from Crypto.Cipher import AES
from Crypto.Random import get_random_bytes

key = get_random_bytes(32)  # AES-256
nonce = get_random_bytes(12)  # Random nonce for GCM
cipher = AES.new(key, AES.MODE_GCM, nonce=nonce)
ciphertext, tag = cipher.encrypt_and_digest(plaintext)
# Store nonce + tag + ciphertext together

Proper password hashing:

import bcrypt
# Hashing
password_hash = bcrypt.hashpw(password.encode(), bcrypt.gensalt(rounds=12))
# Verification
bcrypt.checkpw(password.encode(), stored_hash)

Cryptographically secure random:

import secrets
token = secrets.token_urlsafe(32)  # Cryptographically secure

How to catch it in review

• Search for MODE_ECB -- should almost never be used
• Search for md5, sha1 used for passwords or security tokens (fine for checksums, not for security)
• Search for random. (stdlib) used for tokens, keys, or security values -- should be secrets.
• Search for hardcoded IVs: iv = b', iv = bytes(, nonce = b'\x00
• Search for hashlib used directly for password storage -- should be bcrypt, argon2, or scrypt
• Use bandit which has specific checks for weak crypto (B303, B304, B305)

---

9. Research Findings (2024-2026)

ACM / TOSEM: Security Weaknesses of Copilot-Generated Code in GitHub Projects

Analyzed real-world Copilot-generated code on GitHub. Found 29.5% of Python and 24.2% of JavaScript snippets contained security weaknesses across 43 CWE categories. Top weaknesses: CWE-330 (insufficiently random values), CWE-94 (code injection), CWE-79 (XSS).

Large-Scale GitHub Analysis (October 2025)

Analyzed 7,703 files from 4 AI tools across public GitHub repos. Found 4,241 CWE instances across 77 distinct vulnerability types. ChatGPT-generated code comprised 91.5% of the sample, Copilot 7.5%.

Slopsquatting Research (2025)

576,000 code samples across 16 LLMs: 19.7% of suggested packages were hallucinations (205,474 unique fake names). Open-source models hallucinated at 21.7%, commercial at 5.2%. 43% of hallucinated names appeared consistently (predictable, attackable).

Endor Labs: State of Dependency Management (2025)

80% of AI-suggested dependencies contain known risks. 44-49% of dependencies imported by coding agents contained known security vulnerabilities.

Copilot Code Review Study (2025)

GitHub Copilot's code review feature frequently fails to detect critical vulnerabilities (SQL injection, XSS, insecure deserialization). Primarily flags low-severity issues like coding style.

Sonar: Coding Personalities of Leading LLMs (2025)

Multi-model analysis finding that cryptography misconfiguration appears in 22-24% of vulnerabilities across leading models. Missing input sanitization is the most common flaw category.

GenKubeSec / KubeGuard (2024-2025)

Research on LLM-generated Kubernetes configurations found models confidently recommend insecure configurations and may suggest "allow all" rules to satisfy functional requirements.

OWASP Top 10 for LLM Applications (2025 Update)

Updated to reflect agentic AI risks. Key additions: System Prompt Leakage, Excessive Agency. Improper Output Handling (treating LLM output as trusted) remains a top-5 risk. Core message: treat all LLM output as untrusted data.

Security Degradation in Iterative Generation (2025)

Code security degrades with iterative LLM refinement -- each round of "fix this" prompting can introduce new vulnerabilities while fixing the original one.

---

10. Practical Review Checklist

Use this checklist when reviewing LLM-generated code:

Python

• No string formatting in SQL queries (use parameterised queries)
• No shell=True in subprocess calls
• No verify=False in HTTP requests
• No random. for security values (use secrets.)
• No hashlib.md5/sha1 for passwords (use bcrypt/argon2)
• No hardcoded credentials (search for password =, api_key =, token =)
• Input validation on all API endpoints (Pydantic, marshmallow)
• Error handlers return generic messages, log details internally
• debug=False in production config
• All dependencies exist on PyPI and are pinned to audited versions
• host="127.0.0.1" unless external binding is explicitly required

Shell Scripts

• All variables quoted ("$var" not $var)
• User-provided paths validated with realpath and boundary checks
• No secrets as command-line arguments (use files or env vars)
• No chmod 777 or chmod 666
• ShellCheck passes with no warnings

Kubernetes Manifests

• securityContext present with runAsNonRoot: true, readOnlyRootFilesystem: true, allowPrivilegeEscalation: false
• capabilities.drop: ["ALL"]
• resources.requests and resources.limits defined
• No privileged: true
• No cluster-admin RBAC bindings
• automountServiceAccountToken: false where K8s API access is not needed
• Images pinned to digest or specific version (not :latest)
• Services use ClusterIP by default (not LoadBalancer/NodePort without justification)
• NetworkPolicy exists for the namespace/workload

Helm Charts

• values.yaml includes secure defaults for securityContext, resources, service type
• Templates don't embed secrets in plaintext
• Chart version and appVersion pinned
• helm template renders valid, secure manifests with default values
• values.schema.json validates required security fields

---

11. Gated Model Downloads Require Out-of-Band License Acceptance

What goes wrong

HuggingFace (and similar model hubs) return 403 Forbidden for "gated" models even when the HTTP request carries a valid user token. The hub enforces that the token's user has manually accepted the license agreement on the web UI for that specific model. Automation cannot bypass this — there is no API to accept the license.

This breaks reproducible-build scripts, container bake pipelines, and agent workflows that pull third-party ML models: the first run on a fresh account/token fails with an opaque 403 and no hint that human action is required.

Pattern

Bake a pre-flight check into every model-download script:

def preflight_gated_model(model_id: str, token: str) -> None:
    r = requests.head(f"https://huggingface.co/{model_id}/resolve/main/config.json",
                      headers={"Authorization": f"Bearer {token}"},
                      allow_redirects=True)
    if r.status_code == 403:
        url = f"https://huggingface.co/{model_id}"
        raise SystemExit(
            f"Model {model_id} is gated. Open {url} in a browser, sign in as "
            f"the token owner, accept the license, then rerun this script."
        )
    r.raise_for_status()

Rules

• Surface a human-readable error on 403 — do not retry, do not fall back to a different model silently
• Include the exact URL to visit and the exact action required ("accept the license")
• Check every gated model at pipeline start, not lazily at download time, so the human step is front-loaded
• Document which models are gated in the project's README — license acceptance is per-user, so every new operator needs to do it once

Applies to any project pulling third-party ML models from HuggingFace, Meta's Llama portal, Stability AI's hub, or similar.

---

12. Operational Vulnerabilities in AI-Generated Code (Beyond Traditional SAST)

Traditional SAST tools (Bandit, Semgrep, SonarQube, Snyk) focus on known vulnerability patterns — injection, XSS, hardcoded secrets. But analysis of AI-generated code at scale (538,860 findings across 3,518 scans, SentinaLayer 2026) reveals that the top vulnerability categories are structural and operational, not traditional:

Category	% of P0-P2 Findings	What SAST Misses
CI/CD Integrity Gaps	31%	Gate ordering, missing dependency chains, workflow step sequencing
Backend Reliability	27%	Missing idempotency keys, premature health checks, retry logic gaps
Security Overlay	24%	Supply chain trust gaps, unverified checksums, unsigned artifacts
Supply Chain Provenance	11%	Missing signature verification, unpinned base images in multi-stage builds
Data Layer Integrity	7%	Unsafe write paths, missing concurrent-access guards, race conditions

The single most common P0-P2 finding: missing idempotency keys in webhook handlers (18% of all critical/high/medium findings). This is not a vulnerability that any SAST tool checks for.

Why AI agents produce these

AI coding agents optimise for functional correctness — does the code produce the right output for the happy path? They consistently miss:

1. Retry safety. Webhook handlers, API endpoints, and event processors that work correctly on first invocation but corrupt data on retry. Agents don't model what happens when the same request arrives twice.

2. Ordering dependencies. CI/CD pipelines where steps depend on prior steps' artifacts but the dependency isn't explicit. Works when steps happen to run in order, breaks under parallelism or partial failure.

3. Health check timing. Services that report healthy before their dependencies are ready. The agent sees "return 200 from /health" and implements it, without considering that the database connection pool hasn't warmed yet.

4. Checksum and signature verification. Agents download artifacts, pull images, and install packages without verifying integrity. The code works, but the supply chain is unverified.

5. Concurrent write safety. File operations, database writes, and cache updates that work under single-threaded testing but corrupt under concurrent access. Agents don't think about lock ordering or write-after-read races.

Deterministic checks you can add to CI

These can be implemented as fast pre-scan rules (regex + AST) that run before expensive test suites:

Idempotency:

• Webhook handlers without idempotency key extraction/dedup
• POST endpoints that create resources without checking for existing duplicates
• Event processors without at-least-once safety (no dedup by event ID)

CI/CD Integrity:

• GitHub/Gitea Actions steps that reference artifacts from prior steps without explicit needs:
• Dockerfile COPY --from= referencing stages without explicit ordering
• Helm hooks without hook-weight when ordering matters

Supply Chain:

• curl | bash or wget -O- | sh without checksum verification
• Container images pulled by tag without digest pinning
• Package installation without lockfile or hash verification
• go install / pip install from URLs without integrity checks

Health Checks:

• HTTP health endpoints that return 200 unconditionally (no dependency readiness check)
• Liveness probes identical to readiness probes (should be different — liveness checks "am I stuck?", readiness checks "am I ready to serve?")
• initialDelaySeconds: 0 on readiness probes for services with startup dependencies

Concurrent Access:

• File writes without advisory locking (fcntl.flock / flock)
• Database read-then-write sequences without transactions or SELECT FOR UPDATE
• Cache operations without atomic compare-and-swap

Review checklist (operational)

• Every webhook handler extracts and deduplicates by an idempotency key
• Every POST endpoint that creates resources checks for pre-existing duplicates
• CI/CD steps declare explicit dependencies on prior steps' outputs
• Downloaded artifacts are verified by checksum or signature before use
• Container base images are pinned by digest, not just tag
• Health check endpoints verify dependency readiness, not just "process is alive"
• Readiness and liveness probes serve different purposes
• File and database writes under concurrent access use appropriate locking
• Event processors handle at-least-once delivery (idempotent or deduplicating)
• Multi-stage Docker builds have explicit stage ordering and artifact dependencies

---

Sources

• Security Weaknesses of Copilot-Generated Code in GitHub Projects (ACM TOSEM)
• Security Vulnerabilities in AI-Generated Code: A Large-Scale Analysis (arXiv, Oct 2025)
• The Most Common Security Vulnerabilities in AI-Generated Code (Endor Labs)
• Endor Labs 2025 State of Dependency Management Report
• LLMs' AI-Generated Code Remains Wildly Insecure (Dark Reading)
• Popular LLMs Found to Produce Vulnerable Code by Default (Infosecurity Magazine)
• Slopsquatting: How AI Hallucinations Are Fueling Supply Chain Attacks (Socket.dev)
• Slopsquatting meets Dependency Confusion (Andrew Nesbitt)
• AI-Generated Code Packages Can Lead to Slopsquatting Threat (DevOps.com)
• OWASP Top 10 for LLM Applications 2025
• OWASP LLM Top 10: How it Applies to Code Generation (Sonar)
• The Coding Personalities of Leading LLMs (SonarSource)
• GenKubeSec: LLM-Based Kubernetes Misconfiguration Detection
• KubeGuard: LLM-Assisted Kubernetes Hardening
• Security Degradation in Iterative AI Code Generation (arXiv)
• GitHub Copilot's Code Review: Can AI Spot Security Flaws? (arXiv)
• Security Risks of Vibe Coding and LLM Assistants (Kaspersky)
• The Risks of Hardcoding Secrets in Code Generated by LLMs (Cycode)
• Security Flaws in DeepSeek-Generated Code (CrowdStrike)