best-practices/database-selection.md

Database Selection

The Rule: SQLite Is Not a Production Database

Any service that meets ANY of the following criteria MUST use PostgreSQL (or equivalent server-grade database) from day one:

• Attached to a FQDN (has a real domain name, even internal)
• Serves traffic from more than one process (API consumers, CI runners, webhooks, polling)
• Backs infrastructure that other systems depend on (Git hosting, container registries, auth providers)
• Will be accessed concurrently by automated systems (ArgoCD, CI runners, cron jobs)

Do not use SQLite for these workloads. Not temporarily. Not "to start with." Not "we'll migrate later."

SQLite uses file-level locking — only one writer at a time, and writes block reads. Under concurrent access, requests queue up waiting for the write lock, causing cascading timeouts. The failure mode is insidious: the service appears to work fine under light load but becomes intermittently unresponsive under real workloads. By the time you notice, everything that depends on it is also failing.

The Cost of "We'll Migrate Later"

The Gitea SQLite→PostgreSQL migration (2026-03-28) cost nearly a full day of productivity:

• Hours of accumulated unresponsiveness across multiple projects before root cause was identified
• Planning and implementation of the migration itself
• Migration complexity that didn't need to exist: Gitea 1.23 has no restore command, doctor convert only handles charset conversion, docker cp corrupted PostgreSQL directory permissions, SSH authorized_keys weren't regenerated
• Downstream impact on ArgoCD (20 apps polling a locked database), CI runners (continuous 500 errors), container registry pulls (timeouts)

The PostgreSQL container takes 5 minutes to add to a Docker Compose stack at initial setup time. The migration took a day. Always pay the 5 minutes upfront.

When SQLite Is Acceptable

SQLite is fine for:

• Local development databases (single developer, single process)
• Embedded application data stores (mobile apps, desktop apps, CLI tools)
• Read-heavy workloads with rare writes and a single writer process
• Test fixtures and throwaway data
• Configuration stores read at startup (not at request time)

Implementation Pattern

For Docker Compose services that need a database:

services:
  postgres:
    image: postgres:17-alpine
    restart: unless-stopped
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: myapp
      POSTGRES_PASSWORD: {{ db_password }}
    volumes:
      - /opt/postgres-myapp:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U myapp -d myapp"]
      interval: 10s
      timeout: 5s
      retries: 5
      start_period: 30s
    networks:
      - app_internal
    deploy:
      resources:
        limits:
          memory: 1G

  myapp:
    depends_on:
      postgres:
        condition: service_healthy
    networks:
      - app_internal
      - external_network

networks:
  app_internal:
    driver: bridge
    internal: true

Key points:

• PostgreSQL on an internal bridge network (no external access needed)
• Application depends on PostgreSQL health before starting
• Resource limits to prevent runaway memory usage
• Separate data directory per application (/opt/postgres-myapp, not shared)
• PostgreSQL container UID is 999 (not 1000) — set directory ownership accordingly

For Kubernetes Deployments

Use the application's Helm chart PostgreSQL subchart, or deploy a standalone PostgreSQL instance:

• Bitnami PostgreSQL Helm chart for simple deployments
• CloudNativePG operator for production-grade PostgreSQL with HA, backups, and failover
• Never use SQLite with emptyDir or even PVC-backed volumes in multi-replica deployments

Checklist for New Service Deployment

Before deploying any new service, check:

1. What database does the default configuration use?
2. If SQLite: does the service support PostgreSQL? (Almost all do — Gitea, Authelia, Headscale, Zulip, etc.)
3. Switch to PostgreSQL before the first deployment, not after problems appear
4. Add the database password to SOPS-encrypted secrets
5. Verify the database connection works before adding consumers