101 lines
4.1 KiB
Markdown
101 lines
4.1 KiB
Markdown
# Database Selection
|
|
|
|
## The Rule: SQLite Is Not a Production Database
|
|
|
|
**Any service that meets ANY of the following criteria MUST use PostgreSQL (or equivalent server-grade database) from day one:**
|
|
|
|
- Attached to a FQDN (has a real domain name, even internal)
|
|
- Serves traffic from more than one process (API consumers, CI runners, webhooks, polling)
|
|
- Backs infrastructure that other systems depend on (Git hosting, container registries, auth providers)
|
|
- Will be accessed concurrently by automated systems (ArgoCD, CI runners, cron jobs)
|
|
|
|
**Do not use SQLite for these workloads. Not temporarily. Not "to start with." Not "we'll migrate later."**
|
|
|
|
SQLite uses file-level locking — only one writer at a time, and writes block reads. Under concurrent access, requests queue up waiting for the write lock, causing cascading timeouts. The failure mode is insidious: the service appears to work fine under light load but becomes intermittently unresponsive under real workloads. By the time you notice, everything that depends on it is also failing.
|
|
|
|
## The Cost of "We'll Migrate Later"
|
|
|
|
The Gitea SQLite→PostgreSQL migration (2026-03-28) cost nearly a full day of productivity:
|
|
|
|
- **Hours of accumulated unresponsiveness** across multiple projects before root cause was identified
|
|
- **Planning and implementation** of the migration itself
|
|
- **Migration complexity** that didn't need to exist: Gitea 1.23 has no `restore` command, `doctor convert` only handles charset conversion, `docker cp` corrupted PostgreSQL directory permissions, SSH authorized_keys weren't regenerated
|
|
- **Downstream impact** on ArgoCD (20 apps polling a locked database), CI runners (continuous 500 errors), container registry pulls (timeouts)
|
|
|
|
The PostgreSQL container takes 5 minutes to add to a Docker Compose stack at initial setup time. The migration took a day. Always pay the 5 minutes upfront.
|
|
|
|
## When SQLite Is Acceptable
|
|
|
|
SQLite is fine for:
|
|
- Local development databases (single developer, single process)
|
|
- Embedded application data stores (mobile apps, desktop apps, CLI tools)
|
|
- Read-heavy workloads with rare writes and a single writer process
|
|
- Test fixtures and throwaway data
|
|
- Configuration stores read at startup (not at request time)
|
|
|
|
## Implementation Pattern
|
|
|
|
For Docker Compose services that need a database:
|
|
|
|
```yaml
|
|
services:
|
|
postgres:
|
|
image: postgres:17-alpine
|
|
restart: unless-stopped
|
|
environment:
|
|
POSTGRES_DB: myapp
|
|
POSTGRES_USER: myapp
|
|
POSTGRES_PASSWORD: {{ db_password }}
|
|
volumes:
|
|
- /opt/postgres-myapp:/var/lib/postgresql/data
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pg_isready -U myapp -d myapp"]
|
|
interval: 10s
|
|
timeout: 5s
|
|
retries: 5
|
|
start_period: 30s
|
|
networks:
|
|
- app_internal
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
memory: 1G
|
|
|
|
myapp:
|
|
depends_on:
|
|
postgres:
|
|
condition: service_healthy
|
|
networks:
|
|
- app_internal
|
|
- external_network
|
|
|
|
networks:
|
|
app_internal:
|
|
driver: bridge
|
|
internal: true
|
|
```
|
|
|
|
Key points:
|
|
- PostgreSQL on an **internal bridge network** (no external access needed)
|
|
- Application **depends on PostgreSQL health** before starting
|
|
- **Resource limits** to prevent runaway memory usage
|
|
- **Separate data directory** per application (`/opt/postgres-myapp`, not shared)
|
|
- PostgreSQL container UID is **999** (not 1000) — set directory ownership accordingly
|
|
|
|
## For Kubernetes Deployments
|
|
|
|
Use the application's Helm chart PostgreSQL subchart, or deploy a standalone PostgreSQL instance:
|
|
- Bitnami PostgreSQL Helm chart for simple deployments
|
|
- CloudNativePG operator for production-grade PostgreSQL with HA, backups, and failover
|
|
- Never use SQLite with `emptyDir` or even PVC-backed volumes in multi-replica deployments
|
|
|
|
## Checklist for New Service Deployment
|
|
|
|
Before deploying any new service, check:
|
|
|
|
1. What database does the default configuration use?
|
|
2. If SQLite: does the service support PostgreSQL? (Almost all do — Gitea, Authelia, Headscale, Zulip, etc.)
|
|
3. Switch to PostgreSQL **before the first deployment**, not after problems appear
|
|
4. Add the database password to SOPS-encrypted secrets
|
|
5. Verify the database connection works before adding consumers
|