8aa400a5d4
Add 37 new entries and update 7 existing entries across 13 topic files. Major contributions from agent-runtimes (K8s secrets, CI, Docker gotchas), cluster-bootstrap (ArgoCD SSA, etcd tuning, DB migrations, Compose networking), and cluster-apps/octopus-deploy (Helm vs raw manifests, ArgoCD source types). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
38 lines
2.8 KiB
Markdown
38 lines
2.8 KiB
Markdown
# Networking & Infrastructure
|
|
|
|
## nftables Flush Ruleset on Remote Hosts
|
|
|
|
On remote hosts, `nftables flush ruleset` followed by a failed rule load leaves the host with NO firewall. SSH survives only on existing connections — new connections are blocked or allowed depending on the default policy.
|
|
|
|
**Always validate rules before applying:** `nft -c -f <rulefile>` does a dry-run parse. For extra safety, deploy a cron-based auto-rollback timer that reverts rules unless explicitly confirmed (similar to `shutdown -c` pattern).
|
|
|
|
## systemd Socket Activation Overrides Config File Ports
|
|
|
|
On modern Linux systems (Ubuntu 24.04+), systemd socket activation controls the listening port for services like SSH. Editing the service config file alone (e.g., `sshd_config Port 2222`) has no effect — the socket unit still binds the original port.
|
|
|
|
**Check socket activation first:** `systemctl cat <service>.socket` shows whether socket activation is in play. If so, override the socket unit's `ListenStream` directive, not the service config.
|
|
|
|
## Docker Sets iptables FORWARD Policy to DROP
|
|
|
|
Docker sets the iptables FORWARD chain default policy to DROP. This affects ALL forwarding on the host, not just Docker traffic. Non-Docker forwarding (VPN, VM bridges, custom NAT) silently breaks.
|
|
|
|
**Fix:** Add explicit ACCEPT rules in the `DOCKER-USER` chain for non-Docker forwarding needs. This chain is processed before Docker's own rules and persists across Docker restarts.
|
|
|
|
## HTTP Host Header vs TLS SNI Are Different Layers
|
|
|
|
When proxying to a backend over HTTPS, two independent identifiers must be set correctly:
|
|
- **TLS SNI** — sent during the TLS handshake, used for certificate selection. Missing SNI causes `x509: cannot validate certificate for <IP>`.
|
|
- **HTTP Host header** — sent after TLS is established, used for virtual host routing. Missing or wrong Host header causes 404 from the backend.
|
|
|
|
A reverse proxy must set both. They often need to be the same value, but they're configured independently.
|
|
|
|
## Wildcard Certs in Auto-Renewing Proxies
|
|
|
|
Auto-renewing proxies (Caddy, Traefik with Let's Encrypt, etc.) that also support file-loaded certificates treat file-loaded certs as globally available. A wildcard cert loaded for one site block will match ALL matching subdomains, silently preventing automatic certificate issuance for other sites.
|
|
|
|
**Rule:** Use automatic certificate management for all sites. Don't mix file-loaded and automatic certs unless you understand the matching priority.
|
|
|
|
## Cilium DNAT Resolves LB VIP Before NetworkPolicy Evaluation
|
|
|
|
Cilium performs DNAT on LoadBalancer VIP traffic before evaluating NetworkPolicy. Traffic to a VIP is rewritten to a backend pod IP before the policy check. For egress to LoadBalancer services in CiliumNetworkPolicy, use `toEndpoints` targeting the backend pods (by namespace/label), not `toCIDR` targeting the VIP.
|