09 — Infrastructure and deployment

Layers
| Layer | Tool | Scope |
|---|---|---|
| Hardware / cloud | Pulumi (real providers) | VPS, volumes, firewall, DNS, frp config, Coolify install |
| Runtime & routing | Coolify (Docker + Traefik) | Projects, apps, Compose services, TLS, deployments |
| Tenant composition | Control plane + Pulumi stacks with adapters | Per-tenant resources, via Coolify's REST API |
The separation follows the discussion's conclusion: do not mutate Coolify's runtime behind its back; use its API for everything it manages.
Coolify facts that constrain the design
- Self-hosted REST API under
/api/v1; API must be enabled; bearer tokens are bound to a user/team withread/write/deploy/deletepermissions and project/environment/resource access. - A restricted write token is not a server security boundary (Compose and bind mounts are privileged). Treat the Coolify token as root-equivalent on the host: only the worker holds it, rotate on schedule.
- Webhook/Git auto-deploy are not used for tenants; the CP triggers deployments.
Projects (Coolify) ↔ tenancy
| Project | Contents |
|---|---|
core-infra | Zitadel (+ login), CP api/worker, console, Infisical, LiveKit, Egress, RustFS, core Postgres/Redis |
tenant-data-pool | Postgres shards + PgBouncer |
tenant-<slug> | go-meet, redis, tuwunel (+ future modules) |
Network: office-suite-net (external Docker network from the drafts) is used by
core-infra. Tenant projects get their own network; the only path to the shared
plane is via published/internal service names the stack is allowed to reach
(control plane internal API, Zitadel, RustFS, the shard host). On a single host
these names resolve over the Docker bridge; on multiple hosts, over the
private network.
Edge and ports (diagram 10)
- DNS (Cloudflare):
*.<platform-domain>→ public VPS;livekit.<platform-domain>DNS-only (UDP does not traverse Cloudflare's proxy). - frp:
frpcon the private host connects out tofrpson the VPS and forwards TCP 80/443 → Traefik, TCP 7881 and UDP 7882 → LiveKit. - LiveKit:
use_external_ip: false;node_ipis resolved from the A record ofLIVEKIT_DOMAINbylivekit-init(draftcompose.livekit.yaml), because STUN would discover the wrong address behind frp. - Traefik terminates TLS (Let's Encrypt). Zitadel needs
h2cto the API container and a path split for login v2 (/ui/v2/loginpriority 100), as incompose.zitadel.yaml. - Certificates: per-tenant hostnames under one wildcard (DNS-01 via the DNS provider) avoid per-host rate limits; otherwise per-host HTTP-01.
Reference of what exists vs. what to change
| Draft file | State | Required change |
|---|---|---|
compose.livekit.yaml | LiveKit, init container, Egress | Remove global RUSTFS_* from Egress; webhook URL → CP router; keys held by broker |
compose.meet.yaml | La Suite Django backend + frontend (Keycloak OIDC) | Superseded by go-meet tenant template; keep as parity reference |
compose.zitadel.yaml | Zitadel + login v2, bootstrap PAT volume | Keep; add platform org bootstrap + Fortspace project via provisioner |
compose.keycloak.yaml | Keycloak | Deprecated |
compose.yaml | Original monolith stack (Django meet) | Reference only |
postgres/01..04_*.sh | Schemas per product | Add 05–07 (control plane, infisical, pulumi); move meet to tenant pool |
livekit-server.yaml | Keys via LIVEKIT_KEYS, Redis TLS via sed | Add webhook: block pointing at the CP router; add room defaults |
go-meet dev-local/ | Local compose (Postgres, RustFS, LiveKit, Keycloak/Zitadel, Traefik) | Keep for dev; mirror tenant template |
Tenant Compose template (sketch)
# rendered per tenant by the provisioner; only non-secret values inlined
services:
go-meet:
image: ghcr.io/<org>/go-meet:${GOMEET_VERSION}
entrypoint: ["infisical","run","--path","/tenants/${SLUG}/meet","--","/go-meet"]
environment:
INFISICAL_CLIENT_ID: ${INFISICAL_CLIENT_ID}
INFISICAL_CLIENT_SECRET: ${INFISICAL_CLIENT_SECRET}
OIDC_USE_PKCE: "true"
LIVEKIT_BROKER_URL: https://cp.${PLATFORM_DOMAIN}/internal/v1
networks: [tenant, shared-egress]
redis:
image: redis:7
command: ["redis-server","--requirepass","${REDIS_PASSWORD}"]
networks: [tenant]
tuwunel:
image: ghcr.io/matrix-construct/tuwunel:${TUWUNEL_VERSION}
volumes: ["tuwunel-data:/var/lib/tuwunel"]
networks: [tenant]
networks: { tenant: {}, shared-egress: { external: true } }
volumes: { tuwunel-data: {} }
Environments
| Env | Purpose | Notes |
|---|---|---|
| local | Developer machine | go-meet/dev-local/ compose (+ mock mode); CP with docker-compose + Zitadel dev |
| staging | Full topology, small | One VPS pair; acceptance tests listed in go-meet STAGING-ACCEPTANCE.md |
| production | EU-resident | Public VPS + private host(s); regional pinning in tenant row |
Scaling plan
| Axis | Now | Next |
|---|---|---|
| Tenants per host | tens | Add Coolify servers; tenants record the server in their orchestrator reference |
| Postgres | 1 shard | Add shards on demand, relocate large tenants |
| LiveKit | 1 node | N nodes need distinct public media endpoints; Redis already shared; consider moving media off frp |
| Egress | 1 worker | Scale workers horizontally (CPU bound) |
| Control plane | 2 API + 2 workers | Split broker as its own deployment if join latency demands |
Migration path off Coolify
Store only generic references in tenants; implement DokployAdapter or a
Kubernetes adapter; migrate tenant by tenant: dump DB, deploy on the new
engine, restore volumes, switch DNS, destroy the old stack. Base infra stays
untouched in Pulumi.