Skip to main content

09 — Infrastructure and deployment

Infrastructure topology

Layers​

LayerToolScope
Hardware / cloudPulumi (real providers)VPS, volumes, firewall, DNS, frp config, Coolify install
Runtime & routingCoolify (Docker + Traefik)Projects, apps, Compose services, TLS, deployments
Tenant compositionControl plane + Pulumi stacks with adaptersPer-tenant resources, via Coolify's REST API

The separation follows the discussion's conclusion: do not mutate Coolify's runtime behind its back; use its API for everything it manages.

Coolify facts that constrain the design​

  • Self-hosted REST API under /api/v1; API must be enabled; bearer tokens are bound to a user/team with read/write/deploy/delete permissions and project/environment/resource access.
  • A restricted write token is not a server security boundary (Compose and bind mounts are privileged). Treat the Coolify token as root-equivalent on the host: only the worker holds it, rotate on schedule.
  • Webhook/Git auto-deploy are not used for tenants; the CP triggers deployments.

Projects (Coolify) ↔ tenancy​

ProjectContents
core-infraZitadel (+ login), CP api/worker, console, Infisical, LiveKit, Egress, RustFS, core Postgres/Redis
tenant-data-poolPostgres shards + PgBouncer
tenant-<slug>go-meet, redis, tuwunel (+ future modules)

Network: office-suite-net (external Docker network from the drafts) is used by core-infra. Tenant projects get their own network; the only path to the shared plane is via published/internal service names the stack is allowed to reach (control plane internal API, Zitadel, RustFS, the shard host). On a single host these names resolve over the Docker bridge; on multiple hosts, over the private network.

Edge and ports (diagram 10)​

  • DNS (Cloudflare): *.<platform-domain> → public VPS; livekit.<platform-domain> DNS-only (UDP does not traverse Cloudflare's proxy).
  • frp: frpc on the private host connects out to frps on the VPS and forwards TCP 80/443 → Traefik, TCP 7881 and UDP 7882 → LiveKit.
  • LiveKit: use_external_ip: false; node_ip is resolved from the A record of LIVEKIT_DOMAIN by livekit-init (draft compose.livekit.yaml), because STUN would discover the wrong address behind frp.
  • Traefik terminates TLS (Let's Encrypt). Zitadel needs h2c to the API container and a path split for login v2 (/ui/v2/login priority 100), as in compose.zitadel.yaml.
  • Certificates: per-tenant hostnames under one wildcard (DNS-01 via the DNS provider) avoid per-host rate limits; otherwise per-host HTTP-01.

Reference of what exists vs. what to change​

Draft fileStateRequired change
compose.livekit.yamlLiveKit, init container, EgressRemove global RUSTFS_* from Egress; webhook URL → CP router; keys held by broker
compose.meet.yamlLa Suite Django backend + frontend (Keycloak OIDC)Superseded by go-meet tenant template; keep as parity reference
compose.zitadel.yamlZitadel + login v2, bootstrap PAT volumeKeep; add platform org bootstrap + Fortspace project via provisioner
compose.keycloak.yamlKeycloakDeprecated
compose.yamlOriginal monolith stack (Django meet)Reference only
postgres/01..04_*.shSchemas per productAdd 05–07 (control plane, infisical, pulumi); move meet to tenant pool
livekit-server.yamlKeys via LIVEKIT_KEYS, Redis TLS via sedAdd webhook: block pointing at the CP router; add room defaults
go-meet dev-local/Local compose (Postgres, RustFS, LiveKit, Keycloak/Zitadel, Traefik)Keep for dev; mirror tenant template

Tenant Compose template (sketch)​

# rendered per tenant by the provisioner; only non-secret values inlined
services:
go-meet:
image: ghcr.io/<org>/go-meet:${GOMEET_VERSION}
entrypoint: ["infisical","run","--path","/tenants/${SLUG}/meet","--","/go-meet"]
environment:
INFISICAL_CLIENT_ID: ${INFISICAL_CLIENT_ID}
INFISICAL_CLIENT_SECRET: ${INFISICAL_CLIENT_SECRET}
OIDC_USE_PKCE: "true"
LIVEKIT_BROKER_URL: https://cp.${PLATFORM_DOMAIN}/internal/v1
networks: [tenant, shared-egress]
redis:
image: redis:7
command: ["redis-server","--requirepass","${REDIS_PASSWORD}"]
networks: [tenant]
tuwunel:
image: ghcr.io/matrix-construct/tuwunel:${TUWUNEL_VERSION}
volumes: ["tuwunel-data:/var/lib/tuwunel"]
networks: [tenant]
networks: { tenant: {}, shared-egress: { external: true } }
volumes: { tuwunel-data: {} }

Environments​

EnvPurposeNotes
localDeveloper machinego-meet/dev-local/ compose (+ mock mode); CP with docker-compose + Zitadel dev
stagingFull topology, smallOne VPS pair; acceptance tests listed in go-meet STAGING-ACCEPTANCE.md
productionEU-residentPublic VPS + private host(s); regional pinning in tenant row

Scaling plan​

AxisNowNext
Tenants per hosttensAdd Coolify servers; tenants record the server in their orchestrator reference
Postgres1 shardAdd shards on demand, relocate large tenants
LiveKit1 nodeN nodes need distinct public media endpoints; Redis already shared; consider moving media off frp
Egress1 workerScale workers horizontally (CPU bound)
Control plane2 API + 2 workersSplit broker as its own deployment if join latency demands

Migration path off Coolify​

Store only generic references in tenants; implement DokployAdapter or a Kubernetes adapter; migrate tenant by tenant: dump DB, deploy on the new engine, restore volumes, switch DNS, destroy the old stack. Base infra stays untouched in Pulumi.