Secure Multi-Tenant Cloud Deployment Foundation

The constraint

Onboarding a new tenant took roughly three weeks of tickets across networking, security and platform teams. Every tenant ended up subtly different, which meant no two incidents looked alike and nothing could be automated safely.

Approach

Isolation boundary = GCP project, not namespace or VPC. Projects give you a clean IAM boundary, independent quota, and — critically — the ability to delete everything a tenant touched by deleting one object.

Guardrails are inherited, not applied. Organisation policy constraints at the folder level mean a tenant project cannot violate the rules even if someone with project-owner rights tries. Constraints applied per-project are configuration; constraints inherited from a folder are architecture.

Enforced at the folder level: no external IPs on VMs, no service account key creation, uniform bucket-level access required, and allowed regions pinned for residency.

Architecture

A Terraform module takes a tenant name and tier, then produces: project, VPC with no default routes to the internet, Cloud NAT for controlled egress, per-tenant service accounts with least-privilege bindings, budget alert, and a logging sink to the central security project.

CI runs plan on every PR and posts the diff as a comment — so a reviewer sees the infrastructure change in the same place they review code. apply runs only on merge, from a service account authenticated via Workload Identity Federation. There are no service account JSON keys anywhere in the pipeline.

State lives in a GCS bucket with versioning and object-level locking, one prefix per tenant. Blast radius of a bad apply is one tenant.

What broke

The first iteration let tenants request IAM roles through a variable. Within a month someone had requested roles/owner and a reviewer had approved it without reading the diff closely — which is exactly the failure mode policy-as-code is supposed to prevent.

Replaced with a fixed enum of tier-based role bundles. You pick standard, data or ml; you cannot pick arbitrary roles. Removing flexibility was the security improvement — a conclusion that is unpopular right up until the first near-miss.

Outcome

Tenant provisioning fell from three weeks to about twelve minutes. Zero long-lived credentials. Every tenant identical, so runbooks actually apply and incidents are comparable.