Terraform & IaC

Terraform module structure for multi-environment AWS

A layout for Terraform that keeps dev, staging and production consistent without copy-paste: modules, environment roots, state per account, and the review flow that keeps it safe.

13 Oct 20266 min read

Most Terraform code I am asked to fix has the same history. It started as one directory for one environment. Then staging was needed, so the directory was copied. Then production. A year later the three copies have drifted, nobody is sure which differences are intentional, and every change has to be made three times. This is the structure I use instead, and how I move existing code into it.

The goals

A multi-environment layout should give you:

  • One definition, several instances. Staging and production are built from the same code. Differences are explicit inputs, not edits in a copied file.
  • Small blast radius. A change to the DNS layer cannot destroy a database. A mistake in dev cannot touch production.
  • Promotion. A change is applied to dev, then staging, then production, with the same code at each step.
  • Readable plans. A reviewer can see exactly what will change, in which environment, before it happens.

Accounts first

On AWS, I put each environment in its own account: dev, staging and production at least, plus a shared account for things like DNS and container registries. AWS Organizations and IAM Identity Center make this cheap to run.

Separate accounts are the strongest isolation AWS offers. Permissions, quotas and billing are separated by default, and a wrong role in a pipeline cannot reach production resources because it does not have credentials there. Folders and workspaces inside one account do not give you that.

The layout

modules/
  network/          # VPC, subnets, NAT, endpoints
  eks-cluster/      # cluster, node groups, add-ons
  rds-postgres/     # database, parameter group, alarms
  service/          # one application's IAM, queues, buckets
live/
  dev/
    network/        # main.tf calls modules/network with dev inputs
    eks/
    data/
  staging/
    network/
    eks/
    data/
  prod/
    network/
    eks/
    data/

Two kinds of directories, with different rules:

  • modules/ contain resources and logic, and no environment names. They take inputs, such as CIDR ranges, instance sizes and the number of replicas, and expose outputs. They have no backend and no provider credentials.
  • live/<env>/<component>/ are root modules: the directories you run terraform plan in. Each one is short. It configures the backend and provider for its account, calls one or more modules with that environment's values, and nothing else.

Split state by component

Each directory under live/ has its own state. I split by how often things change and how much damage a mistake can do:

  • network: changes rarely, everything depends on it;
  • cluster or compute: changes with upgrades;
  • data: databases and buckets, protected with prevent_destroy and deletion protection;
  • services: change often, owned by application teams.

A small state plans in seconds and can only break what it contains. Components read what they need from each other through outputs, either with terraform_remote_state or, better, by looking resources up by tag or name with data sources, so the dependency is on the resource and not on another state file's layout.

State lives in S3 in each environment's own account, with locking and versioning. Recent Terraform versions lock with a lock file in the same bucket, so a separate DynamoDB table is no longer needed.

Modules worth having

The temptation is to wrap every AWS resource in a module. That produces a thin layer that hides nothing and must be updated with every provider release. I write modules for units that mean something to the team:

  • "A network", not "a subnet": VPC, subnets in three zones, routing, NAT and endpoints, with sensible defaults.
  • "A Postgres database for a service": the instance or cluster, parameter group, subnet group, security group, backups, alarms and the secret.
  • "A service": its IAM role, queues, buckets and the permissions between them.

A good module has few required inputs, safe defaults (encryption on, public access off, deletion protection on for data), and outputs that other components need. Where a well-maintained community module already does the job, such as the VPC or EKS modules, I use it pinned to a version rather than rewriting it.

Versioning and promotion

Modules are versioned. In a mono-repo that can be a Git tag referenced by ?ref=; in a larger organisation, a private module registry. Each environment pins a version:

module "network" {
  source = "git::https://gitlab.example.com/platform/terraform-modules.git//network?ref=network-v1.4.0"

  name       = "prod"
  cidr_block = "10.30.0.0/16"
  azs        = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
}

Promotion is then a one-line change: dev moves to network-v1.5.0 first, then staging, then production. The diff between environments is visible in Git, and rolling back is reverting that line.

For a small team with one repository, relative paths (source = "../../../modules/network") are fine, as long as changes are still applied environment by environment.

Keeping the root modules short

The copy-paste problem comes back if root modules grow. A few rules keep them thin:

  • Environment differences are inputs. Sizes, counts, CIDRs and feature flags go into the module call or a small terraform.tfvars. No if env == "prod" logic inside modules; pass the value instead.
  • Providers carry the account. Each root assumes a role in its environment's account and sets default tags (environment, owner, cost centre), so every resource is tagged without anyone remembering to.
  • Pin versions of Terraform and providers in every root, and upgrade them deliberately, environment by environment.

Wrappers like Terragrunt can generate the backend and provider blocks for you. They are useful with dozens of roots; with a handful, plain Terraform and a short template are easier for everyone to read.

The review and apply flow

Structure only helps if changes go through it:

  1. A merge request changes a module or a root.
  2. CI runs terraform fmt -check, validate, a linter and a security scanner, then plan for every root the change touches, and posts the plans on the merge request.
  3. A reviewer reads the plan, not only the code.
  4. After merge, CI applies the saved plan to dev automatically; staging and production apply after a manual approval, using the plan that was reviewed.

Nobody applies from a laptop, and nobody clicks in the console. If something has to be changed by hand during an incident, it is imported or reverted the next day, and a scheduled drift check (plan with no changes expected) shows anything that was missed.

Moving existing code into this layout

You rarely start from zero. The migration I use keeps resources in place:

  1. Pick one environment's copy as the reference, usually production, and turn it into modules with the same resources.
  2. Move state, not resources. moved blocks, or terraform state mv for older versions, map old addresses to the new module paths. The plan must show no changes before anything else happens.
  3. Bring the other environments onto the same modules one at a time, turning every difference into an explicit input. Each difference you find is a question for the team: intentional, or drift?
  4. Import what was created by hand, with import blocks, so it is managed from then on.

Done carefully, every step ends with a plan that shows no changes, and production never notices.

Summary

  • One account per environment; state per environment and component.
  • Modules for meaningful units, with safe defaults and no environment names.
  • Root modules that only wire modules together for one environment.
  • Pinned module versions, promoted from dev to production.
  • Plans reviewed in merge requests and applied by CI.

If your Terraform has grown into copied directories, or you are setting up AWS environments from scratch, this is work I do often. The contact form is the quickest way to start.