3 Ways to Manage Terraform Variables and State Across Environments and Workspaces

Darryl Diosomito

Darryl Diosomito

Managing Terraform variables and state across environments can become burdensome and error prone as environments within an organization grow. This is a typical problem organizations face with Terraform deployments. This article reviews three solutions for Terraform variables and state management when deploying across multiple environments and introduces terraform workspaces.

Directory Structure with Unique .tfvars Files

For each environment that we deploy infrastructure, we will create a unique directory. This directory contains a copy of the HCL code and a terraform.tfvars file that specifies the unique values for the designated environment.

The example directory structure below is for three separate environments called development, production and staging. When running a Terraform apply in a specified directory, infrastructure will be created with the unique environment values from the terraform.tfvars file and a state file will be created.

.
├── development
│   ├── main.tf
│   ├── outputs.tf
│   ├── terraform.tfstate
│   └── terraform.tfvars
│   └── variables.tf
├── production
│   ├── main.tf
│   ├── outputs.tf
│   ├── terraform.tfstate
│   └── terraform.tfvars
│   └── variables.tf
└── staging
    ├── main.tf
    ├── outputs.tf
    ├── terraform.tfstate
    └── terraform.tfvars
    └── variables.tf

Directory Structure with Centrally Managed Variables

A downside to keeping environment specific terraform.tfvars files in separate directories is that individual values for each environment can get buried within the directory structure. This can lead to outages and makes configuration difficult to audit and manage.

An alternative approach is to use a central configuration management system, like CloudTruth, to pass Terraform values for defined HCL variables in at runtime. In this example our directory structure is again created for separate environments called development, production and staging. However we no longer need to put a terraform.tfvars file in each directory.

.
├── development
│   ├── main.tf
│   ├── outputs.tf
│   ├── terraform.tfstate
│   └── variables.tf
├── production
│   ├── main.tf
│   ├── outputs.tf
│   ├── terraform.tfstate
│   └── variables.tf
└── staging
    ├── main.tf
    ├── outputs.tf
    ├── terraform.tfstate
    └── variables.tf

A CloudTruth project was created with parameters using the Terraform TF_VAR_ environment variable naming convention that allows us to pass these variables directly to terraform apply. CloudTruth allows us to centrally manage the keys and values for our projects while giving us the control of customizing values for our organization’s environment hierarchy.

Terraform variables per environment
Terraform variables per environment

Now we can deploy infrastructure calling the Terraform project with the CloudTruth cli. With the run command, we can pass the variables from a specific project and environment into a terraform apply command.

cloudtruth --project Terraform --env development run -- terraform apply

Terraform Workspaces with Centrally Managed Variables

Workspaces is a Terraform feature that allows us to organize infrastructure by environments and variables in a single directory. This allows us to keep our configurations DRY and is an alternative to a wrapper language like Terragrunt.

HashiCorp recommends using workspaces as part of your workflow.

When using workspaces we can setup a single directory structure. Therefore, we can then manage multiple environments with one code source.

.
├── infrastructure
│   ├── main.tf
│   ├── outputs.tf
│   └── variables.tf

From the infrastructure directory we can create workspaces for our development, production and staging environments.

terraform workspace new development
terraform workspace new production
terraform workspace new staging

Each directory contains unique Terraform workspaces. We can view the currently selected workspace and view all of the existing workspaces with the terraform workspace list command.

terraform workspace list
  default
* development
  production
  staging

Now from the development workspace we can run terraform apply. This example continues to use CloudTruth to control Terraform tfvars, we could create individual tfvars files for each environment and then pass them into apply with the -var-file switch. However, we prefer a centrally managed approach for configuration.

 cloudtruth --project Terraform --env development run -- terraform apply -auto-approve

Now our directory structure has an isolated state file for the development workspace. When running a plan or apply each workspace will create a directory for remote or local state.

.
├── workspace
│   ├── terraform.tfstate.d
│       ├── development
│           ├── terraform.tfstate
│       ├── production
│       ├── staging
│   ├── main.tf
│   ├── outputs.tf
│   └── variables.tf