Managing Terraform Workspaces

Darryl Diosomito

Darryl Diosomito

You have already been using a Terraform workspace with the CLI you just may not know it yet! Every directory creates a default workspace when a terraform init is run. We are going to show you how to create and manage Terraform CLI workspaces as well as when to use multiple workspaces.

Use cases for workspaces

Workspaces allow you to keep separate local or remote states within the same directory for specific resources or multiple environments that you are managing . A typical pattern used without workspaces is to create multiple working directories to separate configuration or environments. However, this results in dealing with repetition throughout multiple directories where state files, caches and modules exist. Therefore, using workspaces for state management can help keep Terraform code DRY by using the same HCL from a single directory.

The first use case will walkthrough deploying an AWS instance using workspaces for production, staging and development environments from a single working directory. We will use CloudTruth as a centralized parameter and secret store for Terraform Inputs.

The second use case will demonstrate how to leverage Terraform workspaces with modules to help manage larger systems. We then review what this looks like when using a remote backend.

Managing Terraform workspaces

The terraform workspace command will be used to create and manage workspaces. We will deploy an AWS instance from a single directory structure:

.
├── infrastructure
│   ├── main.tf
│   ├── outputs.tf
│   └── variables.tf

You can follow along or inspect the folder structure contents from the this repo.

Now let’s walkthrough the workspace commands!

terraform workspace new

From a dedicated working directory we will create a workspace layout by executing the workspace new command. Consequently, this will create development, production and staging workspaces.

terraform workspace new development
terraform workspace new production
terraform workspace new staging

terraform workspace list

You may have noticed that creating a new workspace switched you directly into that created workspaces context. You can view the workspaces in your working directory with the workspace list command. The asterisk represents the current workspace context.

terraform workspace list
  default
  development
  production
* staging

terraform workspace select

To switch workspaces use the workspace select command.

Switch to the development workspace:

terraform workspace select development
Switched to workspace "development".

terraform workspace show

workspace show will display the currently selected workspace.

terraform workspace show
development

terraform workspace delete

workspace delete removes an existing workspace.

terraform workspace delete staging
Deleted workspace "staging"!

Terraform will not allow you to delete a workspace if it contains active state or is your current workspace. If you have infrastructure deployed you will get a warning when attempting to delete that workspace. If you want to leave your infrastructure in place and delete the workspace you can use the -force flag.

terraform workspace delete development
Workspace "development" is not empty.

Deleting "development" can result in dangling resources: resources that
exist but are no longer manageable by Terraform. Please destroy
these resources first.  If you want to delete this workspace
anyway and risk dangling resources, use the '-force' flag.

Deploying infrastructure to a Terraform workspace

Now that we have reviewed how to manage Terraform workspaces we will deploy an AWS instance to the development workspace.

Select the created development workspace:

terraform workspace select development
Switched to workspace "development".

Initialize Terraform from the “infrastructure” working directory:

terraform init

Apply the configuration with cloudtruth run to pass in TF_VAR environment variables directly to Terraform. You can view the variables we are configuring in the CloudTruth project here in our docs. Alternatively, you can also create and pass .tfvars file in your local working directory.

cloudtruth --project Terraform --env development run -- terraform apply -auto-approve

When a new workspace is created, Terraform instantiates a directory called terraform.tfstate.d and a structure for each workspace where it stores local-state files. Deploying infrastructure to the development workspace creates a local terraform.tfstate file.

.
├── infrastructure
│   ├── terraform.tfstate.d
│       ├── development
│           ├── terraform.tfstate
│       ├── production
│       ├── staging
│   ├── main.tf
│   ├── outputs.tf
│   └── variables.tf
Managing Terraform workspaces with CloudTruth

CloudTruth: Create DRY Terraform Inputs

CloudTruth is a solution that provides a single record of truth and for all of your secrets and parameters. Using CloudTruth while managing Terraform workspaces creates DRY configuration management by eliminating .tfvar files scattered in each working directory.

CloudTruth: Manage Terraform Inputs

Instead of wondering what the difference is between Terraform workspace inputs you can easily compare your environments.

CloudTruth environment compare with Terraform workspaces
Add a caption

In addition you can stop configuration errors before they happen with versioning controls and perform config validation by placing checks and balances from typos and other frequently occurring input errors

Managing Terraform Workspaces with re-usable modules

For large systems, Hashicorp does not recommend Terraform workspaces alone as a solution to isolate environments. You can use modules with workspaces to further breakdown components of your architecture to use different backends across different Terraform workspaces.

Deploy infrastructure using modules and workspaces

We can deploy component groups such as AWS instances using a module targeting different workspaces. This can become a blueprint on how to deploy other pieces of your architecture across multiple environments with workspaces.

We will deploy an AWS instance with a module from a single directory structure:

.
├── module-infrastructure
│   ├── main.tf
│   └── variables.tf

You can follow along or inspect the folder structure contents from the this module repo.

The main.tf uses source to call a Terraform module we created that will deploy an aws_instance. Additionally this main.tf configures variables required for the child module to deploy.

Create a workspace for each environment for the instance component in the module-infrastructure working directory.

terraform workspace new instances-development
terraform workspace new instances-production
terraform workspace new instances-staging

Switch to the instances-development workspace:

terraform workspace select instances-development
Switched to workspace "instances-development".

Initializing Terraform with modules will download the child modules into the .terraform/modules working directory.

terraform init

Apply the configuration with cloudtruth run to pass in TF_VAR environment variables directly to Terraform.

cloudtruth --project Terraform --env development run -- terraform apply -auto-approve
Managing terraform workspace along with modules

Terraform Workspaces with a remote backend

Terraform supports multiple backends with workspaces. In this case Terraform stores remote state directly in the configured backend. Terraform must be initialized first prior to creating workspaces when a backend is specified.

As an illustration we will use the following as an example with AWS S3 configured as a backed with the following HCL block:

terraform {
  backend "s3" {
    bucket = "west-2"
    key    = "demo/instance/terraform.tfstate"
    region = "us-west-2"
  }

Initialize terraform and create the development, production and staging workspaces.

terraform init

terraform workspace new development
terraform workspace new production
terraform workspace new staging

As a result, when new workspaces are created the remote backend gets a special file structure annotated with env: along with the workspace name and remote backend key.

aws s3api list-objects --bucket west-2
{
    "Contents": [
        {
            "Key": "env:/development/demo/instance/terraform.tfstate",
            "LastModified": "2021-11-02T17:56:24+00:00",
            "ETag": "\"a21e2ab5050d75d0389969dcd23ddef0\"",
            "Size": 156,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "YOUR.USER",
                "ID": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            }
        },
        {
            "Key": "env:/production/demo/instance/terraform.tfstate",
            "LastModified": "2021-11-02T17:56:07+00:00",
            "ETag": "\"9dff7b2687f6bf60007762509d77399a\"",
            "Size": 156,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "YOUR.USER",
                "ID": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            }
        },
        {
            "Key": "env:/staging/demo/instance/terraform.tfstate",
            "LastModified": "2021-11-02T17:56:16+00:00",
            "ETag": "\"19ea524f9a2df6906e6ea64eaf5c63d7\"",
            "Size": 156,
            "StorageClass": "STANDARD",
            "Owner": {
                "DisplayName": "YOUR.USER",
                "ID": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
            }
        }

    ]
}

In order to visualize what that looks like in S3 the following screenshot shows three workspaces created. Each folder contains a terraform.tfstate representative of each isolated workspace.

Terraform workspace S3 remote backend

Summary

In summary we walked through how to manage Terraform workspaces from the CLI by creating and building infrastructure across different environments. Alternatively, we also showed how to use a modular approach along with workspaces. When managing larger systems HashiCorp recommends this modular approach along with workspaces when deploying across multiple environments. CloudTruth allows you to keep Terraform configuration management DRY by centralizing Terraform inputs removing the need to duplicate .tfvar files. In addition we reviewed workflow differences and folder layout when using workspaces with a remote backend.