Create DRY code with Terragrunt and Terraform Modules

To keep your infrastructure as code DRY Terragrunt is a great Terraform wrapper. This walkthrough will show you how to avoid copying and pasting the same Terraform code over and over again. We provide a base set of Terraform modules that are examples of DRY code with Terragrunt to deploy across multiple environments. To take it one step further, we demonstrate how to apply a secure DRY philosophy to variables and secrets management.

Dry Terraform code with Terragrunt

This example will create an AWS Instance in us-west-2 and an S3 bucket deployed with Terragrunt for a development, production, and staging environment. You can follow along in your AWS account by cloning this repo.

git clone https://github.com/cloudtruth-demo/terragrunt-cloudtruth-deploy.git

The Terragrunt folder structure contains development, production, and staging directories.

# terragrunt-cloudtruth-deploy
├── development
│   ├── instance
│   │   └── terragrunt.hcl
│   ├── s3
│   │   └── terragrunt.hcl
│   └── terragrunt.hcl
│       
├── production
│   ├── instance
│   │   └── terragrunt.hcl
│   ├── s3
│   │   └── terragrunt.hcl
│   └── terragrunt.hcl
│       
└── staging
    ├── instance
    │   └── terragrunt.hcl
    ├── s3
    │   └── terragrunt.hcl
    └── terragrunt.hcl

The root terragrunt.hcl file in each directory generates an aws provider so you only need to specify this code once in the root location.

root terragrunt.hcl

# terragrunt-cloudtruth-deploy/development/terragrunt.hcl
generate "provider" {
  path = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents = <<EOF
provider "aws" {
  profile = "default"
  region  = "us-west-2"
}
EOF
}

The instance and s3 sub-folders also contain a terragrunt.hcl file. This sets the source parameter to point at a specific terragrunt module in the terragrunt-cloudtruth-modules repo. They also include the function find_in_parent_folders which configures the AWS provider from the root terragrunt.hcl. As a result you avoid copying the provider and Terraform code in multiple locations.

Example instance terragrunt.hcl:

terraform {
  source = "git::https://github.com/cloudtruth-demo/terragrunt-cloudtruth-modules.git//instance?ref=v0.0.1"
}

include {
  path = find_in_parent_folders()
}

Terraform modules for DRY code

Terraform modules allow you to keep your HCL DRY by allowing you to reuse and breakdown infrastructure code into smaller pieces. Our structure contains a main.tf, outputs.tf and variables.tf.

The resource is created in main.tf:

resource "aws_instance" "cloudtruth" {
  ami           = var.ami
  instance_type = var.instance_type 
  availability_zone = var.availability_zone_names[0]

  tags = var.resource_tags
}

We set what will be output to the Terraform state file in outputs.tf:

output "zone" {
  value = aws_instance.cloudtruth.availability_zone
}

output "AMI" {
  value = aws_instance.cloudtruth.ami
  sensitive = true
}

output "instance_type" {
  value = aws_instance.cloudtruth.instance_type
}

output "instance_name" {
  value = aws_instance.cloudtruth.tags.Name
}

Variables and secrets required for each module are set in variables.tf:

variable "ami" {   
  description = "Value of the Amazon Machine Image"
  type        = string
  }

variable "instance_type" {   
  description = "Value of the Instance Type"
  type        = string
  }
  
variable "availability_zone_names"{
  description = "List of available regions"
  type        = list(string)
 }  
 
variable "resource_tags" {
  description = "Tags to set for all resources"
  type        = map(string)
}

Creating DRY Terragrunt Inputs for Terraform modules

Terragrunt’s examples define Terraform input variables for each environment you are deploying to. As a result, you end up repeating yourself by putting the same identical inputs into each terragrunt.hcl. By burying configuration values in environment subdirectories you now have to keep track of all the differences! certainly, this complicates overall management.

Note that in this example, there are no inputs defined in the root terragrunt.hcl. We are going to pass all of our parameters and secrets to Terragrunt as environment variables. Gruntwork themselves recommend this as the first technique for managing secrets. However, the architect has to implement a solution for storing those secrets and parameters.

CloudTruth as a unified Secrets and Parameter store

We are going to setup CloudTruth as our parameter and secrets manager for this. You can also try CloudTruth with the free community edition. Alternatively, you can use your own parameter and secrets store instead.

Create a CloudTruth Project called Terragrunt in the CloudTruth CLI.

cloudtruth project set Terragrunt

Now, add the parameters to the Terragrunt project that are required by the Terraform modules we are calling in the terragrunt.hcl.

cloudtruth --project Terragrunt parameter set TF_VAR_ami -v ami-830c94e3
cloudtruth --project Terragrunt parameter set TF_VAR_instance_type -v t2.micro
cloudtruth --project Terragrunt parameter set TF_VAR_availability_zone_names -v '["us-west-2a", "us-west-2b"]'
cloudtruth --project Terragrunt parameter set TF_VAR_resource_tags -v '{"Name":"Cloudtruth-Instance","project":"CloudTruth Run Terraform","environment":"default"}'

Set unique resource tags for the EC2 instance and s3 bucket for each environment.

cloudtruth --project Terragrunt --env development parameter set TF_VAR_resource_tags -v '{"Name":"CloudTruth-development","project":"CloudTruth Run Terraform","environment":"development"}'
cloudtruth --project Terragrunt --env production parameter set TF_VAR_resource_tags -v '{"Name":"CloudTruth-production","project":"CloudTruth Run Terraform","environment":"production"}'
cloudtruth --project Terragrunt --env staging parameter set TF_VAR_resource_tags -v '{"Name":"CloudTruth-staging","project":"CloudTruth Run Terraform","environment":"staging"}'

Now your CloudTruth Terragrunt project is setup to manage the TF_VAR variables with unique values for resource tags across our multiple environments.

Running a centrally managed DRY deploy

Terragrunt respects any TF_VAR_xxx variables you’ve manually set in your environment and follows the same variable precedence as Terraform. Using CloudTruth Run we will pass the CloudTruth configured TF_VAR_xxx variables directly to the Terraform modules through Terragrunt for the specified environment.

Change directory to terragrunt-cloudtruth-deploy/development/.

From terragrunt-cloudtruth-deploy/development/ execute the following command which passes variables from the CloudTruth project Terragrunt for the development environment into terragrunt:

cloudtruth --project Terragrunt --env development run -- terragrunt run-all apply

Finally, you can view the outputs that display the parameter values from the CloudTruth Development environment by running terragrunt run-all output.

https://cloudtruth.com/blog/dry-terragrunt-terraform-modules/(opens in a new tab)

AMI = <sensitive>
instance_name = "CloudTruth-development"
instance_type = "t2.micro"
zone = "us-west-2a"
s3_bucket_name = "cloudtruth-grunt-free-panda"
s3_tag_name = "CloudTruth-development"

Additionally, you can change to the production or staging directories and pass the respective CloudTruth environment to deploy various settings across your different infrastructure!

Cleanup

Destroy the AWS resources by passing environment variables the same way we created them.

cloudtruth --project Terragrunt --env development run -- terragrunt run-all destroy

In summary, this is a getting started to creating DRY infrastructure code with Terraform modules and Terragrunt. Consequently, To really take advantage and simplify configuration management we showed you how to use an external parameter and secrets store to bring DRY concepts to inputs and variable management.