Deployment Consistency for Kubernetes Manifests

Deployment, and more importantly, environment consistency, is a major pain for DevOps engineers and software developers. Whether it's environment consistency in a Terraform module or dependencies being passed into an application, the challenge is the same for each team - where to store configuration data and how to retrieve it easily.

In this blog post, you'll learn about the two primary ways to manage deployment consistency for Kubernetes Manifests for multiple environments and how you can take it to the next level with CloudTruth.

To keep things simple yet show the power of each platform, we'll use a basic k8s Manifest that deploys a stateless Nginx web app.

The Helm Way

Thinking about Helm Charts, let's use the example Manifest below. As you can see, on lines 8, 9, 13, and 17, there's syntax that tells the Manifest to pull in configuration data from the Helm Chart.

apiVersion: apps/v1
kind: Deployment
metadata:
      name: nginx-deployment
spec:
     selector:
         matchLabels:
             app:
             replicas:
template:
metadata:
    labels:
        app:
spec:
     containers:
      - name: nginxdeployment
      image:
      ports:
      - containerPort: 80

Then it's up to you to figure out how you want to pass in the data. There are two primary ways.

First, you can store the values.yaml files in the parent directory of the Helm Chart. You'll have to manage each parameter/value for each environment in each of the files.

Second, you can have directories that point to each environment. This makes things a little bit easier because it looks cleaner and more consistent, but the same rules apply - you have to manage each parameter/value for each environment in each of the files.

Then, to deploy for each environment, you have to specify a -f flag to specify the values.yaml file.

helm install nginxdev nginxvalues -n dev -f values-dev.yaml
helm install nginxstaging nginxvalues -n staging -f values-staging.yaml
helm install nginxprod nginxvalues -n prod -f values-prod.yaml

Although it makes life a little easier, there are still:

• Three different locations that you have to manage
• Three different files that you have to manage
• Multiple values to manage in three different locations
• No central place to manage the values for Dev, Staging, and Prod

Which in turn still complicates deployment consistency.

The Kustomize Way

Thinking about the Kustomize way, things get slightly easier. Below is a standard Kubernetes Manifest that you can use with Helm, but notice that there isn't specific syntax like in Helm that you have to specify. Instead, you have a template (or multiple templates) in a base directory.

apiVersion: apps/v1
kind: Deployment
metadata:
      name: nginx-deployment
spec:
     selector:
         matchLabels:
             app: nginxdeployment
     replicas: 2
     template:
         metadata:
             labels:
                 app: nginxdeployment
     spec:
         containers:
         - name: nginxdeployment
         image: nginx:latest
         ports:
         - containerPort: 80

You also have to manage what's called a kustomization.yaml file, which tells Kustomize which Kubernetes Manifest you want to manage with Kustomize, which means you have yet one more file to manage.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- service.yaml

Then, for each environment, it's very similar to Helm. You'll have directories that contain environments and Value files that have values for each environment per the code and screenshot below.

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base/

replicas:
- name: nginx-deployment
count: 1

The same type of problem exists in Kustomize as it does in Helm:

• Multiple locations
• Multiple files to manage
• Multiple environments to manage

Let's see how CloudTruth can help deployment consistency for DevOps engineers and software developers.

The CloudTruth Way

Instead of thinking about it as "CloudTruth vs Helm" or "CloudTruth vs Kustomize", think about it as how can these tools work together?.

Below, you'll see two CloudTruth projects with parameters. One for Kustomize and another for Helm.

Instead of having to worry about value.yaml files or kustomize files all over source control, and have yet more things to manage, you can store the values inside of CloudTruth.

Not only can you store the values in CloudTruth, but you can modify the values for each environment right from CloudTruth.

This method allows you to have a true, central place where you don't have to worry about configuration sprawl or configuration drift. You can instead feel good about having true consistency across any environment.

Learn more about CloudTruth's open-source Kubernetes operator or go to the project on GitHub