How to deploy a simple Kubernetes app with a K8s operator

One of the interesting aspects of Kubernetes is that everything is declarative. This means that everything is driven by configuration and thus driving config and deploying a Kubernetes app with an automated K8s operator is a natural fit.

The CloudTruth integration for Kubernetes (a.k.a kubetruth) is a Kubernetes operator that can be installed via helm. By default, it is set up to push the parameters from CloudTruth projects into ConfigMap and Secret resources that are named after the project. This makes it just as easy to manage the Kubernetes configuration for many services as it is for one.

Whilst managing ConfigMaps and Secrets is a great first step to getting familiar with using CloudTruth to configure your services, there is much more that can be accomplished. Kubetruth allows one to manage any Kubernetes resource by defining a template that represents that resource, and parameterizing it to use data from CloudTruth parameters and templates. Since re-deploying pods is frequently tied to configuration changes, a natural progression is to use CloudTruth to manage a Kubernetes deployment.

Setup a CloudTruth Account

To start with a simple example, we’ll use CloudTruth to manage the lifecycle of an nginx deployment by creating a CloudTruth project to hold the configuration for our application. You can follow along by signing up for a free CloudTruth account, and using the CloudTruth CLI (or Web UI) to input data. The commands below reference files available in a github repository, so your first step should be to clone the repository and switch to the directory containing the files:

git clone https://github.com/cloudtruth/blog-examples
cd blog-examples/kubernetes-deploy/part1

The commands to create the cloudtruth project are also present in the setup.sh script present in that directory.

Install the CLI:

curl -sL https://github.com/cloudtruth/cloudtruth-cli/releases/latest/download/install.sh | sudo sh

Login using the CLI to create an api key, then assign it to an environment variable to save it for installing the operator:

cloudtruth login
export CLOUDTRUTH_API_KEY='<your_api_key>'

Configure a Kubernetes project

Now lets create a project in Cloudtruth called k8s-myapp-nginx:

cloudtruth projects set k8s-myapp-nginx

Then we’ll add some parameters to that project which will be used to control the basic functionality:

cloudtruth --project k8s-myapp-nginx parameter set --value myapp app_name
cloudtruth --project k8s-myapp-nginx parameter set --value 80 app_port
cloudtruth --project k8s-myapp-nginx parameter set --value 80 service_port
cloudtruth --project k8s-myapp-nginx parameter set --value nginx image_name
cloudtruth --project k8s-myapp-nginx parameter set --value 1.20 image_version

At this point, the project in the cloudtruth web ui will look like:

Finally, we’ll create templates for the deployment and service resources that we’d like CloudTruth to manage:

cloudtruth --project k8s-myapp-nginx template set --body deployment.tmpl.yaml deployment
cloudtruth --project k8s-myapp-nginx template set --body service.tmpl.yaml service

Where deployment.tmpl.yaml contains:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: 
  namespace: ctdeploytest
  labels:
    app: 
spec:
  replicas: 1
  selector:
    matchLabels:
      app: 
  template:
    metadata:
      labels:
        app: 
    spec:
      containers:
        - name: 
          image: :
          ports:
            - containerPort: 

and service.tmpl.yaml contains:

apiVersion: v1
kind: Service
metadata:
  name: 
  namespace: ctdeploytest
spec:
  selector:
    app: 
  type: NodePort
  ports:
    - protocol: TCP
      port: 
      targetPort: 

By taking a look at the web UI, we can verify that these templates were created and can drill into to each of them to verify the contents match expectations:

Provision the Kubernetes operator

On the Kubernetes side, if you don’t already have a local Kubernetes cluster to test against, then we recommend setting up minikube. You’ll need to install the operator and configure it to use the templates we defined above for our project. We’ll install kubetruth in a namespace to make cleanup easier at the end of this tutorial:

helm repo add cloudtruth https://packages.cloudtruth.com/charts/
helm install --create-namespace --namespace ctdeploytest --values ctdeploytest-values.yaml --set appSettings.apiKey=$CLOUDTRUTH_API_KEY kubetruth cloudtruth/kubetruth

where ctdeploytest-values.yaml contains the operator configuration:

# Applying deployment resources requires additional permissions
rbac:
  additionalRoleRules:
    - apiGroups: ["", "extensions", "apps"]
      resources: ["deployments", "replicasets", "pods", "services"]
      verbs: ["*"]

# Kubetruth is configured using a CustomResourceDefinition called
# ProjectMapping, which allows one to specify how the operator
# transforms the data from your CloudTruth projects into kuberenetes
# resources
projectMappings:

  # Define the root project mapping, skipping all projects except for the
  # example we care about
  root:
    scope: "root"
    environment: default
    skip: true

  # Define an override ProjectMapping to enable deployment of CloudTruth projects named k8s*
  k8s-deploy:
    scope: "override"
    skip: false
    project_selector: "^k8s-"
    resource_templates:
      deployment: ''
      service: ''

Verify the Kubernetes app deploy

Thats it! All the configuration and connections are complete, and changes to the CloudTruth project will propagate into Kubernetes automatically. To check that everything ran correctly and that our nginx service is running, you can try the following commands:

kubectl describe -n ctdeploytest deployment kubetruth
kubectl logs -n ctdeploytest deployment/kubetruth
kubectl describe -n ctdeploytest deployment myapp
kubectl logs -n ctdeploytest deployment/myapp
# Use minikube service to open a web browser connected to the deployed service
# force a 404 to see the nginx version string by adding /xxx to the end of the url
minikube -n ctdeploytest service nginx

If you’d rather investigate in a more visual fashion, you can run minikube dashboard to get a web console into your minikube cluster.

One useful aspect to having CloudTruth deploy your components, is that a Kubernetes app deploy can now be triggered with a configuration change. For example, one can bump the nginx version to the newest version, and the deploy will happen automatically. This could easily be done in CI for your own components after building and pushing a new image.

kubectl describe -n ctdeploytest deployment myapp | grep -i image # shows 1.20
cloudtruth --project k8s-myapp-nginx parameter set --value 1.21 image_version
kubectl -n ctdeploytest  exec deployment/kubetruth -- wakeup # trigger an update instead of waiting for polling cycle to elapse
kubectl describe -n ctdeploytest deployment myapp | grep -i image # shows 1.21

Cleanup

To uninstall the operator and clear out any resources it created, run the following series of commands (or run cleanup.sh).

helm delete --namespace ctdeploytest kubetruth
kubectl delete customresourcedefinition projectmappings.kubetruth.cloudtruth.com
kubectl delete namespace ctdeploytest

Conclusion

While this is a fairly simple application of CloudTruth for driving Kubernetes application configuration, it does illustrate the basic methodology of transforming simple configuration into a domain specific representation. This allows the different cohorts of your teams to focus on what’s important to them, with CloudTruth connecting the dots. Your Operations teams (Kubernetes experts) can define common and compliant patterns across all your application components. Thereby simplifying the life of Developers who now only have to set a handful of simple parameters to get their components running in Kubernetes.

Moving on from a toy example, upcoming parts to this blog series will share some tips on how to use CloudTruth to manage a multi-service, multi-team application at scale, including topics like:

• Using checksums to link Deployments to ConfigMaps and Secrets
• Keeping configuration interfaces up to date with a GitOps lifecycle
• Compose-able templates
• Cross project sharing

Further Reading

CloudTruth has open source in it’s DNA. Make sure you check out the kubetruth and CLI repositories if you are interested in seeing how we engineer the code that you’ll be running in your environment. You can find some other usage examples for CloudTruth in Kubernetes there as well.