When you implement a GitOps workflow to streamline your engineering practices, one of the goals is to empower developers to make code and configuration changes together. In theory this should make engineering more productive, and it does… until you get to secrets management. If you are planning to use GitHub Secrets with your shiny new GitOps workflow, consider the following major process gaps.
Only an organization admin or a repository owner can modify GitHub Secrets. Developers will need to be given more access or coordinate with someone who has it.
GitHub Actions workflows can access GitHub Secrets, but systems outside GitHub cannot use them unless you configure your workflows to push them elsewhere.
When configuration and code change together in GitOps, they go through the pull request process so they are reviewed, and then they are committed together. GitHub Secrets go through none of the standard review processes that code goes through, and are not committed with the code changes that need them.
By combining configuration and code in a GitOps workflow, they evolve together in lock-step. When you pull a branch or tag, you get configuration as it was at that time that the code depends on. This is a requirement for reproducible deployments. GitHub Secrets only exist in the present, so one that gets deleted or modified is no longer available to a branch that depends on it.
There are solutions in use today that solve some, but not all, of these GitHub secrets gaps. Based on public-key encryption principles, the git-crypt, sealed-secrets, and sops projects have you generate a private key that is only present on systems that can read the secrets. With the public key, you can encrypt secrets that only those with the private key can decrypt. This allows data to be checked in as part of a pull request and live in a public repository.
These tools solve the fundamental problem of secrets being orphaned outside the GitOps workflow. While this is a step in the right direction, it misses the mark:
CloudTruth allows you to avoid these pitfalls. Managing secrets for multiple environments has never been easier than with our parameter store, which enables environments to derive from other environments. You can define secrets common to all your environments, and secrets specific to just one environment.
Our unique secrets tagging solution allows you to:
Switching over to CloudTruth can happen gradually – no forklift cutover needed! If you already have secrets in Amazon SSM, Secrets Manager, or S3, you can map them using JMESpath syntax into CloudTruth projects. This allows you to work CloudTruth into existing workflows that leverage those services without requiring everything to change all at once.
Once your secrets are managed by CloudTruth they become accessible to more than just GitHub and avoid gaps in secrets management. You can use cloudtruth-cli inside a docker container to pull secrets at boot time, getting just-in-time configuration properties. You can use configure-action, our GitHub Action, to enable tagged secrets to flow into your GitHub workflows. The number of integrations that CloudTruth enables grows almost daily.
Push replication is just on the horizon. You will be able to set up push replication to Amazon SSM or Secrets Manager so your cloud-deployed instances and containers can leverage secrets. You will be able to push secrets into your GitHub repositories and environments. Need to roll things back? Update your tag to a point in the past and we’ll push replicate the past for you.
Secrets are just configuration parameters with different requirements for storage and handling. GitOps on GitHub wants you to use two completely different mechanisms to manage configuration and secrets. Why manage them in two completely different ways that are so badly misaligned?
Visit https://cloudtruth.com/ to learn more about CloudTruth’s converged configuration and secrets management.
About the Author: Jim King is a software engineer, architect, startup founder, and has contributed to a number of open source projects including Apache Thrift and Boost.