Securing Terraform Secrets Easily

Terraform has spread like wildfire in the last few years in the technology community and has become a really useful part of the toolchain for "DevOps" teams. As the name implies, DevOps is all about combining the concepts of the Development phase of a software and the Operations phase of information and technology. The benefits of DevOps include the improvement in the agility and performance of the software, enables collaborative working, improves operational support and faster fixes that can be bought in, etc. Terraform is a DevOps tool which comes under the IaC category, i.e. Infrastructure as Code. The approach of IaC uses a higher level of descriptive language for more effective provisioning of resources and deployment processes of the software. This process of code-based infrastructure method is similar to the software design process where everything from code versioning to production approval of the software is done in a fine-grained manner. In this article, we’ll brush up a bit about Terraform and how a third party provisioner can be integrated with it.

What is Terraform and what makes it special?

Terraform came storming in to the world of DevOps from Hashicorp in 2014. Terraform was introduced by Hashicorp as a tool for building, designing, versioning and changing the infrastructure in a safe and efficient way. With its adaptability to get along easily with the major service providers and custom in-house solutions, Terraform attracted quite a lot of developers towards it. This so-called multi-cloud immutable infrastructure tool which was written in Go is (arguably) the first (and simplest) of its kind when it was released.

Let’s now get to know why a good number of people who are into DevOps prefer Terraform upon various other cloud infrastructure platforms out there:

• One of the most important reasons why people opt for Terraform and made it this big a thing in the sector of DevOps is because the infrastructure can be defined in a detailed way according to our needs using code and makes it a pretty flexible task from the start to the end.

• Rather than forcing you to follow a single code principle, Terraform allows you to follow any code principle as per your need.

• Its adaptability with multiple service provider platforms bought everything under an umbrella.

• The immutable infrastructure of Terraform made every new update of any parameter in the code to create a distinct snapshot.

This basically means that the updating of the development environment can be done in a smooth and bug-free way using Terraform.

Is Terraform the best at this time?

After tracking the impressive progress made by Terraform as a DevOps tool from 2014, the answer to this question is a Yes. But being one of the best in its kind does not free Terraform, or well, anything for that matter, from flaws! There are concerns that DevOps and security experts need to manage while using Terraform. Let us see what are the practical problems faced by users while using Terraform.

Managing states in Terraform can be a pain for its users. It is demanded that the states must be in sync with the version of Terraform and must be saved somewhere pretty secure as it contains secrets. Modifying states too brings along a pile of issues with it. If at all you are constantly redefining your infrastructure and are changing certain resource locations, while applying the changes, you’ll be in a state where it may simply show that certain resources are misplaced. 

Managing secrets is another issue faced by users. It’s a bit confusing when it comes to storing secrets. Secrets contain confidential information that must not be exposed. Terraform provides a vault for this need which solves this problem to a certain extent. One of the issues faced by the developers is that it is not possible to have a configuration setup depending on an infrastructure that is yet to be created.

Dealing with Secrets in Terraform

As already discussed in this article, managing secrets may pop a lot of question marks while using Terraform. The storage space is one thing that makes things a bit confusing. In AWS, using KMS is an option you can proceed with while storing secrets apart from using Hashicorp’s own vault. You can also store secrets in private spaces like your own Git repository if you’re confident about the privacy of the space. If you’re running the whole setup from a CD/CI server, then using env vars for storing secrets can serve as another good option. If these aren’t enough, you can also store all your secret.tf vars files along with the .tf files. Encrypting this using git-secret will gives the secrecy of the whole thing an edge. While following this method, Terraform while running will first use ‘git secret reveal’ to access the secret objects and ‘git secret hide’ right after that.

Usage of Provisioners in Terraform

People usually have to write their own "security provider code" in order to let Terraform pull in secrets from a 3rd party vault-like Onion ID.

resource “aws_instance” “lamp_server”{

Instance_type: “type”

//Instance settings

Provisioner “chef” {

Provisioner options{}

}

}

The above snippet is a code that demands an AWS instance with ‘chef’ being the provisioner for post-deployment configuration. Provisioners are required in Terraform to perform post-deployment configuration. The steps for enabling this facility is to first create an instance, and the provisioner will start executing commands locally with the help of this instance. In the above-mentioned code, chef is quoted as the resource for post-deployment configuration.

In this same way, any third-party source of truth, ie, like Onion ID, can be integrated with Terraform. Yes, you guessed it right. That just opened another way for managing secrets effectively. In Terraform documentation too, the usage of an external data source is mentioned. This allows an external program which implements a specific protocol to act as a data source. By this, you can enable its usage at any part in the Terraform configuration. Here’s an example code for the usage of an external data source:

data "external" "example" {

program = ["python", "${path.module}/example-data-source.py"]

query = {

# arbitrary map from strings to strings, passed

# to the external program as the data query.

id = "abc123"

}

}

(Since the external data source protocol uses JSON, it is recommended to use the utility jq to translate JSON.)

Here’s an example on getting secrets list from the vault of OnionID:

headers = {

OID-TOKEN': 'Bearer ' + TOKEN

}

response = requests.get(PANEL_URL + '/api/v1/secrets/list', headers = headers)

status_code = response.status_code

response_data = response.json()

return (response_data) if status_code == 200 else (False, response_data['message'])

(Note that, this snippet of code is to Get the list of secret keys and not the key values).

Conclusion

In this article, apart from discussing one of the best open-source IaC tools, but also saw what makes it the best and what brings along the pain too. It is possible to pull secrets out of the vault of OnionId by integrating it with Terraform by making it an external data source and with the help of provisioners during post-deployment configuration. This gifts your secrets some more efficient secrecy and security! Please get in touch with us at www.onionid.com ad we would love to help you.