“Developer’s constant urge to use Terraform like a programming language :)”
My impression after enjoying my first terraform applys is that it's kind of rigid, though when someone says Immutable and declarative in the same sentence, you should definitely get that it's
not procedural by design. Therefore, you can’t expect a bash shell script programmability. That's the tricky part for me because I always start playing with the Cloud vendor’s CLI before switching to its Terraform provider... It’s like having a BBQ party on Sunday then turn vegan on Monday :D!! You just wish they could allow a tiny more.
Most of the time, the code inside your terraform configuration is pretty static, more like a reflection of the actual end state (hard coded). In a sense, declarative language is more readable but don’t fit the “if then else” logic proper to procedural programing (single use scripts).That being said, terraform still provides a number of functions and expressions you can use to process strings/number/lists/maps/simple loops etc.
-- “Because we’ll always have that reflex to make our code conditional.” --
What If, I wanted my aws vpc configuration (see config in my GitHubRepo) to accept different security group rules depending on the type of instance attached ? It’ll probably reduce duplication and improve reusability.
This article answers just that as it will demonstrate how dynamic interpolation of a nested variable can be accepted by the parser and used by a for_each loop to make a terraform provisioning more flexible. I’ll explain better through the post (don’t worry).
Terraform available loops
As said above, terraform has few routines and expressions that allow to perform loops, if-statements and other logic.
Available in terraform 0.11, count can control the the number of resources to be created or whether the resource will be created at all using the ternary-operator [
CONDITION ? TRUEVAL : FALSEVAL] => IF:THEN:ELSE
Pros: Perfect for conditional logic using ternary operator (1:create, 0: skip)
count inside of an inline block is not supported (i.e Tags) and referencing a count element using its index can be risky and confusing when deleting specific resources. Some users also think this is so 0.11 ;)!
Available since v0.12, this one is more sophisticated than count, as it’s close to the loop as we know it. Below, we loop over all the map’s content to create as many resources as the length of the map (you can try this yourself).
Pros: helps create multiple resources or inline blocks by looping over maps, set of strings, lists. Easy to reference an element within the looped collection (name instead of index). It even started to support Modules with version 0.13.
Weaknesses: Conditional logic is more complex than in count as it requires a nested for loop within the for_each clause. Referenced collection has to be computed during the plan phase and not upon resource creation (not on the fly).
My pick: The use of for_each is recommended by Hashicorp as it’s more intuitive than count and no longer effected by the order of the variable's values (index). Hence, I will be relying on it in my below use case
“Trying to reduce duplication and maintain readability is the eternal Terraform struggle”
Cool, now let’s describe the reason behind this post in the first place. In my previous vpc configuration (see my GitHubRepo) I hardcoded all the security group rules (Open Ports) which made it un-reusable. The following snippet will illustrate the section of interest ( ingress rules ):
Notice that all the ingress rules are added as inline blocks in the main resource “terra_sg”. However, Terraform also provides a standalone Security Group Rule resource (a single
egress rule) . This could really help to make the deployment more dynamic.
So let’s say, we have 3 cases or sets of sg ingress rules per type of attached instance:
- Case 1: SSH only => port 22
- Case 2: SSH+WEB => port 22, 80, 443
- Case 3: RDP+WEB (Windows)=> ports 3389, 80, 443
The aim here is to replace the old vpc configuration that had a security group with hard coded inline rules (fixed list of ports) by a dynamic iteration of an “aws_security_group_rule” resource using a for_each loop. To do so, I first need to create a map for each set of sg rules.
Is Nested variable call Possible in Terraform?
At this point in I only need to make the for_each expression accept a call to a nested variable that would include 3 variables:
1. sg_type to pick the rule type
2. sg_mapping to fetch the right map variable based on sg_type
3. A wrapper variable that the for_each can call =>
But l quickly realized that terraform doesn’t allow variable substitution within variables as shown below:
After few unsuccessful attempts and hours sniffing forums and other resources online. I sensed that I might have neglected a special type of variables that could help dealing with my substitution woes. The potential silver bullet in question is
terraform local which is a local block where expressions are defined in one or more local variables within a module. I was close but I still needed help from the community on Hashicorp forum.
The final result to represent my nested variable substitution: var.[var.sg_mapping[var.sg_type] is as follows
With Locals, I only needed to move sg_mapping map to a local block and replace its values by each literal sg map variables and voila ! the for_each can now call the local variable which in turn will substitute the sg_type variable.
Each resource will be created and identified according to the selected map value (example “ WEB =>80,443,22”).
for_each is set, Terraform will identify each of the resource instances associated with the dynamic resource block by a map key from the value provided to
for_each. (example : => aws_security_group_rule.terra_sg_rule["SSH"])
Hence, referencing these resources as a list is wrong [aws_security_group_rule.terra_sg_rule.* ] => error
The correct way to output all of our resource instances in my case is to use a for loop to traverse and fetch the resulted map.
- This was an improvement I wanted to complete from my previous blog post, but I hope you enjoyed discovering (like me) how to leverage dynamic features to increase the reusability of a terraform module
- Locals are a solid ally when dealing with variable substitutions within a variable (nested variables)
- Obviously, the over-use of dynamic behavior will hurt readability and maintainability. As terraform team likes to remind us often its famous mantra: “Explicit is better than implicit, and direct is better than indirect”
- TIP : There is also a way to keep all the maps in one main map and then create a local variable that would reference the sub map directly instead of calling an individual map each time .
- It might comfort you to know that balancing the readability and reusability is everybody’s struggle with Terraform ;)
GitHubRepo for this lab: brokedba/terraform-examples/terraform-provider-aws/create-vpc-dynamic