We have a JSON file that, amongst other things, contains a set of parameter
overrides that we want to pass to aws cloudformation deploy --parameter-overrides
:
{
"ParameterOverrides" : {
"Account" : "123456789012",
"Environment" : "Test",
"ImagesBucket" : "my-test-bucket",
"InstanceType" : "t2.small",
"Version" : "1.0"
}
}
However, --parameter-overrides
expects a set of key=value
pairs:
$ aws cloudformation deploy \
--stack-name test \
--template-file template.json \
--parameter-overrides \
Account=123456789012 \
Environment=Test \
ImagesBucket=my-test-bucket \
InstanceType=t2.small \
Version=1.0
With jq
, we can elegantly process parameter-overrides.json
and produce a set of key=value
pairs:
$ jq -r '.ParameterOverrides
| to_entries
| map("\(.key)=\"\(.value)\"")
| join(" ")' parameter-overrides.json
Environment="Test" Account="NonProd" ImagesBucket="my-bucket" InstanceType="t2.small" Version="1.0"
Plug and play:
$ aws cloudformation deploy \
--stack-name test \
--template-file template.json \
--parameter-overrides $(jq -r '.ParameterOverrides
| to_entries
| map("\(.key)=\"\(.value)\"")
| join(" ")' parameter-overrides.json)
Environment specific overrides
We might have our default parameter overrides in a base JSON file and stack specific overrides in another JSON file:
{
"ParameterOverrides" : {
"Account" : "987654321012",
"Environment" : "Prod",
"S3Bucket" : "my-prod-bucket",
"InstanceType" : "t2.large"
}
}
When deploying to “Prod”, we want values in parameter-overrides-prod.json
take
precendence over the values parameter-overrides.json
. We again can do this using jq
:
$ jq -r -s '.[0] * .[1]
| .ParameterOverrides
| to_entries
| map("\(.key)=\"\(.value)\"")
| join(" ")' parameter-overrides.json parameter-overrides-prod.json
Environment="Prod" Account="Prod" ImagesBucket="my-bucket" InstanceType="t2.medium" Version="1.0" S3Bucket="my-prod-bucket"
Rock and roll:
$ aws cloudformation deploy \
--stack-name prod \
--template-file template.json \
--parameter-overrides $(jq -r -s '.[0] * .[1]
| .ParameterOverrides
| to_entries
| map("\(.key)=\"\(.value)\"")
| join(" ")' parameter-overrides.json parameter-overrides-prod.json)
How is this working?
-
The
-r
flag: outputs raw strings -
The
-s
flag: slurps all inputs into an array, in our case producing an array with two elements (one element per file) -
.[0] * .[1]
: merges the first two elements of the array -
.ParameterOverrides
: returns the value of theParameterOverrides
field -
to_entries
for eachk:v
entry in the input, outputs an array including{"key": k, "value": v}
-
"\(…)"
: this is thejq
syntax for string interpolation -
map("\(.key)=\"\(.value)\"")
produces key=value pairs in our expected format (with value enclosed in double quotes) -
join(" ")
adds a whitespace between eachkey=value
pair
Why!?
I have been maintaining and migrating a project at work that is using cfndsl
with
a titanic load of bash scripts. These scripts hackishly use awk
, sed
, tr
, jq
.
The end result is an unmaintainable masterpiece one iceberg away from sinking. A major
rewrite is not possible and not worth it, so I have opted for minor improvements instead.
So much bash can kill you. Ansible and Stacker are better options for more maintainable CloudFormation project.