Abusing JSON: Why AWS CloudFormation sucks?!

AWS CloudFormation (CFN) is a file-based interface for provisioning AWS resources and in that way is Amazons answer on the ‘Infrastructure as Code’ paradigm. I am a huge fan of this approach and CFN is a big step into the right direction as it supports not just common DevOps ideas, by allowing devs to create, update and delete the operational infrastructure they need with their own tools (a text editor, version control and tests).

It also enables you to think different about your IT infrastructure and how to work with it (e.g. just think about immutable stacks). However, I see some issues that become more important if you work with CloudFormation on a daily basis …

JSON Lang, WTF?! #

CloudFormation templates are simple JSON-formatted text files that (in theory) can easily be created and maintained by hand as nowadays every dev (and/ or ops) should be able to read and write valid JSON. However, AWS CloudFormation uses the originally static ‘lightweight data-interchange format’ as a scripting language, allowing e.g. the definition and usage of parameter and (somehow crazy) intrinsic function references.

In my eyes, this is a heavy misusage that leads to several problems regarding complexity and comprehensibility. You can find several deterrent examples that illustrate this right in the official AWS developers guide. If you still don’t think JSON is a syntax-bloated, hard-to-read data format just have a look on how to define DataPipelines using CloudFormation and tell me you like it and that you are able to easily understand all the definitions as well as their interactions. It took me some time…

But that’s not even the worst part. JSON also lacks some feature substance that some devs would regard heavily essential. In contrast to its technical superset YAML, JSON is not aware of e.g. comments or user defined data types. In my eyes, this is a huge downside and a massive argument against Amazons classical CloudFormation design.

All in all I don’t quite get the whys and reasons which made Amazons engineers believe that the abuse of JSON for scripting would be a better and more innovative idea than go for something like an established scripting language like Python or JavaScript.

But enough JSON-bashing for now. Let’s solve at least some of our AWS CloudFormation issues.

CFN-Sphere #

Currently there are a couple of tools around that sit on top of CFN as an abstraction layer and offer not just a YAML-based config approach, but also further functionality that comes in handy if you work with infrastructure as code during your everyday business.

One of those tools is CFN-Sphere, a lightweight Python-tool that aims to simplify AWS CloudFormation handling not just for infrasturcture automation but also for human interaction. Initially built by Marco Hoyer (a former colleague and friend of mine) CFN-Sphere comes with fundamental support for e.g. cross stack parameter references and automatic stack dependency resolution including circular dependency detection. Both are amazing features also for smaller AWS infrastructure definitions like the backup and staging environment for DynamoDB tables I am currently working on (More on that topic in one of the upcoming posts).

So if you are struggling with your with your CloudFormation config or if you have a certain feeling that CFN files having around several hundred lines of code are almost unmaintainable, make sure to check out CFN-Sphere and its stack template feature. And help making it even greater by becoming a contributor!

Written by Jan Brennenstuhl

Jan Brennenstuhl is a Principal Software Engineer, balancing security with friction for users. He helped building an IAM team and spent years in engineering single sign-on (SSO) solutions based on OIDC.