First impressions of the new CDK Pipelines module

Aug 9, 2020

All of the code that I write to run this site (and pretty much every project I work on) makes use of release pipelines to build, test and release my code. This helps me out in a lot of ways - firstly it helps make sure that my code is always built and tested in the same way and helps get away from the “it works on my machine” style of issue. It also means I’ve got a solid process for releasing my code across multiple accounts so that I have at least one non-production environment before I release things to prod. If you look around the industry it’s not hard to see that having a solid release pipeline is a common practice, where this conversation gets interesting is around how I implement them in AWS, which is worth covering before I get in to the new CDK Pipelines project further.

Why I use the AWS Code* suite and the CDK.

Beyond the “I just like AWS” response, one of the main reasons I personally use CodePipeline, CodeCommit and CodeBuild for my “Infrastructure as Code” projects is that I can define the pipelines themselves with the same type of code. AWS CloudFormation is the service that lets us define infrastructure to release in to AWS accounts and regions, and you can define your pipelines with the same type of template. This makes sense because I can get all the benefits of code templates for my pipeline infrastructure like version control, consistency and testability - but I can do it without needing to learn a new approach to get there. I use the same tools, processes and skills to create the template, and then their use is managed by the same permissions models, audit tools and configuration management mechanisms as my main infrastructure solutions.

With this in mind, I came very quickly to the AWS Cloud Development Kit (CDK) to create all of these templates. The CDK is a brilliant way to generate CloudFormation that lets me work in TypeScript (my current language of choice), rather than writing text templates in JSON or YAML. It makes it easier to run unit tests on my template generation logic, and the CDK has a number of best practices built right in, helping make sure I create more secure infrastructure with less lines of code - and less lines of code, means less for me to maintain. In the context of my release pipelines, this was especially important to me - I didn’t want to have to spend a lot of time and effort on keeping those pipelines up to scratch, because they aren’t where I’m adding value to my projects. The CDK made it pretty easy to create some pipelines for releasing my solutions, and you can see an example of one of my first CDK pipelines in my example GitHub repo. There were still some short comings to my approach that I hadn’t bothered to get around to in my initial solution though. The ability to change the pipeline easily was a challenge and required me to manually redeploy that stack by hand, but the bigger issue was that I had some issues with how I was able to use assets in the CDK for things like packaging lambda function code, docker images or files to deploy to S3, which limited the things I could do in my CDK projects, which prevented me from solving some problems I knew were in my apps.

Where CDK Pipelines comes in to it

To simplify this process for everyone, the CDK team have released the CDK Pipelines Developer Preview. Here’s a quick visual of what you get when you start using this for your CodePipeline setup.

An example of CDK Pipelines

The stages it generates for you run like this:

All of this for less than 50 lines of code - not a bad way to get your pipelines up and running if you ask me.

How the CDK Pipelines project helps

The pipelines project will set up your pipelines for you without you need to craft them in full yourself - meaning the GitHub example above that was a few hundred lines of code in total came down to ~50 lines of code for me, plus it solved my issues with self mutability of the pipelines, plus I get full use of CDK assets without me even needing to think about how I would package things! I’ve spent the last few days migrating all of my pipelines to this approach, which has seen me rip lots of code out for my own crafted pipelines - and as much as I love writing code, having less code always makes me happy.

The project is still in developer preview, but there is still a lot of things I really like about it already having put it to use:

  1. The bootstrapping to manage access across accounts is simple and highly secure. Once you’ve set up access to another account as a one off, everything else is done through assumed roles and there is no need for me as a developer to ever have a credential to the production account
  2. The CDK assets just work - I tested this out by adding a docker image for a CodeBuild project I was using and the pipeline just sorted it out. The first run through saw the pipeline modify itself to include a phase to build the docker image, and then when it restarted after the change things released just fine. Even better than that, when it went to the production account it just worked there as well!
  3. The amount of code needed to create these pipelines is tiny compared to crafting them yourself, so I spend less time coding up my pipelines and more time coding the applications themselves

That being said, there are still a few rough edges on it - a couple with workarounds, and a couple that are known issues of the preview:

  1. Context lookups (like Vpc.fromLookup()) aren’t supported yet - not something that I’ve come across as an issue yet, but important to know
  2. The synth build (where the CDK project is compiled) doesn’t yet have a graceful way to run unit tests. You can get the same result by setting the build command to run your tests as well (in NodeJS something like npm run build && npm run test will do the trick). There’s also no support for test report groups yet either.

Looking over the GitHub repo for this project there are a few other small bug reports coming in, as well as feature suggestions - so it’s safe to assume there will be some continual evolution of this project as the team works towards a stable release. I personally am a massive fan of this approach though, and if you’re using CDK projects for your infrastructure this is definitely one to have a look at.

 

Comments

You're signed in as | Sign out

Submitting comment...

There are no comments on this post yet. Be the first to leave one!