Chef’s Role in Your CI/CD Pipeline

Editor’s note: This is a contributed guest post from our partners at Levvel who help their clients to successfully implement digital transformations in their organizations.


You have a lifecycle for your application code—shouldn’t you have a lifecycle for your configuration as code, as well? And everyone loves being meta; how meta would it be to have your configuration as code set up the pipeline for deploying your configuration?

With Chef, you can provision the CI server of your choosing, like Jenkins, TeamCity, or even Gitlab-CI (via GitLab runners). If you really want to get fancy with some of these, you can have the Chef specific builds and jobs created and managed via Chef.

Aside from this Inception-esque meta overload, the real point of CI is increasing your confidence in your code. There are plenty of tools for different lifecycle points to increase your confidence in your Chef recipes (related units of work in Chef). At the most basic, you’ll probably want to lint them with a tool like FoodCritic, and you’ll likely want to create some unit tests with ChefSpec. These are fairly trivial to set up as part of your pipeline.

Going a step further, it’s a great idea to test your recipes by actually running them, but you don’t want to do that on real infrastructure. With TestKitchen and InSpec, you can bring up some Docker containers (or VMs, etc.) that mimic pieces of your infrastructure, and then run integration tests that ensure the bigger picture is working properly.

You’re going through all of this effort to have confidence in your recipes, you don’t want them to accidentally run in upper environments. You’ll definitely want to version your recipes and ‘freeze’ them per environment. With Chef, you can freeze an environment to a version, and that’s the only version of your recipes that can run in that environment until you explicitly bump the frozen version. When your confidence level meets your threshold, you can promote your recipes to a higher environment safely.

So now your configuration pipeline is running just like your application code pipeline, but we’re going to muddy the meta waters even further. What about deploying application code via Chef, or even deploying Chef via Chef (via CI)?

Chef supports capistrano-style deployments, which is a very polarizing topic, as there are some who don’t believe this is the job of configuration management tools. If you want to use the Chef deploy resource, it is there. If you don’t, that doesn’t mean that Chef can’t help out with deployments. At a minimum, Chef can easily install your deployment tool of choice, for example AWS Code Deploy, and configure it for whatever you plan to deploy to that machine.

If you want to get fancy, you can use a tool that is bundled with Chef for deployment called “knife”. You can easily take your current deployment scripts, and let your CI server use knife SSH to run those scripts on all the servers that you need to deploy to. Technically, you aren’t using your configuration management to deploy, so your opinionated co-workers won’t string you up from the nearest yardarm. And, you don’t have to reinvent the wheel since you can use the same scripts you’re already running.

Taking it to the extreme, have your CI server push your recipes to Chef-Server, bump the frozen version for the environment you wish to deploy, and have your Chef-Client configured to run on intervals (say, every five minutes). Now after all the linting, unit tests, and integration tests on your freshly baked recipe, CI updates Chef-Server, and technically it’s the Chef-Client run that actually deploys (pulls, in this case) that code to the servers and executes it. You now have Chef, provisioning your CI, with the CI jobs to test Chef recipes and promote Chef to the Chef-Server, and Chef-Client is pulling recipes and running them on your infrastructure.

Chef can fill an important role in the CI/CD pipeline, and is flexible enough to fit into your own pipeline. It is my go-to provisioning tool and you should definitely take a hard look at using it too.

Author Brandon Dennis

Brandon has been involved with DevOps since 2012, and using Chef at various companies since before Chef Server was rewritten in Erlang. He has given multiple talks on different aspects of Chef, and now works for Levvel as a DevOps consultant to a Fortune 100 company.