Release: Chef Container 0.2.0 (beta)

This week, Chef released a version of the Chef client that can run inside a Linux container. This container-friendly client is called chef-container. In this post we’ll give you an introduction to chef-container, its purpose and its components. We’ll also tell you about a new knife plugin for managing container images. Then, we’ll show you how to launch an instance of Apache2, running inside a Docker container.

Why chef-container?

Linux containers and containerization have been around for some time but the rapid rise in popularity of tools like Docker and the benefits they provide have brought containerization, and the concept of immutable infrastructure, back into the spotlight. For more on this I recommend you check out Julian’s post “Immutable Infrastructure: Practical or Not?”

Hosting your applications inside of Linux containers gives you more speed and flexibility than running them in VMs, but those containers still need proper configuration management. Most people choose something that goes beyond simple shell scripts. Docker’s Dockerfile has some characteristics of infrastructure as code but, deep down, it still defines actions to be taken as shell commands. For systems administrators and developers who want to migrate their existing application stack to containers, the prospect of rewriting all their infrastructure code for application stacks they may not fully understand can be daunting.

This is one of the reasons that we created chef-container. We believe that your automation platform should be able to manage your full stack. In other words, you should be able to use your existing library of cookbooks, whether you are managing the beefiest bare metal servers or the smallest Linux containers. You should be able to do all of this without losing any of the benefits that an automation platform provides.

What is chef-container?

chef-container provides a consistent configuration management experience across container solutions. It does this by bundling the Chef client with runit and chef-init. runit is a lightweight, cross-platform, open-source init system, and chef-init is a RubyGem that acts as the entry point into a container. It also provides the necessary abstractions to make configuring a container image the same as configuring any other piece of infrastructure with Chef. Together, these three pieces of software create a delightful container management workflow takes you from development to production.

The first release of chef-container addresses the unique execution environment inside Docker that, until now, required significant workarounds. By default, a Docker container lacks an init system as well as the ability to attach to a TTY inside the container. These characteristics made it difficult to bootstrap a Docker container or build a Docker image using the existing Chef workflow. chef-container, when used with the knife container plugin, resolves these issues and allows you build and launch Docker images while adhering to Docker best practices.

What is knife container?

The knife container plugin lets you use Chef to manage the lifecycle of container images. The first release of knife container provides the workflow specific to managing Docker images. It allows you to create, delete and manage Docker images.

Example: Launching an Apache2 Docker Instance

We’re going to show you how to launch an Apache2 Docker instance using chef-container and knife container.

Install the Software

You’ll first need to install three pieces of software on your workstation.

  1. Install Docker. You can do this from the Docker installation page.
  2. Install Chef DK. You can do this from the Chef Development Kit page.
  3. Install the knife container plugin with the following command:
    chef gem install knife-container

Plan Your Deployment

Once you’ve installed the software, there are four decisions to make.

  1. What do you want to name your Docker image?
  2. What run list will you use for the Docker image? In other words, what recipes will you use to configure it?
  3. Will you manage the image in local mode or server mode?
  4. Will you manage your cookbooks manually or with Berkshelf?

The Demo

To give you a concrete example, we’re going to create a Docker image and then launch it. We have a run list made up of a single recipe named apache2. We’ll manage the image locally and we’ll use Berkshelf to manage our cookbooks.

Initialize the Docker Context

To initialize the Docker context, use the init command. The knife container uses a folder called dockerfiles to organize all the Docker contexts that you manage. By default, the dockerfiles folder is created in your chef repo.

To initialize the Docker context, type the following command:

knife container docker init demo/apache2 -r 'recipe[apache2]' -z -b

Pass in your image name (in this example,demo/apache2), a run list, a –z and a –b. The –z is for local mode and the –b says to generate a Berksfile.

Define a Node Attribute

You need to define a node attribute for any service resource that you want Chef container to manage. chef-container is distributed with runit because Docker containers lack an init scheme. The chef-init gem provides the container_service resource that takes over the service resource and shims it to runit. Because runit works differently than your normal upstart service would, you need to provide an additional string value, which is the command that runit uses to run the service.

The command string is passed in through a node attribute and automatically recognized by chef-init, so you don’t need to modify existing cookbooks. You can specify the value anywhere node attributes can be specified. For this example we will enter it in the first-boot.jsn file that is created as part of the initialization process. Here’s the node attribute for our demo.

{
   "run\_list": [
      "recipe[apache2]"
   ],
   "container_service": {
      "apache2": {
         "command": "/usr/sbin/apache2 -k start"
      }
   }
}

Build the Docker Image

Use the build command to create a Docker image. The only value you’ll need to pass in is the image name you specified when you ran the init command. In our example cookbook dependencies are first automatically resolved because we’re using Berkshelf. Next, the command performs a docker build on the Docker context and generates a Docker image.

In this example, the name of the image was demo/apache2.

knife container docker build demo/apache2

Once the container is successfully built, you can see that the image exists with the following command:

docker images

Now you can push the image to a registry and launch it on any machine that you’d like.

docker push demo/apache2

Launch the Docker Image

You’re now ready to do a docker run on the Docker image to launch it as a running instance. Type the following command:

docker run –d demo/apache2

When the instance launches, it first runs the Chef client again for any last mile configuration changes that it needs. That run is convergent so it should be very fast. An important point to note is that, with Chef, you can configure containers after they’ve been launched. For example, if you have two containers that need to communicate with each other, you can only configure this after the launch because you won’t know their IP addresses until then.

To see a list of running containers, type:

docker ps

To see the processes running inside a container, type:

docker top container ID

Watch the Video

Watch a video that shows the demo.

Learn More

To learn more about Chef’s support for containers, read Chef for Containers.

To provide feedback on the chef-init or knife-container projects, please submit an issue via Github.

Author Tom Duffield

Tom is an Software Development Engineer with Chef. With Chef he uses his 10+ years of software development and 5+ years of consulting experience to help everyone become successful with Chef. When not working Tom spends his time knowing far too much about astronomy.

  • Sorry, but link to post “Immutable Infrastructure: Practical or Not?” is broken.

    • Hey Alexey – thanks for pointing that out. It has been fixed. Due to caching it may take awhile to appear but you can read it here: http://www.getchef.com/blog/2014/06/23/immutable-infrastructure-practical-or-not/

      • Grace M

        Seems that it puts the Dockerfile folder under .chef for me not in my chef-repo !

        • Hi Grace,

          The dockerfiles folder location is based on what chef/knife calculates your chef-repo folder to be. It does this (typically) by looking at your cookbookpath location and calculating it based off of that. If you want to specify your dockerfiles location, you can use the knife[:dockerfiles_path] setting in your knife.rb file: http://docs.opscode.com/configrb_knife.html

          • Grace M

            cool thanks

  • Rob Bastian

    Tom – It looks like the build command is trying pull my image from docker.io. Any idea what I’m missing?

    Uploading context 446 kB
    Uploading context
    Step 0 : FROM demo/apache2
    Pulling repository demo/apache2
    2014/07/15 20:59:48 Get https://index.docker.io/v1/repositories/demo/apache2/images: dial tcp: lookup index.docker.io on 192.168.1.254:53: no answer from server

    • Christophe Furmaniak

      I had the same problems…that seem to be related to the copy/paste of the “knife container docker init” command which brought strange – char.

      Try with “knife container docker init demo/apache2 -r ‘recipe[apache2]’ -z -b”

      • Thanks Christophe for pointing that out. The blog post should be fixed.

  • David Kinzer

    This is fantastic news! after making now I can have the best of both worlds, great configuration management via Chef and fast instance creation via Docker. I’m so happy!

  • Christophe Furmaniak

    Question: any dockerfile available to build our own base container with chef-init and dependencies?
    Nothing available at https://registry.hub.docker.com/u/chef/ubuntu-12.04/ :(

    • At this time no. In the future that is something we are looking to offer.

  • Hoang Do

    Hello, I’m following the tutorial, and succeeded to build the image.
    But then when docker run the image, it yelled: “File /etc/chef/secure/validator.pem is missing. Please make sure your secure credentials are accessible to the running container”.
    Why do we have to “RUN rm -rf /etc/chef/secure/*” in Dockerfiles ?

    • Secure credentials such as the validator key and the encrypted data bag secret are configured to exist in the /etc/chef/secure folder. Removing that folder is a security measure to prevent individuals from inadvertently uploading a Docker image with their secure data to the Docker Hub. For more information, you can read the docs here: http://docs.opscode.com/containers.html#credential-management

      If you specify the --include-credentials flag with the knife container docker init command, the Dockerfile command to delete the secure credentials will be omitted.

      • Hoang Do

        Thanks Tom, yes I agree that removing secret files is necessary.
        But in this error, chef-client supposes to NOT run again when we run the image (after build). I think I got that error because chef-client try to run when I launched the image, and it didn’t find the secure folder.
        Is it the case ? or I did something wrong with my configuration.

        • The error message you received was thrown by chef-init. chef-init killed your container before it launched Chef Client because it detected that your validator key was missing.

          • I don’t understand, chef-init / chef-client suppose to run only one time when we BUILD the image, right? Now it runs even when I RUN the built image.

            When I keep –include-credentials, chef-client run everytime when I type “docker run”. Should we remove chef-client from the image after building

          • The knife-container workflow, by default, will configure your Dockerfile to run chef-client twice: the first time during the build phase when it runs chef-init –bootstrap; the second time when you launch the container when it runs chef-init –onboot. If you don’t want chef-client to run then at this time you will need to modify the ENTRYPOINT and CMD in the generated Dockerfile to be something other than chef-init and –onboot.

          • Thanks Tom, I’m understand the workflow now.

  • Hoang Do

    Is it true that chef-container only support ubuntu (chef/ubuntu)? How can I create an debian base image with chef-init.

    • We are currently working on a way to allow people to install chef-container into their own container images. However, in the meantime, i will make sure to include a pre-built chef-container debian image in the next release. Please stay tuned!