Chef 0.10 Preview: Encrypted Data Bags

Chef 0.10 Preview: Encrypted Data Bags

Many server roles, such as databases, require setting up passwords as
part of their configuration. A common pattern is to put such
passwords in a data bag stored on the Chef server. Recipes can then
access the data bag to automate the configuration.

A downside of this approach is that passwords are stored in plain text
on the Chef server and also within your version control system if you
use one to manage your cookbooks, roles, and data bags.

In Chef 0.10, you can use Encrypted Data Bags to reduce the exposure
of sensitive data stored in data bags to the servers that need it
for their configuration and the operators that need to modify it.
Here’s what you need to know about encrypted data bags before we jump
into a few examples:

  1. Only the values of an encrypted data bag are encrypted. The keys
    remain in plain text so that you can still use search (on the keys) to
    find items in encrypted data bags.
  2. All values will be encrypted. If you have a data bag containing
    sensitive and non-sensitive information, split it into two separate
    data bag items.
  3. The encryption relies on a shared secret. At this time, you are
    responsible for creating and distributing the shared key. For
    example, you will need to devise a mechanism of making the key
    available to servers that need to decrypt values for their

Using knife to create an Encrypted Data Bag

Before we can create an Encrypted Data Bag, we need a secret key.
We’ll use openssl to generate a random secret.

[sourcecode]$ openssl rand -base64 512 > secret_key

Now we can create an encrypted data bag item within the “prod” data

[sourcecode]$ knife data bag create –secret-file ./secret_key prod passwords
# edited data bag item in editor and saved
Created data_bag[prod]
Created data_bag_item[passwords]

(Note: there are 2 dashes (-) before secret in the above command)

This will pop open your editor. Here’s what I filled in for the example:

"id": "passwords",
"mysql": "open-sesame-123",
"rabbitmq": "open-queue-123"

Let’s take a look at the result without decrypting (if you are
following along, your values will depend on the secret used):

[sourcecode]$ knife data bag show prod passwords
id: passwords
mysql: xtSxLvqHqPP1gHsqP5SlytFtDIfpWMJebJ2aZPd0mGU=
rabbitmq: wVcK/OboqpRcfF5fOKlEHKz2ev7CxSrBsoCwWo9Jcko=

To see the decrypted values, specify the secret key:

[sourcecode]$ knife data bag show –secret-file=./secret_key prod passwords
id: passwords
mysql: open-sesame-123
rabbitmq: open-queue-123

(Note: there areĀ 2 dashes (-) before secret in the above command)

Next, we’ll look at how to use encrypted data bags within recipes for
node configuration.

Accessing Encrypted Data Bags from a Recipe

Recipes can access decrypted values using
Chef::EncryptedDataBagItem.load. If you don’t specify a shared
secret when calling the load method, Chef will look for a file based
on the value of the Chef::Config[:encrypted_data_bag_secret] config
entry (which you can set for a node via /etc/chef/client.rb). The
default value is /etc/chef/encrypted_data_bag_secret.

To demonstrate the use of encrypted data bags on a node, we’ll start
by copying the secret_key file created above to our example node
using scp and moving it to /etc/chef/encrypted_data_bag_secret.

[sourcecode]scp ./secret_key $MY_NODE_IP:~/
sudo mv ./secret_key /etc/chef/encrypted_data_bag_secret

Next, we’ll create a recipe that will log the decrypted values for
demonstration purposes (if these were real secrets, you would want to
avoid logging them).

[sourcecode]knife cookbook create edb_demo

Now edit cookbooks/edb_demo/recipes/default.rb so that it contains
the following:

[sourcecode lang=”ruby”]# cookbooks/edb_demo/recipes/default.rb
passwords = Chef::EncryptedDataBagItem.load("prod", "passwords")
mysql = passwords["mysql"]"The mysql password is: ‘#{mysql}’")

Finally, upload the cookbook and run chef-client on the node. You
should see something like this:

[sourcecode]knife cookbook upload edb_demo
# output clipped
knife ssh name:i-8a436fe5 -a ec2.public_hostname ‘sudo chef-client’
INFO: *** Chef 0.10.0.rc.2 ***
INFO: Run List is

INFO: Run List expands to [edb_demo]
INFO: Starting Chef Run for i-8a436fe5
INFO: Loading cookbooks [edb_demo]
INFO: The mysql password is: ‘open-sesame-123’
INFO: Chef Run complete in 3.122228 seconds
INFO: Running report handlers
INFO: Report handlers complete

As you can see, the recipe was able to decrypt the values in the
encyrpted data bag. It did so by using the shared secret located in
the default location of /etc/chef/encrypted_data_bag_secret.

Wrap Up

Encrypted Data Bags are an easy way to add a layer of security to
protect sensitive configuration data like passwords and cloud
credentials. You can use knife to create and view encrypted data
bags. By placing the shared secret in a known location on a node, you
can allow the node to access the protected data via a recipe. You can
learn more about Encrypted Data Bags and some of the additional
options they support on the Chef Wiki

Author Seth Falcon

  • Pingback: Chef 0.10.0 Released! |

  • Instead of scp, what I wanted was an automated way to deploy the secret key without installing it from a recipe. So what I decided to do was use a bootstrap template to write the file to the server. For example, in knife.rb, you setup a new config value,

    encrypted_data_bag_secret “#{ENV[‘HOME’]}/.chef/encrypted_data_bag_secret”

    and then in the bootstrap template, you can add the following before the line,

    cat <<'EOP'

    ) > /tmp/encrypted_data_bag_secret
    awk NF /tmp/encrypted_data_bag_secret > /etc/chef/encrypted_data_bag_secret
    rm /tmp/encrypted_data_bag_secret

  • Pingback: Opscode’s First Chef Cookbook Contest! |

  • Linuxbass

    I like this feature although nowhere do I see a from file explanation of the feature which is my view is what would make this worth doing. At this point the only difference between encrypted data bags and regular data bags is that they exist within chef server as encrypted. The feature explanation says that encrypted data bag json’s can be done which woudl be great to put into SVN or GIT. But nowhere do I see how to make an encrypted data bag json. Or a from file explanation.

    How is an encrypted data bag file generated. How does one make an encryped data bag json to within a chef repo?

  • Linuxbass

    OK, found it in
    jtimberman’s code blog

    Once an encrypted data bag is created it can be put into a file like so:

    knife data bag show passwords mysql -Fj >data_bags/passwords.json

    Now I really like this feature and will be using it extensively in systems architecture.