The kitchen-ec2 Driver

Within the Chef ecosystem, Test Kitchen is one of the most useful tools. It offers the possibility to quickly test cookbooks in different OS environments on machines with a limited lifetime. That way, you can check if your fancy recipes work the same on RedHat, Centos 6 and Ubuntu. As speed is king, this fast feedback motivates more for early testing and reduces the amount of bugs found in production.

Intro to Kitchen

Test Kitchen covers the full lifecycle of a VM:

• create a new machine
• converge by setting up Chef/ChefDK, uploading cookbooks and executing them
• verify if everything was done as intended (mostly using InSpec tests)
• destroy the machines after testing to avoid inconsistencies later on

All this can also be done in a single step via kitchen test, which will run through the whole cycle. When a machine succeeds, it will be deleted immediately (as it has served is cause) but a failed machine will be left running for diagnostics.

(Side note: if you want to remove the failed machines as well, which is common to CI/CD pipelines, you can use kitchen test --destroy always)

There are other posts detailing how to work with Test Kitchen (TK), configure different platforms and suites. In this series, our focus will be on the “driver” layer of TK which handles the (de)provisioning of virtual machines only.

kitchen-ec2

While it is feasible to use local VMs, mostly via kitchen-vagrant, this quickly hits limitations as memory on a developer workstation can be fairly limited. In addition, an office full of laptop fans roaring at maximum speed does not make for a cozy work atmosphere.

One of the most popular drivers for using Cloud-based VMs is kitchen-ec2 which connects to Amazon Web Services accounts to run your tests. This allows you to easily launch 20 different VMs in parallel to check for different combinations of cookbook settings and operating systems.

All those VMs only run for a few minutes and are paid for this duration only. Also, there is no permanent infrastructure needed which makes it an easy and cheap way to run your tests during development.

An example configuration for this driver would be:

driver:
  name: ec2
  region: eu-west-1
  aws_ssh_key_id: keypair_name_for_connecting
  subnet_id: subnet-12345678
  instance_type: t3.small
  
  # If your cookbooks need some privileges, having a custom role is useful
  iam_profile_name: ChefKitchen
  
platforms:
  - name: rhel-7
  - name: centos-7
    driver:
      image_search:
        name: CentOS Linux 7 *
        owner-alias: aws-marketplace
        architecture: x86_64
        virtualization-type: hvm
      block_device_mappings:
      - device_name: /dev/sda1
        ebs:
          volume_size: 20
          delete_on_termination: true

This example is a bit verbose, but also show some of the possibilities to customize your setup. Notice that the RHEL7 platform does not need any additional information (as lookup of base AMIs is built into the driver), but you can specify a lot of overrides or search expressions within an additional platform-specific driver section.

Our objective with testing is not only coverage, but especially quick feedback. There are some rules of thumb stating that tests running longer than an average coffee break will lead to developers switching tasks and losing focus.

Even though EC2 instances boot up quicker than some local VMs, we could further optimize this by including the needed software in our used images instead of waiting for Test Kitchen to install ChefDK every time we start a VM (losing 2-3 minutes each time). You will want to test with a specific Chef version anyway, to avoid having development drifting into newer, more feature-rich versions than are available in your real environment. So, baking this into the AMIs is a valid approach

Dev/Prod Parity

This stresses one important point: The paradigm of “Dev/Prod Parity” states, that you should always test under the same circumstances that your production system runs in. If you work with on-premise production systems, using AWS will not only differ in the exact OS configuration, but also the whole networking setup including Proxies and DNS. This might mean that you face bugs in good old production systems that could never occur in your fancy AWS environment.

Of course, with additional tooling you could bring AWS and your On-Prem site more into sync. One popular choice is Hashicorp Packer, which allows you to build images once (possibly including Chef already) and deploy them into different datacenters or clouds. That would still lead to different network setups though.

In the next post of this series we will have a look at an alternative driver called kitchen-vcenter which can help you out in this area by testing in your regular datacenter. It is also capable to have complete Windows VMs up and running within less than 20 seconds!