Testing on Physical Machines - Part 2

After introducing how to work with physical machines and Test Kitchen last time, we will look at a feature to allow central orchestration of available machines.

Hard-wiring the IPs of machines for our physical tests only works if there is a 1:1 relation between developers and available servers. Sadly, that is probably not the case in any project due to cost and efficiency reasons. Additionally, every advanced project has some pipeline architecture for automated testing.

For those reasons, kitchen-static v0.10.0 added a queueing feature.

Queueing via Scripts

To keep things simple, the first approach to relies on executing local scripts and processing their output. This way, you can use a Bash script, PowerShell script or even custom Ruby/Go tools to issue any sort of request or release.

driver:
  name: static
  queueing: true
  request:
    execute: request.sh
  release:
    execute: release.sh $STATIC_HOSTNAME

This is the easiest use case, where our request.sh script only returns a string like 203.0.113.17 and nothing else. Without any modifications, this will be written into your .kitchen/*.yml state file as new hostname.

Similarly, you can call a script with the acquired hostname/IP later to release it. For this, the environment variable STATIC_HOSTNAME will contain the value that Test Kitchen used (the prefix has been chosen to prevent variable clashes on Unix systems).

By default, both actions have a timeout of 3600 seconds. If you need more time, you can modify this with the timeout option on both request and release sections.

Adding Information

When working on physical machines and messing some settings up, you will probably be reliant on their Baseboard Management Controller/IPMI. If you are unaware of this feature: for years, most servers have had a built-in or optional embedded system which allows access to hardware parameters, video-/keyboard redirection and even attaching virtual devices. This way, IT technicians don’t need to drive to a remote datacenter every time something breaks: They can just log onto this independent system and work as if you were directly connected to their screen. This feature has a lot of vendor-specific names such as: BMC, ILO, iDRAC, IMM or AMT.

Only returning the IP for Test Kitchen would limit our ability to quickly get to such a BMC. We would need to do some calculations (IP + 100 maybe? Or was it IP \* 10?), look into some Wiki or contact an admin. That does not scale too well.

To avoid these issues, the kitchen-static drivers allows passing banner-style metadata on request actions and you can specify the parts to grab via regular expressions:

driver:
  name: static
  queueing: true
  request:
    execute: request_with_ilo.sh
    match_hostname: "^([.0-9]+):*"
    match_banner: ":(.*)$"

As an example, if the script returned the following string:

203.0.113.17: ILO accessible via http://203.0.113.170:8443/

This would result in the following Test Kitchen output:

-----> Starting Test Kitchen (v2.4.0)
-----> Creating <default-amazon2>...
       [kitchen-static] Queueing request via script handler
       [kitchen-static] Received 203.0.113.17 for testing
       [kitchen-static] >>> ILO accessible via http://203.0.113.170:8443/ <<<
       Finished creating <default-amazon2> (0m0.02s).
-----> Test Kitchen is finished. (0m1.15s)

As you can see, the feature is not limited to displaying BMC information, but could also give more details about the location of the server, who to contact on problems or display an acceptable use policy.

Extensions of the Feature

While the approach with executing scripts is suitable for most practical work, we can assume that most centralized systems for dispatching machines will use a modern RESTful API. There is an issue for adding a built-in REST handler on GitHub already: Implement REST API handler for Queueing · Issue #3. If you are looking forward to this feature, please add a thumbs up to enable proper prioritization.

In case you have specific requirements, you can also write your own handler and use the script handler as template. You can add other handlers to your Test Kitchen execution and use Glob expressions for adding them.

driver:
  name: static
  queueing: true
  queueing_handlers: 
    - /some-directory/handlers/*.rb