Air-Gapped Compliance Scans with InSpec

Chef InSpec offers the possibility to scan any type of device using community-authored compliance profiles. As InSpec needs to connect to the machines, networking and firewall rules make this task tricky in bigger environments.

With the newly released train-awsssm plugin, this situation changes drastically. Read on, if you want to know how.

InSpec Profiles

Chef InSpec is a tool which complements the configuration management aspect of Chef Infra. While Infra sets configuration of a system to a desired state, Chef InSpec is accountable to verify that the current state complies to best practices. The most popular examples for this are guidelines by the Center for Internet Security (CIS) or NIST security technical implementation guide (STIG). There are other examples of checking the current patches as well.

In general there are three categories of InSpec Profiles:

• commercial (sold by Chef as Premium content and audited by CIS and other organizations)
• opensource (e.g. dev-sec.io or MITRE provide several)
• self implemented profiles

Writing your own profiles is pretty easy, as InSpec offers a readable domain specific language:

describe package('apache2') do
  it { should be_installed }
end

describe service('apache2') do
  it { should be_running }
  it { should be_enabled }
end

The good thing is, that InSpec does not need to be installed on the machines you want to monitor ¹ - it uses a system of executing commands via a transport mechanism like SSH or WinRM. In addition, you can just reference profiles by URL and can even point it to Git repositories.

If you want to scan a Linux machine, you could do this:

# Scan a remote Linux machine
inspec exec https://github.com/dev-sec/linux-baseline/archive/2.5.0.tar.gz \
            --target ssh://user@10.20.30.40

# Scan a remote Windows machine
inspec exec https://github.com/dev-sec/linux-baseline/archive/2.5.0.tar.gz \
            --target winrm://user@10.20.30.41 \
            --password secret

The result will be an extensive check and report for the profile. You can even select the format of the reports (text, json, junit or even fancy HTML are possible). Of course you need to provide some sort of credentials during the setup ²

Networking

There are some obvious problems with this architecture. Being a push-based system and not relying on an agent means that you need to be able to reach the machines under test.

In AWS, that can pose some challenges. Hopefully, your machines are not publically accessible but safe and sound in a private subnet? That means you need to have either a local runner for the scans per VPC or have it a central one in a shared services VPC, possibly connected via Transit Gateway.

But one the big problems is possibly maintainability, especially the need to open ports for the scans and create a user locally which can connect. A good pattern for this is (of course) the use of configuration management on the machines to set up the scanning user and adding a separate Security Group which allows TCP/22 (SSH), TCP/5985 (WinRM) from the central scanner.

Train and AWS Systems Manager

To find a better way without needing network connectivity or firewall rules, let’s dive deeper in how InSpec actually works:

InSpec includes a fully fledged plugin system, which adds the opportunity to add custom functionality to the description language (DSL Plugins), extend it’s CLI, inject input parameters or change the type of resulting report. More interestingly, it uses a complete abstraction for connecting to machines under the hood - which is called Train.

Train provides several transports for connecting to systems which generally fall into two categories:

• line oriented (SSH, Docker, Local execution, WinRM, …)
• API oriented (AWS, Azure, GCP, VMware, …)

While the most common ones are bundled with InSpec, there are also community-provided transports like Kubernetes, LXD, Habitat or even exotic ones like Serial/USB and Telnet ³.

Thinking of Amazon Web Services, the AWS Systems Manager also provides a way of executing commands remotely: Run Commands.

As you might know, the SSM Agent is preinstalled on many of the common AWS AMIs already and you only need to assign an IAM profile to your instance to take advantage of it. Combining useful functionality like software inventory, patch management and package distribution, many customers already have SSM in daily use.

To connect these two worlds, I wrote the Train AWS-SSM Transport a while ago which can now be used with InSpec thanks to today’s 0.2.0 release.

This means that you can now scan EC2 instances simply by:

gem install train-awsssm

inspec exec https://github.com/dev-sec/linux-baseline/archive/2.5.0.tar.gz \
            --target awsssm://10.20.30.40

To specify the instance to scan, you can just use the public/private IP, a resolvable hostname or the instance ID with this transport plugin.

This way you do not need to configure networking, VPC peering/Transit Gateways or Security groups. Simply have a role on your InSpec runner (or some other credentials) and enable it to execute Run-Commands.

Of course there are caveats with this way of scanning:

• Run Commands are always executed with administrative privileges as they are also used for software installation and other tasks.
• as InSpec is issuing every scan command by itself, the overhead of sending commands and waiting for response stacks up. Running a Windows baseline scan can be around 30 minutes, while the normal WinRM-based scan is much faster.

Future

To address the problems with executing scans as an administrative user, I plan to include proper Session Manager support in the next version. As Session Manager can at least on Linux use the “Run As” feature, this reduces security concerns. Also, execution will be much faster as it is effectively channeling SSH/WinRM through the SSM Agent instead of single commands.

Feel free to open up issues for your discovered bugs or desired additional features on the train-awsssm GitHub page.