CDK Infrastructure Testing - Part 2b - Unit, Integration and Application Test for Serverless Lambda Functions

After describing the context of the test pyramid for Infrastructure as Code in part 1, and the Web Application in Part 2a - let`s apply that to some Lambda function.

See all CDK Infrastructure Testing posts here

Three Taste Lambda Testing

We will use https://github.com/tecracer/cdk-templates/tree/master/go/lambda-go.

Because of the way GO handles modules and packages, I separate the lambda appand the CDK infrastructure infra.

├── Taskfile.yml
├── app
│   ├── Taskfile.yml
│   ├── go.mod
│   ├── go.sum
│   └── main
├── dist
│   ├── main
│   └── main.zip
├── infra
│   ├── README.md
│   ├── Taskfile.yml
│   ├── cdk.json
│   ├── cdk.out
│   ├── go.mod
│   ├── go.sum
│   ├── lambdago.go
│   ├── lambdago_test.go
│   ├── main
│   ├── stacks.csv
│   └── testdata
├── readme.md
└── stacks.csv

Overview

• Taskfile.yml is used as a makefile
• app the “HelloWorld” LambdaFunction
• app/Taskfile.yml taskfile with the fastdeploy to Lambda (see Blogpost about Lambda deployment)
• dist used to store the Linux build Lambda binary
• infra The CDK app

The definition of the Lambda function in CDK is rather short:

  	(1) lambdaPath := filepath.Join(path, "../dist/main.zip")
  	awslambda.NewFunction(stack,
	(2) aws.String("HelloHandler"),
		&awslambda.FunctionProps{
			MemorySize: aws.Float64(1024),
		(3) Code: awslambda.Code_FromAsset(&lambdaPath, &awss3assets.AssetOptions{}),
			Handler: aws.String("main"),
			Runtime: awslambda.Runtime_GO_1_X(),
		})

Where

1. the path where I put the zipped Lambda Function GO binary
2. Handler
3. the code for “just take the zip and upload it”

You see, that I point the code to the ZIP file, which includes the build GO app. IMHO this is the most efficient way to deploy GO lambda.

When you are developing the Lambda function code, you build the code. The extra second to build the ZIP file is well spend. No need for Containers like in Node.JS or Python.

Lambda Unit Test

The Unit test in CDK level tests the generated CloudFormation. In development we want to skip the integration test at first call, to test step by step. To achieve that we set the “short” flag when calling go test. This go test -short -v call is prepared in the Taskfile.ymlso you can call it from there:

task test-unit
=== RUN   TestLambdaGoStack
--- PASS: TestLambdaGoStack (8.42s)
=== RUN   TestLambdaGoCit
    lambdago_test.go:42: skipping integration test in short mode.
--- SKIP: TestLambdaGoCit (0.00s)
=== RUN   TestLambdaGoApp
    lambdago_test.go:53: skipping integration test in short mode.
--- SKIP: TestLambdaGoApp (0.00s)
=== RUN   TestLambdaGoAppCit
    lambdago_test.go:84: skipping integration test in short mode.
--- SKIP: TestLambdaGoAppCit (0.00s)
PASS
ok  	lambdago	9.038s

The test is defined in infra/lambdago_test.go:

func TestLambdaGoStack(t *testing.T) {
	// GIVEN
	app := awscdk.NewApp(nil)
	// WHEN
	stack := lambdago.NewLambdaGoStack(app, "MyStack", nil)
	// THEN
	bytes, err := json.Marshal(app.Synth(nil).GetStackArtifact(stack.ArtifactId()).Template())
	if err != nil {
		t.Error(err)
	}
	template := gjson.ParseBytes(bytes)
	lambdaruntime := template.Get("Resources.HelloHandler2E4FBA4D.Properties.Runtime").String()
	assert.Equal(t, "go1.x", lambdaruntime)
}

As explained above the CloudFormation Template is generated and the Runtime of the Lambda Function is checked:

lambdaruntime := template.Get("Resources.HelloHandler2E4FBA4D.Properties.Runtime").String()
assert.Equal(t, "go1.x", lambdaruntime)

Integration and App test are skipped, they will FAIL. And they should fail without deployed Stack!

Lambda Integration Test

After testing the generated Templates, we deploy the template, create the AWS resources and test them.

Deploying the Stack with:

task deploy
Profile <yourprofilename>
task: npx  cdk@v2.0.0-rc.7  deploy -c stage=dev --require-approval never --profile $AWSUME_PROFILE
LambdaGoStack: deploying...
...

If you are not using awsume, export your profile name in AWSUME_PROFILE.

Calling the tests:

 task test-infra
task: go test  -v
=== RUN   TestLambdaGoStack
--- PASS: TestLambdaGoStack (7.92s)
=== RUN   TestLambdaGoCit
--- PASS: TestLambdaGoCit (0.57s)
=== RUN   TestLambdaGoApp
--- PASS: TestLambdaGoApp (0.54s)
=== RUN   TestLambdaGoAppCit
--- PASS: TestLambdaGoAppCit (0.18s)
PASS
ok  	lambdago	9.825s

The integration/infrastructure test, cit-enabled:

func TestLambdaGoCit(t *testing.T){
	(1) if testing.Short() {
        t.Skip("skipping integration test in short mode.")
    }
	(2) gotFunctionConfiguration, err := citlambda.GetFunctionConfiguration(aws.String("LambdaGoStack"),
	aws.String("HelloHandler"))
	(3) assert.NilError(t, err, "GetFunctionConfiguration should return no error")
	expectHandler := "main"
	assert.Equal(t, expectHandler, *gotFunctionConfiguration.Handler )
}

Where

1. skips this test if called with “short” to suppress integration tests
2. Get a FunctionConfiguration Lambda object with the name of the CDK stack and the Construct ID of the Lambda function. With the citlambda package its one line.
3. Test - as an example - that the handler is really set to “main”

You get an FunctionConfiguration data structure by calling GetFunctionConfiguration with the stackname and Construct ID.

Lambda Application Test - handmade automation

First we go to the longer way, after that I show you the short way with cit.

This HelloWorld lambda function is simple. For real world tests we would have some json input. Here we have this json object:

{
    "key1": "value1",
    "key2": "value2",
    "key3": "value3"
}

It is stored in testdata/test-event-1.json.

We call the deployed Lambda function with the application test (this will take some time).

The file is lambdago_test.go

1. Get the Function Configuration

gotFunctionConfiguration, err := citlambda.GetFunctionConfiguration(
	aws.String("LambdaGoStack"),
	aws.String("HelloHandler"))
assert.NilError(t, err, "GetFunctionConfiguration should return no error")

This assertion test if the Lambda Resource exists.

2. Get a SDK Lambda Client for invoking the function

cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
 panic("configuration error, " + err.Error())
}
client := lambda.NewFromConfig(cfg)

3. Read the test event

functionName := gotFunctionConfiguration.FunctionName
data, err := ioutil.ReadFile("testdata/test-event-1.json")
if err != nil {
	t.Error("Cant read input testdata")
	t.Error(err)
}

4. Invoke the function

params := &lambda.InvokeInput{
	FunctionName:   functionName,
	Payload:        data,
}
res, err := client.Invoke(context.TODO(), params)

5. Check the response

assert.NilError(t, err, "Invoke should give no error")
assert.Equal(t,"\"Done\"",string(res.Payload))

In the payload you can check details of the response. If you have different test cases, you define different events and call the lambda function. That is exactly the same as if you invoke the Test from the console:

With the result:

The difference is that it is fully automated. Doing regression testing, clicking through, let’s say 5 different test in the console takes time. And to be honest - you would not do it.

But - it took us 30 lines of code for one test. Lets make that shorter:

Lambda Application Test - cit automation

func TestLambdaGoAppCit(t *testing.T){
	if testing.Short() {
		t.Skip("skipping integration test in short mode.")
    }
	
	payload, err := citlambda.InvokeFunction(
		aws.String("LambdaGoStack"),
		aws.String("HelloHandler"),
		aws.String("testdata/test-event-1.json" ))
	assert.NilError(t, err, "Invoke should give no error")
	assert.Equal(t,"\"Done\"",*payload)
}

If you have a second test event, you just add

	payload, err := citlambda.InvokeFunction(
		aws.String("LambdaGoStack"),
		aws.String("HelloHandler"),
		aws.String("testdata/test-event-2.json" ))
	assert.NilError(t, err, "Invoke should give no error")
	assert.Equal(t,"\"SecondAnswer\"",*payload)

and so on.

That is much easier.

I think its so easy that is worth defining some Lambda Application test in GO for your TS/Python/Java/etc Lambda function! Or you take the concept and code your own cit.

The End

I hope this concept or the cit framework implementation will help you with your projects also. For the last couple of projects I started with an integrated or application test and found it quite useful.

Some of the other integration test I used were:

• AWS Workspaces and Workspaces User creation
• AWS Transfer sftp User and read/write test
• Application Load Balancer with Domain and installed software
• Creation of Lambda bases Custom Resource

What about your projects?

For discussion etc please contact me on twitter @megaproaktiv

Appendix

The repositories

Cit - CDK Integration Testing

• https://github.com/megaproaktiv/cit

cdkstat - Show CloudFormation Stack status

• https://github.com/megaproaktiv/cdkstats

CDK Templates using CIT and terratest for testing

• https://github.com/tecracer/cdk-templates

Terratest

Quick Start

• https://github.com/gruntwork-io/terratest

The tools

Awsume

• awsume
• Taskfile

Thanks

Photo by Nathan Dumlao on Unsplash