CDK Lambda Deployment takes about a minute - how about sub second Function Code Deployment?
CDK Lambda Deployment takes about a minute - how about sub second Function Code Deployment?
Creation of Lambda infrastructure with the CDK is really powerful. Updating the Function code is really slow. Here is a fix for that to get to a sub-second Lambda function deployment time.
Setup
Let`s assume we have a simple lambda function we want to test in the cloud by invoking it from the AWS console. We deploy the code to Lambda often, so the deployment should be as fast as possible.
We start with using a cool CDK Lambda Infrastructure Construct, assuming we have a Lambda function called “warmkalt” - the german word for weather in April, “warm and cold”:
const image = DockerImage.fromRegistry("golang:latest")
const backend=new Function(this, 'warmkalt', {
runtime: Runtime.GO_1_X,
handler: 'main',
code: Code.fromAsset(join(__dirname, '../code'),
{
bundling: {
user: "root",
//included GO version to old
//image: Runtime.GO_1_X.bundlingDockerImage,
image,
command: [
'bash','-c', [
'go build -mod=mod -o /asset-output/main main/main.go' ,
].join(' && '),
],
environment: environment,
}
}),
memorySize: 1024,
});
As we like the newest (1.16 at this time) GO version, we use a custom image with the latest GO version. The bundle version with the CDK is older.
We have the following Lambda code in code/main.go. For your own deployment you can use any code and language, you just have to have a build script.
Function Code for Function “warmkalt”
package main
import (
"fmt"
"github.com/aws/aws-lambda-go/lambda"
"os"
)
func hello() (string, error) {
_, err := os.Stat("/tmp/test.txt")
if os.IsNotExist(err) {
fmt.Println(err)
fmt.Println("File does not exist, creating file.")
f, _ := os.Create("/tmp/test.txt")
defer f.Close()
_, err := f.WriteString("Hello World")
if err != nil {
fmt.Println(err)
}
} else {
fmt.Println("File exists.")
}
return "Hello ƛ", nil
}
func main() {
lambda.Start(hello)
}
The code shows that a file created during the Lambda cold start is still there with a warm start.
We deploy it the first time, the resources are created, everything fine.
Update Code in a long Minute
We now change only a single line and deploy again with the cdk:
cdk deploy --force --require-approval "never"
This takes about a minute.
The time output:
5,86s user 0,78s system 10% cpu 1:04,33 total
That takes too long!
Next step - using the AWS cli.
Update code in 4 seconds
Now we build the code separately, so that CDK does not have to build it.
GOOS=linux go build -o dist/main main.go
This is valid for a real life development cycle, because we only deploy pre-build function code. If the build does not work, we would not deploy it. If you work inside a CI/CD pipeline, just stick with the CDK..
Now we can deploy the code build into the static binary directly with the update-function-call:
zip function.zip main
aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip
The median from 5 iterations gives a 3.61 seconds deploy time, which is about 18 times faster.
(This is a german excel, so “1,01” means “1.01”)
Update Code in 3 seconds
How to speed this up even more?
We could zip the build (or build the zip) in the background after each build cycle!
With Task and other build systems it is possible to watch for file changes, here with --watchas a parameter.
build-zip:
desc: Builds lambda
cmds:
- clear && date
- GOOS=linux go build -o dist/main main.go
- cd dist && zip function.zip main
sources:
- main.go
generates:
- dist/main
- dist/function.zip
silent: true
ignore_error: true
When we start task with:
task build-zip --watch
It will update the dist/funktion.zip each time the main.go is changed.
So we change the deploy code (also in Taskfile.yml) as follows to deploy the pre-build zip:
deploy2:
dir: dist
deps: [ build-zip ]
desc: Deploy from local disc
cmds:
- aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip
Update Code in 2 seconds
Updating from a S3 Bucket should be even faster. So we upload the zip file to s3 also in the background:
build-zip-s3:
desc: Builds lambda
cmds:
- clear && date
- GOOS=linux go build -o dist/main main.go
- cd dist && zip function.zip main
- cd dist && aws s3 cp function.zip s3://cdktoolkit-stagingbucket/functions/warmkalt.zip --quiet && echo "Upload complete"
sources:
- main.go
generates:
- dist/main
- dist/function.zip
silent: true
ignore_error: true
and run
task build-zip-s3 --watch
To deploy we use:
deploy3:
dir: dist
deps: [ build-zip-s3 ]
desc: Deploy from s3
cmds:
- aws lambda update-function-code --function-name "warmkalt" --s3-bucket cdktoolkit-stagingbucket --s3-key functions/warmkalt.zip
And now we are down to 1,56 seconds, about 40 times faster.
Sub 1 second
The remove the last slow step, we replace the aws-cli with a less than 30 lines GO programm:
package main
import (
"context"
"fmt"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/lambda"
)
func main() {
cfg, err := config.LoadDefaultConfig(context.TODO())
if err != nil {
panic(err)
}
client := lambda.NewFromConfig(cfg)
params := &lambda.UpdateFunctionCodeInput{
FunctionName: aws.String("warmkalt"),
S3Bucket: aws.String("cdktoolkit-stagingbucket"),
S3Key: aws.String("functions/warmkalt.zip"),
}
response, err := client.UpdateFunctionCode(context.TODO(), params);
if err != nil {
fmt.Println("Error: ", err)
}else{
fmt.Println("Deploy:",response.LastUpdateStatus)
}
}
This just calls the UpdateFunctionCode Lambda api with the parameters, where to find the zip on the bucket.
We compile it in the binary tools/deploylf with go build -o deploylf main.go
The task changes to:
deploy4:
deps: [ build-zip]
desc: Deploy with go programm
cmds:
- tools/deploylf
Now we are down to 0.6 seconds for a Lambda Function deployment, wich is about 100times faster than the CDK deployment.
Fast Edit setup
So when i save the code:
The build is started, zipped & uploaded in the background in the build&upload window.
Now i can deploy the Lambda function code in less than one second:
If the update process is not running we “slow back” to 6 seconds:
Taskfile as whole, try yourself!
# https://taskfile.dev
version: '3'
tasks:
build:
desc: Builds lambda
cmds:
- clear && date
- GOOS=linux go build -o dist/main main.go
sources:
- main.go
generates:
- dist/main
silent: true
ignore_error: true
deploy1:
dir: dist
deps: [ build ]
desc: Deploy from local disc
cmds:
- zip function.zip main
- aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip
build-zip:
desc: Builds lambda
cmds:
- clear && date
- GOOS=linux go build -o dist/main main.go
- cd dist && zip function.zip main
sources:
- main.go
generates:
- dist/main
- dist/function.zip
silent: true
ignore_error: true
build-zip-s3:
desc: Builds lambda
cmds:
- clear && date
- GOOS=linux go build -o dist/main main.go
- cd dist && zip function.zip main
- cd dist && aws s3 cp function.zip s3://cdktoolkit-stagingbucket/functions/warmkalt.zip --quiet && echo "Upload complete"
sources:
- main.go
generates:
- dist/main
- dist/function.zip
silent: false
ignore_error: true
deploy2:
dir: dist
deps: [ build-zip ]
desc: Deploy from local disc
cmds:
- aws lambda update-function-code --function-name "warmkalt" --zip-file fileb://function.zip
deploy3:
dir: dist
deps: [ build-zip ]
desc: Deploy from s3
cmds:
- aws lambda update-function-code --function-name "warmkalt" --s3-bucket cdktoolkit-stagingbucket --s3-key functions/warmkalt.zip
deploy4:
deps: [ build-zip-s3]
desc: Deploy with go programm
cmds:
- tools/deploylf
