Automating ACM Certificates with Serverless Framework

Automating ACM Certificate creation with the Serverless Framework

Encryption is the basis for secure communication in our modern world. For most web applications this boils down to using HTTPS to encrypt traffic between the client and server. HTTPS or the underlying protocols TLS/SSL rely on Public Key Infrastructure and Encryption to establish the Authenticity of the communication partner. Authenticity in this context means that the client can be sure he is communicating with Google if they access https://google.com and not somebody impersonating Google.

Authenticity is implemented via a certificate that testifies a specific public key belongs to an entity. It’s implemented using a cryptographic signature by a certificate authority. But let’s not get into too much detail here. The important point is this: if we want our clients to be able to communicate with us securely, we need a valid public certificaten (I’m ignoring self-signed certificates on purpose - those don’t count).

Let’s see how we can automate this with Serverless.

AWS provides a service for you to create and manage these certificates - the appropriately named AWS Certificate Manager (ACM). It offers several (advanced) features, but for our use case it’s important that we can get free certificates for our domains from ACM with the limitation being that those can only be used on AWS resources (such as ELB or CloudFront).

One of the great things about AWS is that everything is an API and as such you can automate everything. Infrastructure as Code is great as well because it leads to the Repo as the single source of truth. In AWS CloudFormation is the way to describe infrastructure as code. With ACM certificates being resources you can create in the console you’d expect to be able to create these with CloudFormation as well - and you’d be correct! Well, sort of a little - not really. There is the AWS::CertificateManager::Certificate resource, but it has a catch:

Important

When you use the AWS::CertificateManager::Certificate resource in an AWS CloudFormation stack, the stack will remain in the CREATE_IN_PROGRESS state. Further stack operations will be delayed until you validate the certificate request, either by acting upon the instructions in the validation email, or by adding a CNAME record to your DNS configuration. For more information, see Use Email to Validate Domain Ownership and Use DNS to Validate Domain Ownership.

tl;dr: The resource will block the stack until domain ownership is validated manually. Wait, what?!

Let’s take a step back. I lied to you earlier (not on purpose though, I just forgot we need to cover this)- we’ll have to talk about a little detail of public certificates. In order for a certificate authority to be able to issue a certificate for a domain to you it needs to make sure that you actually control the domain in question. There are many ways to do this, such as sending a contract signed with your blood to the CA. Just kidding, it’s more cruel than that. We need to mess with DNS (I exaggerate, DNS has a special place in my heart). AWS supports two ways to verify domain ownership, one being verification via E-Mail to a special address of that domain and the other (much better) way of setting a CNAME-record on the domain. The idea is: if you’re able to set DNS-Records for the domain you control this domain. Hopefully this makes sense.

You’d expect that CloudFormation would be able to do this DNS-based verification itself if the Domain is hosted in Route53, but you’d be wrong. If you’re issuing the certificate from ACM in the GUI/Console it allows you to set the verification records from there, but CloudFormation is not that advanced… yet.

Fortunately you can automate everything that vanilla CloudFormation can’t do with Custom Resources. Those are basically Lambda Functions that are called in response to stack events (CREATE, DELETE, UPDATE, …). Let’s do this!

That’s what I thought and then laziness kicked in and I decided that I’m probably not the first person to come upon this limitation. Lazy me was correct - the good folks at binx.io wrote a blog and created custom resources for this that they released under the Apache 2.0 license on Github. Great news, we don’t have to re:invent the wheel!

After boring you with lots of details and background information - let’s get to core topic: Adapting the binx.io custom resources to be used with the Serverless Framework.

I created a sample project for this on GitHub if you’re interested in using this yourselves.

Setting this up was a little annoying, because the code relies on the jsonschema library for validating the format of the input it gets from CloudFormation. This particular library doesn’t play that well with Lambda and the serverless-python-requirements plugin I basically always use to manage and package python libraries.

The serverless-python-requirements plugin for the serverless framework allows you among other things to set the slim: True parameter, which removes some of the directories from the deployment packages that aren’t necessary for the libs to function properly. This includes tests and distribution information. It has the benefit of making the deployment packages smaller.

custom:
  pythonRequirements:
    dockerizePip: non-linux
    slim: true
    layer: true

Unfortunately the jsonschema library doesn’t like that, as I had to find out the hard way. It uses some strange way to figure out it’s own version that doesn’t work with the slim parameter. This means your deployment package is going to be slightly larger than necessary (it took me quite a few hours to figure out that this was the problem as I couldn’t reproduce it on my local machine).

After configuring the serverless framework the rest is pretty straightforward:

1.) Configure the Lambda Function in the serverless.yml

  CFNCustomProvider:
    handler: src.binx_io_cfn_certificates.provider.handler
    timeout: 300
    layers:
      - {Ref: PythonRequirementsLambdaLayer}
    iamRoleStatementsName: CertProvider-${self:custom.Stage}
    iamRoleStatements:
      - Effect: Allow
        Action:
          - acm:RequestCertificate
          - acm:DescribeCertificate
          - acm:UpdateCertificateOptions
          - acm:DeleteCertificate
        Resource:
          - '*'
      - Effect: Allow
        Action:
          - lambda:InvokeFunction
        Resource:
          - !Join
            - ""
            - - "arn:aws:lambda:"
              - !Ref "AWS::Region"
              - ":"
              - !Ref "AWS::AccountId"
              - ":function:${self:service}-${self:custom.Stage}-CFNCustomProvider"

This function has permissions to create certificates in ACM and to invoke itself. The latter is used to wait for the certificate validation without running into a timeout. You can find the code behind the handler in the Github Repo I linked above.

2.) Create the custom resources:

This creates a certificate similarly to the vanilla CloudFormation resource with the exception that it won’t block until it’s validated.

    WildcardCertificateRequest:
      Type: Custom::Certificate
      DependsOn: CFNCustomProviderLambdaFunction
      Properties:
        DomainName: ${self:custom.Domain}
        ValidationMethod: DNS
        ServiceToken:
          !Join
          - ""
          - - "arn:aws:lambda:"
            - !Ref "AWS::Region"
            - ":"
            - !Ref "AWS::AccountId"
            - ":function:${self:service}-${self:custom.Stage}-CFNCustomProvider"

This resource extracts the information about the CNAME record we need to create to validate the certificate.

    CertificateDNSRecord:
      Type: Custom::CertificateDNSRecord
      DependsOn: CFNCustomProviderLambdaFunction
      Properties:
        CertificateArn: !Ref WildcardCertificateRequest
        DomainName: ${self:custom.Domain}
        ServiceToken:
          !Join
          - ""
          - - "arn:aws:lambda:"
            - !Ref "AWS::Region"
            - ":"
            - !Ref "AWS::AccountId"
            - ":function:${self:service}-${self:custom.Stage}-CFNCustomProvider"

This is the CNAME record to validate the certificate, it relies on the information provided by the resource above.

    DomainValidationRecord:
      Type: AWS::Route53::RecordSetGroup
      DependsOn: CFNCustomProviderLambdaFunction
      Properties:
        HostedZoneId: ${self:custom.HostedZoneId}
        RecordSets:
          - Name: !GetAtt CertificateDNSRecord.Name
            Type: !GetAtt CertificateDNSRecord.Type
            TTL: '60'
            Weight: 1
            SetIdentifier: ${self:custom.Domain}
            ResourceRecords:
              - !GetAtt CertificateDNSRecord.Value

Last but not least there is this resource which waits until ACM has validated the certificate.

    WildcardCertificate:
      Type: Custom::IssuedCertificate
      DependsOn: CFNCustomProviderLambdaFunction
      Properties:
        CertificateArn: !Ref WildcardCertificateRequest
        ServiceToken:
          !Join
          - ""
          - - "arn:aws:lambda:"
            - !Ref "AWS::Region"
            - ":"
            - !Ref "AWS::AccountId"
            - ":function:${self:service}-${self:custom.Stage}-CFNCustomProvider"

3.) ???

4.) Profit!

Now that you’ve set up the resources you can do a serverless deploy and should be able to see your certificate beeing validated automagically.

If there are any questions, feel free to reach out to me on Twitter: @maurice_brg

The title image is a still from Dr. Werner Vogels’ keynote: https://www.youtube.com/watch?v=nFKVzEAm-ts&feature=youtu.be&t=47m54s