Search
  • Tim Burns

Automating Serverless Deployments with AWS CodeBuild


Photo by Shawn Lee on Unsplash


I remember when DevOps was a new idea and a colleague said, "DevOps separates out the real developers." I agreed with him at the time that most developers do not have the skills to practice DevOps. This was over ten years ago, and development teams have gotten more sophisticated, but I also think that teams need more education on how to practice DevOps and SecDevOps.


I'm starting an article series on SecDevOps (which is like DevOps, but with security). I'll be touching on AWS CodeBuild, AWS CodePipeline, and finally integrating into applications like GitHub and OWASP.


What is Cloud Formation?

CloudFormation is a declarative language for specifying AWS services and you can configure any AWS application or resource using CloudFormation. From the point of Operations as code, CloudFormation is the gospel.


Getting Started with AWS CodeBuild for Serverless

AWS CodeBuild is a fully managed continuous integration service for compiling, deploying, and testing code as part of a development pipeline. You can use CloudFormation to create the build pipeline using declarative YAML code.


AWS documents the format and I provide the example to jump-start the impatient. The service defined creates a build environment that will compile and deploy the source code you have provided in the file s3://my-code-build/MyCode.zip.


In my next installment, I will describe how to replace the MyCode.zip bundle with a hook to GitHub, but I want to start simply because the technology is complex.


The MyCode.zip contains an API Gateway/Lambda serverless stack. It takes three files: buildspec.yml, src/lambda/mycode.py, and src/lambda/template.yml and zips them up. This is a toy and would otherwise be the contents of a pull request, but again: simplicity.


rm -f MyCode.zip
zip MyCode.zip buildspec.yml src/lambda/mycode.py src/lambda/template.yml
aws s3 sync . s3://my-code-build --exclude="*" --include "MyCode.zip"

So for cut and paste's sake, here are the files.


The buildspec.yml contains an analogous command sequence to "make" or "npm" and CodeBuild will execute the commands in this file to build the serverless stack.

# Buildspec Reference Doc: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html

version: 0.2

phases:
  install:
    commands:
      - echo "[+] Scanning for security vulnerability within dependencies"
      # Setup dependency checking
      # See: https://aws.amazon.com/blogs/apn/how-to-bake-open-source-security-into-your-aws-codebuild-pipeline/
  pre_build:
    commands:
      - echo "[+] Setting up any variables or dependencies..."

  build:
    commands:
      - echo "[+] Installing dependencies...."
      - echo "Running Cloud Formation scripts on `date` in `pwd`"
      - aws cloudformation package --template-file src/lambda/template.yml --s3-bucket my-code-deploy --output-template-file packaged-template.yaml
      - aws cloudformation deploy --template-file packaged-template.yaml --stack-name MyCode --capabilities CAPABILITY_NAMED_IAM
  post_build:
    commands:
      - echo "Build completed on `date`"

artifacts:
  files:
    - packaged-template.yaml
  discard-paths: yes

The src/lambda/mycode.py is a serverless Lambda function that executes whenever the API event triggers it.

import json


def get_status(event, context):
    result = {"hello": "Mr Tim"}
    return {
        "statusCode": 200,
        "body": json.dumps(result)
    }

The src/lambda/template.yml is a cloud formation template that builds a service API Gateway stack so that we can expose hello world as a web service https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/status/.


AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
    Monitoring function for the Owlmtn and Azrius Pipeline

Globals:
    Function:
        Timeout: 5

Resources:
    MyCodeFunction:
        Type: AWS::Serverless::Function
        Properties:
            CodeUri: .
            Handler: MyCode.get_status
            Runtime: python3.7
            Events:
                Status:
                    Type: Api
                    Properties:
                        Path: /status
                        Method: get

Outputs:
    MyCodeApi:
      Description: "API Gateway endpoint URL for monitoring function"
      Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/status/"

    MyCodeFunction:
      Description: "MyCode Lambda Function ARN"
      Value: !GetAtt MyCodeFunction.Arn

    MyCodeFunctionIamRole:
      Description: "Implicit IAM Role created for MyCode"
      Value: !GetAtt MyCodeFunctionRole.Arn
                                                                          

After building the template, examine the Outputs section in CloudFormation to understand what is happening.


The Buckets used in the CodeBuild Pipeline

The CodeBuild Template will take two S3 buckets. One the build bucket of s3://my-code-build and the deploy bucket s3://my-code-deploy. In a separate Cloud Formation template, I create these buckets as private. Here you can specify the buckets as parameters and reuse this CloudFormation template "as-is."


AWSTemplateFormatVersion: "2010-09-09"

Description: >
  Code Pipeline YAML to build S3 Buckets.
  From https://s3.amazonaws.com/cloudformation-examples/user-guide/continuous-deployment/basic-pipeline.yml

Parameters:
  S3DeployBucket:
    Description: The name of the S3 bucket that contains the source artifact, which must be in the same region as this stack
    Type: String

  S3BuildBucket:
    Description: The name of the S3 bucket that contains the source to checkout for build
    Type: String

Resources:
  ArtifactStoreBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref S3DeployBucket
      VersioningConfiguration:
        Status: Enabled
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        IgnorePublicAcls: true
        BlockPublicPolicy: true
        RestrictPublicBuckets: true

  ArtifactBuildBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Ref S3BuildBucket
      VersioningConfiguration:
        Status: Enabled
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        IgnorePublicAcls: true
        BlockPublicPolicy: true
        RestrictPublicBuckets: true

Outputs:
  ArtifactStoreBucket:
    Description: The bucket containing the deployment content
    Value: !Ref ArtifactStoreBucket

  ArtifactBuildBucket:
    Description: The bucket containing the build content
    Value: !Ref ArtifactStoreBucket

The CodeBuild Cloud Formation Template

Putting it all together, finally, we have the cloud formation template and I will highlight the import sections.


The definition of the CodeBuild project.

MyCodeBuildProject:
  Type: AWS::CodeBuild::Project
  Properties:
    Artifacts:
      Location: "my-code-deploy"
      Name: MyCode
      NamespaceType: NONE
      Path: /hello_world
      Type: S3
    Description: Simple Hello World Lambda API
    Environment:
      ComputeType: BUILD_GENERAL1_SMALL
      Image: aws/codebuild/amazonlinux2-x86_64-standard:3.0
      Type: LINUX_CONTAINER

    Name: my-code-build-project

    ServiceRole: !Ref 'CodeBuildRole'
    Source:
      BuildSpec: "buildspec.yml"
      Location: "my-code-build/MyCode.zip"
      Type: S3

    Tags:
      - Key: name
        Value: "my-code"

The role needed of the CodeBuild to build with CloudFormation.

CodeBuildRole:
  Type: AWS::IAM::Role
  Description: Role for Standard Code Build
  Properties:
    RoleName: !Join
      - '-'
      - - !Ref 'AWS::StackName'
        - CodeBuild
    AssumeRolePolicyDocument:
      Statement:
      - Action: sts:AssumeRole
        Effect: Allow
        Principal:
          Service: codebuild.amazonaws.com
    Policies:
      - PolicyName: CodeBuildReadOnlyPolicy-Monitoring
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Action:
                - 's3:ListBucket'
                - 's3:GetBucketAcl'
                - 's3:GetBucketLocation'
              Effect: Allow
              Resource:
                - 'arn:aws:s3:::my-code-build'
            - Action:
                - 's3:GetObject'
                - 's3:GetObjectVersion'
              Effect: Allow
              Resource:
                - 'arn:aws:s3:::my-code-build/*'
      - PolicyName: CodeBuildBasePolicy-Monitoring
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Action:
                - 'logs:CreateLogGroup'
                - 'logs:CreateLogStream'
                - 'logs:PutLogEvents'
              Effect: Allow
              Resource: '*'
            - Action:
                - "s3:PutObject"
                - "s3:GetObject"
                - "s3:GetObjectVersion"
                - "s3:GetBucketAcl"
                - "s3:GetBucketLocation"
              Effect: Allow
              Resource:
                - 'arn:aws:s3:::my-code-deploy'
                - 'arn:aws:s3:::my-code-deploy/*'
            - Action:
                - "cloudformation:*"
              Effect: Allow
              Resource: '*'
            - Action:
                - 'iam:GetRole'
                - 'lambda:*'
              Effect: Allow
              Resource: '*'


Putting all of these snippets together in Cloud Formation you will run the build and it will create a project that can be used to automate your code build process.


aws cloudformation deploy \
   --template-file src/delivery/packaged-codePipeline.yaml \
    --stack-name MyCodePipeline \
    --parameter-overrides PipelineName="my-code-pipeline" \
       S3BuildBucket="my-code-build" S3DeployBucket="my-code-deploy" \
    --capabilities CAPABILITY_NAMED_IAM

Conclusion

Integrate automation into everything you do - whether it is building code deployment pipelines, building web stacks, or testing. It won't be easy at first and I recommend you use the console to get a feel for what is created; however, don't rely on the console and automate everything. I provide these snippets not for you to copy blindly, but rather to understand and build your own. I hope they help.

15 views0 comments

Recent Posts

See All