Get Test Coverage with PyTest on Local Systems
Momentum is building behind pure serverless environments with API Gateway and Lambda, but the design patterns have not caught up. In particular, the serverless articles I've read only address automation by using CloudFormation to deploy the stack. It's not good enough.
Developers need the ability to write failing unit tests locally, fix those tests, and then deploy the code to integration. Running CloudFormation on every deploy is too much overhead. It will lead to poor design where the developer is tweaking something until it works, and then throwing it over the fence into the AWS ecosystem.
I've come up with a better way. Let's start with a simple problem: parsing a string with a regular expression. Say I want to write tests to parse out various portions of a file. Given the file name:
"api-gateway/tests/data/apple_health_tracking_201911231045steps.csv"
I want to see the following parsed out, and so I write a test.
self.assertEqual(result["directory"], "api-gateway/tests/data") self.assertEqual(result["company"], "apple") self.assertEqual(result["application"], "health_tracking") self.assertEqual(result["date"], "20191123") self.assertEqual(result["time"], "1045") self.assertEqual(result["activity"], "steps")
If I implemented a Lambda function to resolve the test, I would compile the code, test it, tweak it, test it, but would take a while. I could do this using curl functions, but there is a better way: create a Python library as a Lambda Layer, and then build the gateway on top of that layer.
Here is how to go about it.
Build out a Lambda layer on an EC2 instance
Deploy the zip file as a Lambda Layer
Compile your Python code to a Lambda Layer
Developers on Windows for the Linux environment used by AWS serverless is a challenge. To address this, create an EC2 build server. Use Ubuntu 18.02 and install the following on the server.
sudo apt update
sudo apt install python3-pip
sudo apt install awscli
sudo apt-get install zip unzip
Configure your credentials on the server with "aws configure".
The utility in https://github.com/timowlmtn/api-gateway will build on the server by issuing remote ssh calls through the "paramiko" library. For example
setup_commands = ( f"rm -rf {layer_name}_deploy", f"mkdir -p {layer_name}_deploy/python/lib/python3.6/site-packages/{layer_name}" )
Execute it with
stdin, stdout, stderr = ssh_client.exec_command(command)
Running the script in the Makefile target deploy-lambda will generate a zip file an S3 bucket in a format for Lambda Layers.
Create a Lambda Layer using the Boto3 Client
The Boto3 Client is the final step to creating the Lambda Layer in your servless system.
Connect to the AWS account and deploy the Lambda Layer
session = boto3.Session(profile_name=args.profile) lambda_client = session.client('lambda') deploy_lambda_aws(args, lambda_client)
Create the file with the API and save the output.
response = lambda_client.publish_layer_version( LayerName=args.layer_name, Description=args.layer_description, Content={ 'S3Bucket': args.s3_bucket, 'S3Key': f"{args.layer_name}.zip" }, CompatibleRuntimes=[ 'python3.6', 'python3.7', 'python3.8', ] )
Stay tuned for my next article where I wire all this up together with API Gateway for a functional web service.
ความคิดเห็น