gbourniq / lambda-restapi

Quickly develop and deploy serverless REST APIs with FastAPI, AWS Lambda, and AWS API Gateway. This includes a CI/CD pipeline with Travis and the AWS SAM Framework.

Overview

This project contains a base files and make commands to quickly develop and deploy serverless RESTful APIs running on FastAPI and deployed on AWS Lambda. An AWS API Gateway is used to make the function reachable via HTTP/S and for additional security at the API Gateway level.

Infrastructure, application code and package dependencies are managed by the AWS Serverless Application Framework, and a CI/CD build configuration pipeline is defined in the .travis.yaml file.

Contents

• Prerequisites
• Virtual environment and git-hooks setup
• Build python dependencies as a Lambda layer
• Local development and testing
• API Gateway and Lambda deployment to AWS
• Secure the API Gateway endpoint
• Production endpoint with a Custom Domain Name
• CI/CD

Prerequisites

• Configure the AWS CLI with your AWS credentials.
• Install Docker
• Install Miniconda
• Install Poetry
• Install Make
• Create an AWS S3 Bucket to store application code and dependencies artifacts

Virtual environment and git-hooks setup

To make things easy, you can create the conda environment and install the dependencies by running:

make env
conda activate lambda-restapi

Note: the environment can then be updated using the make env-udpate command when poetry packages are modified.

A pre-commit package is used to manage git-hooks. The hooks are defined in .pre-commit-config.yaml, and will automatically format the code with autoflake, isort and black, and run code analysis and testing with pylint and pytest, on each commit. To set them up, run:

make pre-commit

Make sure to run source .env to avoid import errors

Build python dependencies as a Lambda layer

Application code dependencies must be located at ./bin/python, which will be used by the SAM Framework to build a Lambda layer. To package dependencies into ./bin/python, run:

make build

Lambda application code dependencies are managed by Lambda layers, which helps reduce the size of uploaded archives and make it faster to deploy your application code.

Local development and testing

1. Set main environment variables in the following files:

• Makefile: for the S3 bucket name, and application name
• sam-application/.env: for any FastAPI and Lambda configurations

2. Run the FastAPI server to test endpoints locally

make start-fastapi-server

3. Run tests against the FastAPI Test Client

make test

To view the unit-tests coverage report, run make open-cov-report

4. Run tests against the mock API Gateway, which forwards HTTP requests to the Lambda function Start the local (mock) API Gateway with

make start-api # To run

Note the above command will create a local HTTP server hosting all of your functions. When accessed (via browser, cli etc), it will launch a Docker container locally to invoke the Lambda function

In a separate terminal, run tests against the mock HTTP server with

make test-local-api

API Gateway and Lambda deployment to AWS

1. Upload application code and dependencies artifacts to AWS S3

make package

2. Deploy the serverless infrastructure (Lambda and API Gateway) and application code + dependencies

make deploy

Note the above 2 commands can be run subsequently with a convenient make update command. If the package dependencies are updated, please make sure to run make build first.

3. Utilities commands:

• make get-dev-endpoint: Retrieve the API Gateway Endpoint URL & the API Key (if configured).
• make get-logs: Tail Lambda logs from CloudWatch Logs into the terminal.
• make stack-delete: To delete the deployed stack resources: API Gateway and Lambda function

Secure the API Gateway endpoint

The API Gateway integration allows for the Lambda function to be reachable via HTTP/S. To view the endpoint URL, run make get-dev-endpoint. This public endpoint is not secured by default, which means anyone can access it and potentially occur unwanted costs.

To secure the endpoint, the API Gateway dev deployment stage needs to be associated with an API Key and API Usage Plan, from within the API Gateway online portal. Then please take note of the API Key name and update the API_KEY_NAME variable in the Makefile.

You can ensure the API is now secured, by running

make check-api-is-secured

Production endpoint with a Custom Domain Name

1. Follow the steps in Setting up a regional custom domain name in API Gateway
2. In the API Gateway portal, the desired API version to new API deployment stage, eg. called prod
3. Navigate to the Custom Domain API Mappings section and map the prod stage to a desired path (eg. api) to be able to access the API via https://<custom-domain-name/api/ instead of the default API Gateway Endpoint https://<ApiGtwId>.execute-api.eu-west-2.amazonaws.com/prod/
4. In the API Gateway settings, disabled the default API Gateway Endpoint so that Lambda can only be reached by the production / custom domain name.

CI/CD

The .travis.yml build configuration file defines the following CI/CD pipeline:

• Continuous Integration

make env                          <-- Create conda environment and install dependencies
conda activate lambda-restapi     <-- Activate conda environment
make test                         <-- Run unit-tests against the built-in FastAPI Testing server

• Continuous Deployment

make build                        <-- Build Python dependencies as a Lambda layer
make start-api &                  <-- Start a local mock API Gateway server as a background process
make test-local-api               <-- Run unit-tests against the mock API Gateway server
make package                      <-- Upload application code and Lambda layer to S3
make deploy                       <-- Deploy/Update the Cloudformation stack: API Gateway and Lambda
make check-api-is-secured         <-- Ensure the API Gateway endpoint is secured with an API Key
make check-healthcheck-endpoint   <-- Ping the healthcheck endpoint with the API Key

Note that the following secret environment variables must be set in the Travis Repository settings, so that the Travis server can deploy and update AWS resources.

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
AWS_DEFAULT_REGION

Additional resource to your application

The application template sam-template.yaml uses AWS Serverless Application Model (AWS SAM) to define serverless application resources such as Lambda, API Gateway, DynamoDB, etc.

AWS SAM is an extension of AWS CloudFormation with a simpler syntax for configuring common serverless application resources such as functions, triggers, and APIs. For resources not included in the SAM specification, you can use standard AWS CloudFormation to extend the template.