nodejs-lambda-layer-builder

Creates an AWS Lambda Layers structure that is optimized for: Lambda Layer directory structure, compiled library compatibility, and minimal file size.

Note: This script requires Docker and uses a container to mimic the Lambda environment. If you also want to use the publish.sh script, make sure you have the AWS CLI and jq installed as well

Features

Builds either a zip file or a raw directory strucutre (e.g. if you want to use frameworks like Serverless for packaging) containing Node dependencies and places the libraries into the proper directory structure for lambda layers
Ensures compiled libraries are compatible with Lambda environment by using the lambci/lambda Docker container that mimics the lambda runtime environment
Allows specifying AWS Lambda supported Node versions: 8.10, 10.x and 12.x
Automatically searches for package.json file in several locations:
- Same directory as script
- Parent directory or script (useful when used as submodule)
- Function sub-directory of the parent directory

Installation

This function can be cloned for standalone use, into a parent repo or added as a submodule.

Clone for standalone use or within a repo:

# If installing into an exisiting repo, navigate to repo dir
git clone --depth 1 https://github.com/tobilg/nodejs-lambda-layer-builder _build_layer

Alternatively, add as a submodule:

cd {repo root}
git submodule add https://github.com/tobilg/nodejs-lambda-layer-builder _build_layer
# Update submodule
git submodule update --init --recursive --remote

Usage

Building the layer

$ ./build.sh -h
AWS Lambda Layer Builder for Node libraries

Usage: build.sh [-l NODEJS_RUNTIME_VERSION] [-n NAME] [-r] [-h] [-v]
  -l NODEJS_RUNTIME_VERSION     : Node runtime version to use: 8.10, 10.x, 12.x (default 10.x)
  -n NAME                       : Name of the layer
  -r                            : Raw mode, don't zip layer contents
  -h                            : Help
  -v                            : Display build.sh version

Run the builder with the command ./build.sh or _build_layer/build.sh if installed in sub-dir
It uses the first package.json file found in these locations (in order):
- Same directory as script
- Parent directory of script (useful when used as submodule)
- Function sub-directory of the parent directory (useful when used as submodule)
Optionally specify the Node runtime Version
- -l NODEJS_RUNTIME_VERSION: Node runtime version to use: 8.10, 10.x, 12.x (default 10.x)
- -n NAME: Name of the layer

Publishing the layer

You can use the included publish.sh script to publish your newly built layer.

$  ./publish.sh -h
AWS Lambda Layer Publisher

Usage: publish.sh [-l NODEJS_RUNTIME_VERSION] [-n NAME] [-b BUCKET_NAME] [-c] [-h] [-v]
  -l NODEJS_RUNTIME_VERSION     : Node runtime version to use: 8.10, 10.x, 12.x (default 10.x)
  -n NAME                       : Name of the layer
  -b BUCKET_NAME                : Name of the S3 bucket to use for uploading the layer contents
  -c                            : Create S3 Bucket for layer upload
  -h                            : Help
  -v                            : Display publish.sh version

Run the publisher with the command ./publish.sh or _build_layer/publish.sh if installed in sub-dir
Optionally specify the following flags
- -l NODEJS_RUNTIME_VERSION: Node runtime version to use: 8.10, 10.x, 12.x (default 10.x)
- -n NAME: Name of the layer (should fit to what you used with build.sh)
- -b BUCKET_NAME: Name of the S3 bucket to use for uploading the layer contents
- -c: Flag for creating a S3 bucket for uploading the layer's contents (default name: layer-uploads-$AWS_ACCOUNT_ID)

How to create a Lambda layer for Node

For example, we would like to build a Node Lambda layer for the sharp image processing module. This module needs some OS-specific (meaning: Amazon Linux) libraries, which means you can't just zip your node_modules directory on a Mac or Windows machine.

First step is to create a package.json in the same folder you cloned or checked out this repo.

{
    "name": "sharp-layer",
    "description": "Dependencies for building the sharp layer",
    "version": "0.1.0",
    "license": "MIT",
    "dependencies": {
      "sharp": "0.25.2"
    }
}

Second step is to run the build.sh script with the proper parameters:

$ ./build.sh -n sharp

This will create a zip file named sharp_node10.x.zip.

Third step is to publish the newly created layer contents to AWS via the publish.sh script:

$ ./publish.sh -n sharp -c

This will create a new S3 bucket named layer-uploads-$AWS_ACCOUNT_ID, upload the zip file created in step two, and trigger the layer publishing. The output will be something like

Creating S3 bucket
Layer file sharp_node10.x.zip found
Uploading layer file to S3
Publishing layer
Publish successful
Layer ARN: arn:aws:lambda:us-east-1:$AWS_ACCOUNT_ID:layer:sharp-lambda-layer:1

where $AWS_ACCOUNT_ID is your AWS account id.

Custom cleaning logic

You can edit the _clean.sh file if you want to add custom cleaning logic for the build of the Lambda layer. The above part of the file must stay intact:

#!/usr/bin/env bash
# Change to working directory
cd $1
# ----- DON'T CHANGE THE ABOVE -----

# Cleaning statements
# ----- CHANGE HERE -----
rm test.xt

The _make.sh script will then execute the commands after the Python packages have been installed.

Uninstall

If installed as submodule and need to be removed:

# Remove the submodule entry from .git/config
git submodule deinit -f $submodulepath
# Remove the submodule directory from the superproject's .git/modules directory
rm -rf .git/modules/$submodulepath
# Remove the entry in .gitmodules and remove the submodule directory located at path/to/submodule
git rm -f $submodulepath
# remove entry in submodules file
git config -f .git/config --remove-section submodule.$submodulepath

tobilg / nodejs-lambda-layer-builder