The Things Stack is an open-source LoRaWAN stack. Users can deploy the stack as Open Source (free) or Enterprise (requires a license) from images available on Docker Hub.
This repository facilitates packaging The Things Stack into an AWS IoT Greengrass V2 component named aws.greengrass.labs.TheThingsStackLoRaWAN. This component enables customer use cases that require a private LoRaWAN Network Server (LNS) at the edge. For Enterprise deployments, ingestion into AWS IoT Core can then be achieved by deploying The Things Stack AWS IoT default integration into your AWS account.
The Things Stack is also offered as an AWS Marketplace AMI. This Greengrass component is a logical equivalent, but pitching at the edge.
- Architecture
- Repository Contents
- Requirements and Prerequisites
- Getting Started
- The Things Stack Configuration Tips
- Development
An overview of the system architecture is presented below.
The aws.greengrass.labs.TheThingsStackLoRaWAN component is a thin wrapper around a conventional The Things Stack deployment. The Things Stack is composed of multiple Docker containers, using Docker Compose. Subject to configuration, there may be a Redis container and either a PostreSQL or CockroachDB container, in addition to The Things Stack container.
The Things Stack is delivered as Docker images on Docker Hub. This component downloads Docker images from Docker Hub with the help of the Docker application manager managed component.
The Things Stack configuration includes a considerable amount of sensitive information. Accordingly, the configuration is stored as a secret in AWS Secrets Manager. This component gets its configuration from Secrets Manager with the help of the Secret manager managed component.
For Enterprise deployments, ingestion into AWS IoT Core can be best achieved by deploying The Things Stack AWS IoT default integration into your AWS account.
The Things Stack offers an API and various integrations that can be used for remote ingestion or control, or can be used by other Greengrass components to achieve local data processing and actions. The Pub/Sub integration is a good option for achieving ingestion into AWS IoT Core from the Open Source edition of The Things Stack.
Item | Description |
---|---|
/artifacts | Greengrass V2 component artifacts that run on the Greengrass edge runtime. |
/cicd | CDK Typescript app for a CodePipeline CI/CD pipeline. |
/images | Images for README files. |
/libs | Python libraries shared by Python scripts. |
/recipes | Greengrass V2 component recipe templates. |
/robot | Robot Framework integration tests. |
/tests | Pytest unit tests |
/tts-config | The Things Stack configuration files. |
create_certs.sh | Creates self-signed TLS certificates for a given domain name or IP address. |
create_component_version.py | Creates a new version of The Things Stack LoRaWAN component in the Greengrass cloud service. |
create_config_minimal.py | Creates a minimal configuration needed to deploy the component for a given domain name or IP address. |
create_config_secret.py | Creates or updates The Things Stack configuration secret in Secrets Manager. |
deploy_component_version.py | Deploys a component version to the Greengrass core device target. |
quickstart.sh | Creates and deploys a component with default configuration for a given domain name or IP address. |
This component requires that the Greengrass device be running a Linux operating system. It supports all architectures supported by Greengrass itself.
The Greengrass edge runtime needs to be deployed to a suitable machine, virtual machine or EC2 instance. Please see The Things Stack stated prerequisites for guidance on the resources required. For a small number of devices and gateways, the processing and memory requirements are small.
The Greengrass machine or instance should not be running any process that uses any ports that may be needed by The Things Stack. The specific ports used depends on your particular configuration of The Things Stack. The configuration generated by quickstart.sh requires ports 443, 1700, 6379, 8881, 8882, 8883, 8884, 8885, 8887, 26256 and 26257 to be available.
The Things Stack is generally composed as an application with multiple Docker containers, using Docker Compose. Images are obtained from Docker Hub.
Therefore your core device must meet the requirements to run Docker containers using Docker Compose and Docker Hub.
Not all releases of these third-party container images support all Greengrass architectures. Care must be taken to select appropriate images and releases to suit your target architecture.
This component requires both python3 and pip3 to be installed on the core device.
This component downloads artifacts from an S3 bucket named greengrass-tts-lorawan-ACCOUNT-REGION. Therefore your Greengrass core device role must allow the s3:GetObject permission for this bucket. For more information: https://docs.aws.amazon.com/greengrass/v2/developerguide/device-service-role.html#device-service-role-access-s3-bucket
Additionally, this component downloads sensitive The Things Stack configuration from Secrets Manager. Therefore your Greengrass core device role must also allow the secretsmanager:GetSecretValue permission for the greengrass-tts-lorawan-ID secret.
Policy template to add to your device role (substituting correct values for ACCOUNT, REGION and ID):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::greengrass-tts-lorawan-ACCOUNT-REGION/*"
},
{
"Effect": "Allow",
"Action": [
"secretsmanager:GetSecretValue"
],
"Resource": "arn:aws:secretsmanager:REGION:ACCOUNT:secret:greengrass-tts-lorawan-ID"
}
]
}
It may be necessary to upgrade your AWS CLI if you wish to use any greengrassv2 commands, as these are relatively recent additions to the CLI.
If you wish to generate your own self-signed TLS certificates for your domain, or use quickstart.sh, then cfssl is required. Example installation on an Ubuntu or Debian machine:
sudo apt update
sudo apt install -y golang-cfssl
Or using Homebrew (Mac or Linux):
brew install cfssl
Alternatively consider installing from released binaries available in the cfssl GitHub repository. Windows installation executables are available.
Most of the scripts in this repository are Python scripts. They are Python 3 scripts and hence python3 and pip3 are required.
Package dependencies can be resolved as follows:
pip3 install -r requirements.txt
pip3 install -r robot/requirements.txt
Please consider to use a virtual environment.
The quickstart.sh and create_certs.sh scripts are Bash scripts. If using a Windows machine, you will need a Bash environment.
Here we define two ways to get started: Quickstart or Slowstart.
All scripts used should be compatible with Linux, Mac or Windows operating systems, provided a Bash environment is available.
The quickstart.sh bash script is supplied to help you get going fast. For experimentation in non-production settings.
Before running the script, users must:
- Deploy Greengrass V2 to an x86-compatible Linux machine, virtual machine or EC2 instance. The default The Things Stack configuration settings will pull amd64 images, hence the x86-compatibility requirement.
- Initialize the Greengrass core device to satisfy the the requirements to run Docker containers using Docker Compose and Docker Hub.
- Install cfssl on the developer machine.
The Quickstart script will:
- Install required Python packages.
- Create self-signed certificates for a given domain name or IP address.
- Create a minimal The Things Stack configuration for the given domain name or IP address.
- Upload the configuration to a configuration secret in Secrets Manager.
- Create a new component version in Greengrass cloud services, with artifacts in an S3 bucket.
- Prompt you to add permissions for the configuration secret and artifacts bucket to the Greengrass core device role.
- Deploy the new component version to the Greengrass core (creating The Things Stack admin user in the process).
- Run Robot Framework integration tests to confirm The Things Stack is running under Greengrass.
The script accepts 6 arguments:
- Domain name or IP address (this can be an EC2 default domain name).
- AWS region.
- The password of The Things Stack admin user that will be created.
- The email of The Things Stack admin user that will be created.
- Component version. (Semantic major.minor.patch.)
- Greengrass Core device name.
Example execution:
bash quickstart.sh example.com ap-southeast-1 mypassword user@example.com 1.0.0 GGTheThingsStackLoRaWAN
For any serious use of the component, Quickstart shall not be appropriate.
If not using Quickstart, you must perform the following steps:
- Deploy the Greengrass runtime to your machine, virtual machine or EC2 instance. This should be a Linux machine, but can be of any architecture.
- Configure the LoRaWAN stack as per https://www.thethingsindustries.com/docs/getting-started/installation/configuration/. Take care to select Docker images with the correct architecture.
- Configure certificates as per https://www.thethingsindustries.com/docs/getting-started/installation/certificates/.
- If BYO/generating custom TLS certificates (rather than using Lets Encrypt), then add them to the /tts-config directory.
- If using an Enterprise license file, add it to the /tts-config directory.
- Run create_config_secret.py to create the configuration secret in Secrets Manager.
- Run create_component_version.py to create a component version in Greengrass cloud service, and upload artifacts to S3.
- Add permissions for the configuration secret and artifacts bucket to the Greengrass core device role.
- Run deploy_component_version.py to deploy the new component version to your Greengrass device.
For iterative configuration changes, repeat steps 2, 6, 7 and 9.
Example of steps 6, 7 and 9:
python3 create_config_secret.py ap-southeast-1 mypassword user@example.com
python3 create_component_version.py 1.0.0 ap-southeast-1
python3 deploy_component_version.py 1.0.0 ap-southeast-1 MyCoreDeviceThingName
This example:
- Operates in AWS region ap-southeast-1.
- Creates and deploys component version 1.0.0 to Greengrass core device MyCoreDeviceThingName.
- Creates an admin user for The Things Stack with password mypassword and email user@example.com.
This repository offers a CodePipeline CI/CD pipeline as a CDK application. This can be optionally deployed to the same account as the Greengrass core.
This CI/CD pipeline automates steps 7 and 9. Following deployment, it performs automated smoke tests to ensure that The Things Stack has started correctly. With the pipeline deployed, users can make iterative configuration changes, update the configuration secret using create_config_secret.py, and then trigger the CI/CD pipeline to handle the rest.
This repository includes an automated test suite built on top of Robot Framework. This can be run on demand from the command-line but it is also included as part of the CI/CD pipeline.
Configuration of the LoRaWAN stack is mainly an exercise in editing tts-config/docker-compose.yaml and tts-config/config/stack/ttn-lw-stack-docker.yml. Please consult The Things Stack configuration documentation for details.
The tts-config/docker-compose.yaml and tts-config/config/stack/ttn-lw-stack-docker.yml files in this repository are The Things Stack Enterprise example configuration files, with the following divergence:
- All Enterprise configuration elements are commented out so that it defaults to Open Source edition.
- Acme Let's Encrypt TLS certificates are disabled and custom TLS certificates are enabled.
- All non-TLS ports and endpoints are disabled.
This is to facilitate minimal effort in deploying this component to a server that only has an IP address, a local network DNS name or an EC2 default domain name.
As compared to Open Source, an Enterprise license adds:
- Device Claim Server
- Multi-tenant support
- Storage Integration
- AWS IoT Default Integration support
- Support plan
Per The Things Stack recommendations, production deployments should reference specific Docker image tags (not just latest) and consideration should also be given to using external managed instances of the databases.
CockroachDB is not available for Arm architecture. You must use PostgreSQL instead.
The Things Stack images tagged latest on Docker Hub are not available for Arm archiecture. Only formal releases are.
The Pub/Sub integration can be used to achieve ingestion into AWS IoT Core. This is particularly useful for the Open Source edition of the stack since the The Things Stack AWS IoT default integration is only available for Enterprise deployments. In configuring the Pub/Sub integration, the Server URL should define the AWS IoT Core endpoint as follows:
mqtts://ENDPOINTID-ats.iot.ap-REGION.amazonaws.com:8883
Static analysis is performed using Pylint. Example execution:
pylint artifacts libs tests *.py
Unit tests are performed using pytest and moto.
Example execution:
pytest --cov=artifacts --cov=.
Producing an HTML coverage report into the htmlcov directory:
pytest --cov=artifacts --cov=. --cov-report=html
Producing a coverage report for just the on-device artifacts (100% coverage):
pytest --cov=artifacts