This Docker application was created by factoring out many reusable code artifacts from my various projects over a number of years. Since this work was not a part of a group effort, the test coverage is predictably abysmal 🤨 and Python documentation notably absent 😑. This package takes a submodule dependency on another one of my common packages. While this application is almost entirely boilerplate, it can run as a stand-alone application and serves the basis for any well-behaved Python application. The design is opinionated with the use of ZeroMQ but this is not a strict requirement. This project has a required dependency on 1Password for both build and run time (explained later). If this is unacceptable, you'll need to fork this project or send a pull request for a substitute like Bitwarden or equivalent.
Enough talk! What do I get?
- An Alpine Docker application that is specifically designed to act as a Docker base image for derived applications, or can be forked to run as is. This includes a variety of boilerplate entrypoint scripts that can be trivially overridden.
- Powerful threading and inter-thread data handling functions with significant resilience to unchecked thread death.
- Sample Healthchecks cron job with built-in container setup (you'd think that this would be simple and well documented).
- Pre-configured process control using supervisor.
- Automatic syslog configuration to log to the Docker host rsyslog.
- Support for AWS-CLI if appropriate AWS environment variables are present, like
AWS_DEFAULT_REGION. - Python dependency management using Poetry.
Here is a breakdown of some of the sample application features and structure:
- app.init.py: Sets a global for
APP_NAMEandWORK_DIRwhich currently assumes the location/opt/app. - app.main.py: The naming of this entrypoint is intentional so that derived application containers can easily override this and existing application bootstrapping logic just worksTM. After basic imports, a 1Password connect SDK CredsConfig struct is instantiated which tells pylib which credentials to pull from the 1Password connect server. Next, a variety of pylib imports are done to bring the needed functionality into the application. A ZeroMQ URL is defined called
URL_WORKER_APPfor inter-thread communication using their "lockless programming" paradigm. DataReader demonstrates the use of theAppThreadandCloseablefunctions which abstract away thread instantiation, thread death tracking and bring the context manager to gracefully handle errors and shutdown. A tenet of ZeroMQ is to not share sockets between threads and is managed for you with this implementation. A few lines of code bring a lot of powerful resilience features.DataReaderpretends to fetch some useful data and then uses its ZeroMQPUSHsocket to forward any to whatever has a sink URL toURL_WORKER_APP. EventProcessor illustrates the consumer which is analogous to the main application loop. The main entrypoint instantiates these classes, echoes some environment variable keys to the log that are visible to the application which is useful for debugging environment bootstrap. All threads wait on a Pythonthreading.Eventobject for quick shutdown should the signal arrive. When set, the main application and all non-daemon threads have a chance to complete, aided by a helper. Another helper routine zmq_term helps with getting ZeroMQ shutdown done. By design ALL ZeroMQ sockets must be closed gracefully in order for the application to exit. Any non-trivial applications need active management of sockets to avoid spending hours working out why the application shutdown is blocked. This is one of a few powerful features of this application and package.
Technologies that help make this package useful:
Also:
Here is some detail about the intended use of this package.
Beyond the Python dependencies defined in the Poetry configuration, the package init carries hardcoded dependencies on Sentry and 1Password in order to function. Unless you want these and are effectively extending my base project, you're likely better off forking this package and cutting out what you do not need.
Install these tools and make sure that they are on the environment $PATH.
-
taskfor project build orchestration: https://taskfile.dev/installation/#install-script -
dockeranddocker-composefor container builds and execution: https://docs.docker.com/engine/install/ -
mvnMaven for Java build orchestration: https://maven.apache.org/download.cgi -
poetryfor Python dependency management: https://python-poetry.org/docs/#installation -
javaandjavacfor Java build and runtime: https://aws.amazon.com/corretto/ -
pythonispython3for Python runtime: https://www.python.org/downloads/
- 🛑 This project uses 1Password Secrets Automation to store both application key-value pairs as well as runtime secrets. It is assumed that the connect server containers are already running on your environment. If you do not want to use this, then you'll need to fork this package and make the changes as appropriate. It's actually very easy to set up, but note that 1Password is a paid product with a free-tier for secrets automation. Here is an example of how this looks for my application and the generation of the docker-compose.yml relies on this step. Your secrets automation vault must contain an entry called
ENV.base_appwith these keys:
DEVICE_NAME: For naming the container. This project usesbase-app.APP_NAME: Used for referencing the application's actual name for the logger. This project usesbase_app.OP_CONNECT_HOST,OP_CONNECT_TOKEN,OP_CONNECT_VAULT: Used to specify the URL of the 1Password connect server with associated client token and Vault ID. See 1Password for more.HC_PING_URL: Healthchecks URL of this application's current health check status.
With these configured, you are now able to build the application.
In addition to this, additional runtime configuration is used by the application, and also need to be contained within the secrets vault. With these configured, you are now able to run the application.
- Clone the repo
git clone https://github.com/tailucas/base-app.git
- Verify that the git submodule is present.
git submodule init git submodule update
- Make the Docker runtime user and set directory permissions. ✋ Be sure to first review the Makefile contents for assumptions around user IDs for Docker.
make user
- Now generate the docker-compose.yml:
make setup
- And generate the Docker image:
make build
- If successful and the local environment is running the 1Password connect containers, run the application. For foreground:
For background:
make run
make rund
I have various projects that extend this Docker application. This Base Project serves as my Docker base image from which other projects are derived. While I may briefly run it locally in order to get basic functions, it it usually built, tagged and pushed to Docker so that the other applications can extend the functionality as needed.
Distributed under the MIT License. See LICENSE for more information.
