IQTLabs / edgetech-s3-uploader

Builds upon the IQT Labs Edgetech-Core functionality to instantiate an MQTT client that uploaded files from a specified directory to a specified S3 bucket.

EdgeTech-S3-Uploader

This repo builds upon the IQT Labs EdgeTech-Core functionality to instantiate an MQTT client uploads all the files saved by the edgetech-filesaver. Functionality is also included to respond to a Command and Control module to trigger the upload. All of this functionality is wrapped in a Docker container for cross-platform compatibility.

Make Contribution · Report Bug · Request Feature

Built With

Getting Started

To run this repo, simply run:

docker-compose up

The s3-uploader is containerized and dependencies are managed using poetry.

Prerequisites

Running this repo requires that you have Docker installed.

Usage

Spinning up this system requires an MQTT server and this container to be included in your docker-compose.yml. You can find an example of this workflow in this repository's docker-compose.yml. Additionally, some editing of relevant environment variables will be required based upon your system's configuration of topics to subscribe to and MQTT configuration. Examples of these environment variables can be found in this repository's .env file.

Copying the s3-uploader docker-compose statements into a master docker-compose.yml and .env files with your entire system of containers is the preferred workflow. Find an application architecture diagram example of how the usage of this module was envisioned below.

flowchart TD
    c2(C2) -- Command & Control Topic --> mqtt{MQTT}
    mqtt{MQTT} -- Subscribed to C2 Topic --> s3-uploader(S3 Uploader)
    

style mqtt fill:#0072bc,color:#ffffff
style s3-uploader fill:#F9D308,color:#ffffff
style c2 fill:#f05343,color:#ffffff

Roadmap

• TBA

See the open issues for a full list of proposed features (and known issues).

Contributing

1. Fork the Project
2. Create your Feature Branch (git checkout -b dev)
3. Commit your Changes (git commit -m 'adding some feature')
4. Run (and make sure they pass):

black --diff --check *.py

pylint --disable=all --enable=unused-import *.py

mypy --allow-untyped-decorators --ignore-missing-imports --no-warn-return-any --strict --allow-subclassing-any *.py

If you do not have them installed, you can install them with pip install "black<23" pylint==v3.0.0a3 mypy==v0.991.

5. Push to the Branch (git push origin dev)
6. Open a Pull Request

See CONTRIBUTING.md for more information.

License

Distributed under the Apache 2.0. See LICENSE.txt for more information.

Contact IQTLabs

• Twitter: @iqtlabs
• Email: [e-mail]

See our other projects: https://github.com/IQTLabs/

(back to top)