DominicBurkart / leoronic

OTP clusters exposed to Python

Leoronic

Simple distributed computing.

note: leoronic is a prototype to understand grid computing.

Leoronic is a distributed computing library. Acting as a thin (~1500 lines) layer over OTP, it provides the same interface as Python's multiprocessing library and requires minimal setup. Leoronic automatically handles the loss or addition of new nodes to your cluster, allowing you to increase throughput or handle hardware failure on the fly, without stopping your applicaton or modifying your code.

System requirements

• Erlang/OTP >= 20
• bash
• perl

Current Dev Goals

Completeness

• fix broken codecov analysis.
• Terminate jobs that exceed their allotted system resources
• Handle kwargs in python
• Create integration tests for task requirement setting
• Create unit tests for task requirement setting in python
• add tests for ability to correctly throw custom python errors

New Features

• fully support linux.
• allow commands to be run outside of docker.
• provide support for Rust and R.
• allow commands to be passed to a waiting server program, which must be defined by the leoronic API for that language, and which must have a consistent (as of yet undefined) interface.
• package leoronic as a standalone binary (without docker).
• provide a GUI for leoronic.

Dev Installation

To set up your dev environment, make sure that you have Erlang (available in every major system package manager) installed and updated. Then, download the leoronic repo and install the git hooks as such:

git clone https://github.com/DominicBurkart/leoronic.git &&
cd leoronic &&
bash dev_install_contributer.sh

Finally, you'll also need to install shellcheck.

Happy coding!

Versioning

The version for the application is declared in ebin/Leoronic.app (look for vsn).