stoney95 / pypely

From local functions to cloud deployed pipelines

Home Page:https://stoney95.github.io/pypely/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

pypely Twitter

PyPI GitHub release PyPI download month PyPI download week Lint Code Base GitHub stars GitHub followers PRs Welcome codecov

From local functions to cloud deployed pipelines - build pipelines in a functional manner. This package simplifies and streamlines the development of pipelines. You can start locally and deploy the pipelines later using your favorite framework. Further benefits are:

  • You don't lock your pipelines into a framework. You can even convert your pypely code to the framework of your desire to develop your pipelines further in that framework.
  • The dependencies for each step are detected automatically. So, you don't need to manage the dependencies.
  • The compatability of steps is checked during buildtime. Errors are catch as early as possible.
  • You keep your pipelines easily testable.

Installation

pip install pypely

Usage

Use pypely to chain functions and structure your data processing code in a readable way.

use_pypely = pipeline(
    open_favourite_ide,
    create_new_conda_environment,
    activate_environment,
    install_pypely,
    have_fun_building_pipelines 
)

use_pypely() # -> 🥳

If you want to learn more check out the following links.

Links

Contributing

If you want to contribute:

  1. Woohoo! 🥳
  2. Please check out the contribution guide.
  3. See the issues to find a contribution possibility or create one to tell your plan and start a discussion.

About

From local functions to cloud deployed pipelines

https://stoney95.github.io/pypely/

License:GNU General Public License v3.0


Languages

Language:Python 94.8%Language:HTML 5.2%