luigi
There are 4 repositories under luigi topic.
spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
Language:Python 17361pharmbio / sciluigi
A light-weight wrapper library around Spotify's Luigi workflow library to make writing scientific workflows more fluent, flexible and modular
Language:Python 332- gokart
m3dev / gokart
Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.
Language:Python 299 - Language:Python 248
unnati-xyz / scalable-data-science-platform
Content for architecting a data science platform for products using Luigi, Spark & Flask.
Language:Jupyter Notebook 162ClarityNLP / ClarityNLP
An NLP framework for clinical phenotyping. Docker | Python | Solr | OMOP. http://claritynlp.readthedocs.io/en/latest/
Language:Jupyter Notebook 105groupon / luigi-warehouse
A luigi powered analytics / warehouse stack
Language:Python 87spiside / docker-luigi
A project to help develop Luigi pipelines using Docker ✳️
Language:Shell 78- Language:Python 65
axiom-data-science / docker-luigi
🐍 🐳 Luigi in Docker - alpine and ubuntu images available
Language:Dockerfile 50shirosaidev / saisoku
Saisoku is a Python module that helps you build complex pipelines of batch file/directory transfer/sync jobs.
Language:Python 44maxhollmann / voxceleb-luigi
Luigi pipeline to download VoxCeleb(2) audio from YouTube and extract speaker segments
Language:Python 41Oslandia / osm-data-classification
Migrated to: https://gitlab.com/Oslandia/osm-data-classification
Language:Python 24- Language:Python 21
cereniyim / Wine-Rating-Predictor-ML-Model
Automated ML pipeline with Python, Docker, Luigi, SciKit-Learn and Pandas to predict wine quality ratings
Language:Jupyter Notebook 17- Language:Python 15
- Language:Python 13
1bk / simple-airports-analysis
A simple project to analyse Malaysian airports - an opportunity to play with tools like Luigi, Docker, and Metabase as part of an end-to-end analytics framework.
Language:Python 11adilkhash / luigi-course-materials
Материалы для курса Введение в Data Engineering: дата пайплайны
Language:Python 11havrikov / covering-input-structure
Replication package for the paper "Systematically Covering Input Structure" published at ASE 2019
Language:Python 9Maseya / SMB-Utility
An SMB editor originally created by M.K.S
Language:C 9papaizaa / etl-pipeline
ETL Pipeline using Luigi
Language:Python 9optimizely / kubeluigi
Run Luigi tasks as Jobs on K8s - An updated Kubernetes contrib
Language:Python 8SAP-samples / kyma-mtx-faq-management
The FAQ management service allows customers to manage a list of frequently asked questions through an SAP Fiori interface. For this, it leverages multiple SAP technologies such as the Kyma runtime, SAP Cloud Application Programming Model (CAP), SAP Application Router, Luigi, Fundamental Vue, and SAP Fiori elements.
Language:Go 8slub / efre-lod-elasticsearch-tools
collection of python scripts for a toolchain to feed an ElasticSearch-Index with bibliographic metadata and harvest RDF out of it
Language:Python 8- Language:Python 7
- Language:Python 7
- Language:C 7
Learn how to build a Luigi micro-frontend application with React and UI5. It demonstrates the usage of the Luigi open source project.
Language:JavaScript 7- Language:JavaScript 6
- Language:Python 6
andreizolotarev / MLOps
This repository contains examples of using various libraries/tools for MLOps.
Language:Python 5- Language:C++ 5
- Language:Python 5
sergii1989 / FastMLFramework
FastML Framework is a python library that allows to build effective Machine Learning solutions using luigi pipelines.
Language:Python 5binste / pipecutter
pipecutter provides a few tools for luigi such that it works better with data science libraries and environments such as pandas, scikit-learn, and Jupyter notebooks.
Language:Python 4
