pipeline-framework

There are 21 repositories under pipeline-framework topic.

google-ai-edge / mediapipe
Cross-platform, customizable ML solutions for live and streaming media.
android audio-processing c-plus-plus calculator computer-vision deep-learning framework graph-based graph-framework inference machine-learning mediapipe mobile-development perception pipeline-framework stream-processing video-processing
Language:C++ 31874
nextflow
nextflow-io / nextflow
A DSL for data-driven computational pipelines
aws bioinformatics cloud dataflow docker groovy hello hpc nextflow pipeline pipeline-framework reproducible-research reproducible-science sge singularity singularity-containers slurm workflow-engine
Language:Groovy 3202
yuanxiaosc / Entity-Relation-Extraction
Entity and Relation Extraction Based on TensorFlow and BERT. 基于TensorFlow和BERT的管道式实体及关系抽取，2019语言与智能技术竞赛信息抽取任务解决方案。Schema based Knowledge Extraction, SKE 2019
tensorflow entity-extraction relation-extraction pipeline-framework bert-model competition-code
Language:Python 1228
pipeless-ai / pipeless
An open-source computer vision framework to build and deploy apps in minutes
artificial-intelligence computer-vision multimedia multimedia-applications cloud deep-learning machine-learning object-detection video yolo ffmpeg gstreamer inference-server python vision-framework inference perception pipeline-framework stream-processing video-processing
Language:Rust 768
yzhao062 / combo
(AAAI' 20) A Python Toolbox for Machine Learning Model Combination
machine-learning data-mining data-science ensemble-learning python model-combination aggregation pipeline-framework machine-learning-pipelines
Language:Python 657
Neuraxle
Neuraxio / Neuraxle
The world's cleanest AutoML library ✨ - Do hyperparameter tuning with the right pipeline abstractions to write clean deep learning production pipelines. Let your pipeline steps have hyperparameter spaces. Design steps in your pipeline like components. Compatible with Scikit-Learn, TensorFlow, and most other libraries, frameworks and MLOps environments.
pipeline pipeline-framework machine-learning deep-learning framework python-library hyperparameter-optimization hyperparameter-tuning hyperparameter-search hyperparameters scikit-learn parallel neuraxle
Language:Python 614
CubicZebra / informatics
Framework of fast implementation data processing and operating pipelines
data-visualization machine-learning mathematical-statistics pipeline-framework scientific-computing neural-network transformer
Language:Python 585
minerva-ml / open-solution-home-credit
Open solution to the Home Credit Default Risk challenge :house_with_garden:
machine-learning deep-learning kaggle pipeline feature-engineering reproducible-experiments reproducibility pipeline-framework lightgbm xgboost neptune competition credit-scoring credit-risk open-source python python3 python35
Language:Python 457
pipefunc
pipefunc / pipefunc
Lightweight fast function pipeline (DAG) creation in pure Python for scientific (HPC) workflows 🕸️🧪
dag hpc parallel-computing pipeline-framework pipelines reproducible-research slurm workflow-engine
Language:Python 428
thomhurst / ModularPipelines
Write your pipelines in C# !
build build-system build-tool build-tools csharp pipeline pipeline-deploy pipeline-framework pipelines dotnet dotnet-core dotnetcore
Language:C# 408
MegEngine / MegFlow
Efficient ML solution for long-tailed demands.
deep-learning computer-vision megengine rust python3 megenginelite pipeline-framework pipeline
Language:Rust 407
neptune-ai / open-solution-mapping-challenge
Open solution to the Mapping Challenge :earth_americas:
data-science machine-learning deep-learning kaggle python satellite-imagery data-science-learning lightgbm unet unet-image-segmentation unet-pytorch neptune machine-learning-lab mapping-challenge crowdai competition pipeline pipeline-framework
Language:Python 388
gokart
m3dev / gokart
Gokart solves reproducibility, task dependencies, constraints of good code, and ease of use for Machine Learning Pipeline.
gokart luigi mlops machine-learning pipeline-framework
Language:Python 334
s8sg / faas-flow
Function Composition for OpenFaaS
openfaas openfaas-template faas function-chain serverless automation pipeline-framework faas-flow function-composition pipeline integration workflow-engine
Language:Go 261
analysiscenter / batchflow
BatchFlow helps you conveniently work with random or sequential batches of your data and define data processing and machine learning workflows even for datasets that do not fit into memory.
python3 data-science machine-learning python pipeline-framework pipeline workflow workflow-engine
Language:Python 202
FluentMediator
ivanpaulovich / FluentMediator
:twisted_rightwards_arrows: FluentMediator is an unobtrusive library that allows developers to build custom pipelines for Commands, Queries and Events.
fluent-design mediator-pattern ddd-cqrs message-bus event-sourcing event-driven chain-of-responsibility chain-methods pipelines dotnet-core craftmanship tdd csharp pipeline-framework fluent-interface event-handlers
Language:C# 195
datajoint-python
datajoint / datajoint-python
Relational data pipelines for the science lab
cloud-computing data-analysis data-pipelines databases datajoint mysql pipeline-framework python relational-algebra relational-databases relational-model s3 scientific-computing workflow-management
Language:Python 184
whitaker-io / machine
Machine is a workflow/pipeline library for processing data
golang golang-library golang-package workflow workflow-engine pipeline pipeline-framework codespaces golangci-lint github-actions mit-license stream-processing streaming-data
Language:Go 163
neptune-ai / open-solution-salt-identification
Open solution to the TGS Salt Identification Challenge
deep-learning machine-learning data-science python python3 image-processing image-segmentation pytorch pipeline pipeline-framework neptune steppy steppy-toolkit unet unet-image-segmentation
Language:Python 121
AustralianSynchrotron / lightflow
A lightweight, distributed workflow system
workflow workflow-engine workflow-automation pipeline-framework pipeline celery networkx python python3
Language:Python 114
pwwang / pipen
A pipeline framework for python
pipeline-framework pipeline jinja2 dot workflow-engine workflow python
Language:Python 108
Geoweaver
ESIPFed / Geoweaver
boost data pipeline's tangibility, enhance research productivity, reduce work anxiety
esip-lab ai earth-science workflow workflow-engine workflow-management workflow-tool pipeline pipeline-framework scientific-computing productivity tangibility
Language:JavaScript 91
maxim2266 / strit
Package strit introduces a new type of string iterator, along with a number of iterator constructors, wrappers and combinators.
fluent-interface string-processing iteration function-composition golang pipeline-framework
Language:Go 84
benfoster / Flo
Lightweight library for composing chain of responsibility pipelines
asp-net-core asp-net csharp pipeline-framework flow-based-programming
Language:C# 82
CyberReboot / vent
Vent is a light-weight platform built to automate network collection and analysis pipelines using a flexible set of popular open source tools and technologies. Vent is python-based, extensible, leverages docker containers, and provides both an API and CLI.
docker pipeline-framework prototyping pipelines containers cli automation processing deployment rabbitmq redis syslog elasticsearch gpu scheduling tui
Language:Python 78
marsggbo / torchline
Easy to use Pytorch
demo pipeline-framework pytorch pytorch-lightning torchline
Language:Python 70
bsc-wdc / compss
COMP Superscalar (COMPSs) is a framework which aims to ease the development and execution of applications for distributed infrastructures, such as Clusters, Grids and Clouds.
distributed-computing workflows java python c workflow-management-system pipeline-framework hpc docker singularity slurm
Language:Java 58
Matts966 / alphasql
AlphaSQL provides Integrated Type and Schema Check and Parallelization for SQL file set mainly for BigQuery
bigquery zetasql typechecker parallelization graphviz dot-language datawarehouse datawarehouseautomation docker workflow-automation workflow-tool pipeline-framework dependency-graph dependency-analysis elt cycle-detection
Language:C++ 56
Cambricon / CNStream
CNStream is a streaming framework for building Cambricon machine learning pipelines http://forum.cambricon.com https://gitee.com/SolutionSDK/CNStream
c-plus-plus computer-vision inference pipeline-framework graph-framework machine-learning cambricon cambricon-cnstream mlu
Language:C++ 53
houmo-ai / mediapipe_plus
The purpose of this project is to apply mediapipe to more AI chips.
mediapipe tensorrt blazeface xavier jetson facedetect onnx c-plus-plus machine-learning computer-vision deep-learning opencv opengl vision inference pipeline-framework stream stream-processing graph-framework video-processing
Language:C++ 52
minerva-ml / open-solution-googleai-object-detection
Open solution to the Google AI Object Detection Challenge :maple_leaf:
data-science machine-learning deep-learning deep-neural-networks pipeline pipeline-framework reproducible-research reproducible-experiments reproducibility object-detection google-ai-challange challenge detection-network retina-net mask-rcnn
Language:Python 45
cginternals / gloperate
C++ library for defining and controlling modern GPU rendering/processing operations.
c-plus-plus c-plus-plus-11 framework rendering-engine pipeline-framework script-injection opengl-framework
Language:C++ 43
minerva-ml / open-solution-value-prediction
Open solution to the Santander Value Prediction Challenge :tropical_fish:
machine-learning deep-learning data-science reproducibility reproducible-experiments open-source education training python lightgbm xgboost sklearn pipeline-framework ensemble-model neptune santander competition banking-applications
Language:Python 39
MERlin
emanuega / MERlin
MERlin is an extensible analysis pipeline applied to decoding MERFISH data
pipeline-framework modular transcriptomics merfish cloud-computing image-processing segmentation
Language:Python 35
CMU-INF-DIVA / pyturbo
Pyturbo package: A pipeline system for efficient concurrent execution
pipeline-framework parallel
Language:Python 28
virelay / corelay
CoRelAy is a tool to compose small-scale (single-machine) analysis pipelines.
xai interpretability python machine-learning spectral-clustering pipeline-framework
Language:Python 28