pipeline

There are 67 repositories under pipeline topic.

jina
jina-ai / jina
🔮 Build multimodal AI services via cloud native technologies · Neural Search · Generative AI · Cloud Native
neural-search cloud-native deep-learning machine-learning framework vector-search-engine grpc kubernetes microservices semantic-search multimodal crossmodal creative-ai aiops mlops pipeline workflow airflow fastapi generative-ai
Language:Python 17837
vector
vectordotdev / vector
A high-performance observability data pipeline.
events forwarder logs metrics observability parser pipeline router rust stream-processing vector
Language:Rust 13037
argoproj / argo-cd
Declarative continuous deployment for Kubernetes.
argo argo-cd cd ci-cd cicd continuous-delivery continuous-deployment devops docker gitops hacktoberfest helm ksonnet kubernetes pipeline
Language:Go 12485
PrefectHQ / prefect
The easiest way to orchestrate and observe your data pipelines
python workflow data-engineering data-science workflow-engine prefect infrastructure ml-ops data-ops automation orchestration orion data observability pipeline
Language:Python 11539
airbytehq / airbyte
Data integration platform for ELT pipelines from APIs, databases & files to warehouses & lakes.
data pipeline data-analysis data-engineering java python etl airbyte change-data-capture data-collection data-ingestion data-integration elt bigquery redshift snowflake
Language:Python 10059
kedro
kedro-org / kedro
A Python framework for creating reproducible, maintainable and modular data science code.
experiment-tracking hacktoberfest kedro machine-learning mlops pipeline python
Language:Python 8196
great-expectations / great_expectations
Always know what to expect from your data.
pipeline-tests dataquality datacleaning datacleaner data-science data-profiling pipeline pipeline-testing cleandata dataunittest data-unit-tests eda exploratory-data-analysis exploratory-analysis exploratorydataanalysis data-quality data-engineering pipeline-debt data-profilers mlops
Language:Python 8137
tektoncd / pipeline
A cloud-native Pipeline resource.
tekton pipeline kubernetes cdf hacktoberfest
Language:Go 7743
prql
PRQL / prql
PRQL is a modern language for transforming data — a simple, powerful, pipelined SQL replacement
data pipeline sql
Language:Rust 6965
proposal-pipeline-operator
tc39 / proposal-pipeline-operator
A proposal for adding a useful pipe operator to JavaScript.
javascript operator pipeline proposal syntax tc39
Language:HTML 6924
brunch / brunch
:fork_and_knife: Web applications made easy. Since 2011.
javascript workflow build-automation pipeline brunch
Language:JavaScript 6839
nteract / papermill
📚 Parameterize, execute, and analyze notebooks
jupyter notebooks notebook-generator nteract publishing pipeline notebook python r julia scala
Language:Python 5168
gaia-pipeline / gaia
Build powerful pipelines in any programming language.
automation build continuous-delivery continuous-integration continuous-testing cplusplus deployment devops devops-tools distributed-systems gitops go java kubernetes pipeline python ruby
Language:Go 5019
projectdiscovery / httpx
httpx is a fast and multi-purpose HTTP toolkit that allows running multiple probes using the retryablehttp library.
http bugbounty ssl-certificate pipeline pentest-tool cybersecurity osint hacktoberfest cli lib
Language:Go 4613
jx
jenkins-x / jx
Jenkins X provides automated CI+CD for Kubernetes with Preview Environments on Pull Requests using Cloud Native pipelines from Tekton
kubernetes cicd continuous-integration continuous-delivery accelerator openshift gitops environments pipeline devops golang tekton hacktoberfest
Language:Go 4349
Production-Level-Deep-Learning
alirezadir / Production-Level-Deep-Learning
A guideline for building practical production-level deep learning systems to be deployed in real world applications.
ai artificial-intelligence deep-learning deployment kubeflow machine-learning pipeline practical-machine-learning production-system scalable-applications system-design tfx
3788
mage-ai / mage-ai
🧙 The modern replacement for Airflow. Build, run, and manage data pipelines for integrating and transforming data.
machine-learning artificial-intelligence data data-engineering data-science python elt etl pipelines data-pipelines orchestration data-integration sql spark dbt pipeline reverse-etl transformation
Language:Python 3677
kestra
kestra-io / kestra
Kestra is an infinitely scalable orchestration and scheduling platform, creating, running, scheduling, and monitoring millions of complex pipelines.
data data-engineering data-integration data-orchestration data-orchestrator data-pipeline data-quality elt etl low-code orchestration pipeline reverse-etl scheduler workflow workflow-engine
Language:Java 3240
kubeflow / pipelines
Machine Learning Pipelines for Kubeflow
kubeflow-pipelines mlops kubeflow machine-learning kubernetes pipeline data-science
Language:Python 3139
alibaba / pipcook
Machine learning platform for Web developers
js machine-learning tensorflow pipeline
Language:TypeScript 2388
EntilZha / PyFunctional
Python library for creating data pipelines with chain functional programming
data datascience functional-programming pipeline python
Language:Python 2145
mara / mara-pipelines
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
data data-integration etl pipeline postgresql python
Language:Python 1986
nextflow
nextflow-io / nextflow
A DSL for data-driven computational pipelines
bioinformatics workflow-engine pipeline pipeline-framework nextflow cloud groovy sge slurm aws docker singularity hpc singularity-containers reproducible-science reproducible-research dataflow hello
Language:Groovy 1983
towhee-io / towhee
Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
computer-vision convolutional-networks embedding-vectors embeddings feature-extraction feature-vector image-processing image-retrieval machine-learning milvus pipeline towhee transformer unstructured-data video-processing vision-transformer vit
Language:Python 1935
hu17889 / go_spider
[爬虫框架 (golang)] An awesome Go concurrent Crawler(spider) framework. The crawler is flexible and modular. It can be expanded to an Individualized crawler easily or you can use the default crawl components only.
spider crawler go schedule pipeline
Language:Go 1804
edyoda / data-science-complete-tutorial
For extensive instructor led learning
machine-learning scikit-learn linear-regression nearest-neighbors feature-selection decision-trees numpy pandas pipeline
Language:Jupyter Notebook 1667
GameDevMind
gonglei007 / GameDevMind
最全面的游戏开发技术图谱。帮助游戏开发者们在已知问题上节省时间，省出更多的精力投入到更有创造性的工作中去。 | The most comprehensive technical map of game development. Help game developers save time on known problems and save more energy for more creative work.
game-development unity3d 3d programming unreal-engine unrealengine game shader game-server framework mmorpg roadmap scrum pipeline game-framework cpp management devops gamedev unity
Language:Shell 1625
lastbackend
lastbackend / lastbackend
System for containerized apps management. From build to scaling.
cloud docker hosting continuous-delivery continuous-integration backend go golang containers lastbackend paas kubernetes service-discovery devops orchestration devtools pipeline container ci cd
Language:Go 1618
alan-turing-institute / MLJ.jl
A Julia machine learning framework
machine-learning julia pipelines tuning data-science tuning-parameters predictive-modeling classification regression statistics clustering stacking ensemble-learning pipeline
Language:Julia 1572
MLBox
AxeldeRomblay / MLBox
MLBox is a powerful Automated Machine Learning python library.
machine-learning auto-ml kaggle deep-learning stacking pipeline optimization preprocessing encoding prediction distributed xgboost drift classification regression lightgbm keras automated-machine-learning automl data-science
Language:Python 1399
go-streams
reugn / go-streams
A lightweight stream processing library for Go
stream-processing pipeline etl kafka data-stream kafka-streams streams redis pulsar data-pipeline streaming-data stream-processor aerospike websocket nats-streaming streaming-api windowing throttling low-code workflow
Language:Go 1369
BloodAxe / pytorch-toolbelt
PyTorch extensions for fast R&D prototyping and Kaggle farming
pytorch kaggle image-classification image-segmentation deep-learning segmentation python image-processing machine-learning focal-loss jaccard-loss tta test-time-augmentation augmentation object-detection pipeline
Language:Python 1365
ropensci / drake
An R-focused pipeline toolkit for reproducibility and high-performance computing
data-science drake high-performance-computing makefile peer-reviewed pipeline r r-package reproducibility reproducible-research ropensci rstats workflow
Language:R 1321
alibaba / dawn
:sunrise: Dawn is a lightweight task management and build tool for front-end and nodejs.
build-tool build front-end dawn dawn-cli task pack pipeline middleware construction nodejs javascript
Language:JavaScript 1137
galaxyproject / galaxy
Data intensive science for everyone.
bioinformatics workflow genomics science sequencing ngs dna usegalaxy pipeline workflow-engine hacktoberfest
Language:Python 1077
firecow / gitlab-ci-local
Tired of pushing to test your .gitlab-ci.yml?
cd ci git gitlab gitlab-ci local pipeline push uncomitted untracked
Language:TypeScript 1067