dag

There are 29 repositories under dag topic.

airflow
apache / airflow
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
airflow apache apache-airflow python scheduler workflow automation dag data-engineering data-integration data-orchestrator data-pipelines data-science elt etl machine-learning mlops orchestration workflow-engine workflow-orchestration
Language:Python 43118
marimo-team / marimo
A reactive notebook for Python — run reproducible experiments, query with SQL, execute as a script, deploy as an app, and version with git. Stored as pure Python. All in a modern, AI-native editor.
notebooks python data-science machine-learning artificial-intelligence data-visualization developer-tools reactive pipeline web-app dag dataflow sql
Language:Python 16960
argoproj / argo-workflows
Workflow Engine for Kubernetes
workflow kubernetes argo dag knative airflow machine-learning argo-workflows workflow-engine hacktoberfest cloud-native cncf k8s gitops mlops batch-processing data-engineering pipelines
Language:Go 16166
dagger / dagger
An open-source runtime for composable workflows. Great for AI agents and CI/CD.
agents ai caching ci-cd containers continuous-deployment continuous-integration dag dagger devops docker graphql workflows
Language:Go 14942
sogou / workflow
C++ Parallel Computing and Asynchronous Networking Framework
consul dag http kafka mysql redis sogou tasking
Language:C++ 14159
antvis / X6
🚀 JavaScript diagramming library that uses SVG and HTML for rendering.
antv dag diagram diagramming erd erdiagram flowchart graph graph-editor javascript javascript-diagramming-library svg typescript uml-diagram
Language:TypeScript 6272
hatchet-dev / hatchet
🪓 Run Background Tasks at Scale
concurrency dag distributed distributed-systems durable-execution event-driven fastapi golang nodejs python queue typescript workflow-engine
Language:Go 6185
alibaba / butterfly
🦋Butterfly，A JavaScript/React/Vue2 Diagramming library which concentrate on flow layout field. (基于JavaScript/React/Vue2的流程图组件)
bpmn butterfly connectivity dag diagram diagramming edge flow flow-dag flow-layout flowchart mindmap react svg tree treemap-diagram vue
Language:JavaScript 4631
orchest / orchest
Build data pipelines, the easy way 🛠️
data-science machine-learning pipelines ide jupyter cloud self-hosted jupyterlab notebooks docker python data-pipelines orchest deployment kubernetes airflow dag etl etl-pipeline
Language:TypeScript 4143
Netflix / maestro
Maestro: Netflix’s Workflow Orchestrator
analytics automation batch-processing dag data-engineering data-ops data-orchestrator data-pipelines data-science elt etl java machine-learning maestro mlops orchestration scheduler workflow workflow-engine workflow-orchestration
Language:Java 3642
apache / hamilton
Apache Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage/tracing and metadata. Runs and scales everywhere python does.
data-science python dag data-engineering dataframe etl etl-framework etl-pipeline feature-engineering machine-learning pandas software-engineering data-analysis lineage llmops mlops orchestration hacktoberfest rag
Language:Jupyter Notebook 2303
CGraph
ChunelFeng / CGraph
【A common used C++ & Python DAG framework】一个通用的、无三方依赖的、跨平台的、收录于awesome-cpp的、基于流图的并行计算框架。欢迎star & fork & 交流
dag threadpool workflow taskflow pipeline graph python pybind11 framework multi-language
Language:C++ 2156
teamclairvoyant / airflow-maintenance-dags
A series of DAGs/Workflows to help maintain the operation of Airflow
airflow dag workflow airflow-maintenance-dags maintenance cleanup apache-airflow
Language:Python 1754
Qiskit / rustworkx
A high performance Python graph library implemented in Rust.
dag graph graph-theory python rust
Language:Rust 1503
DTStack / Taier
Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display
distributed-schedule-system dag scheduler job-scheduler workflow-scheduling-system task-schedule cronjob-scheduler data-schedule flink spark hive hadoop azkaban chunjun
Language:Java 1265
bellaj / Blockchain
Compilation of useful documents and scientific papers about Blockchain & cryptocurrencies.
bitcoin-ethereum blockchain bitcoin dag hyperledger ethereum whitepapers cryptography cryptocurrencies corda quorum aeternity mchain fractal
Language:JavaScript 1191
caicloud / cyclone
Powerful workflow engine and end-to-end pipeline solutions implemented with native Kubernetes resources. https://cyclone.dev
ai-pipeline ci-cd continuous-delivery continuous-integration dag kubernetes workflow
Language:Go 1072
swimlane / ngx-graph
Graph visualization library for angular
dag directed-graph chart svg angular angular2 data-viz workflow
Language:TypeScript 980
PaddlePaddle / Serving
A flexible, high-performance carrier for machine learning models（『飞桨』服务化部署框架）
paddle-serving rpc-service gpu python docker serving pipeline paddle deep-learning online-service prediction predictor dag micro-service microservice-toolkit
Language:C++ 919
stitchfix / hamilton
A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton
python pandas dag data-science data-engineering numpy software-engineering etl-framework etl-pipeline etl feature-engineering featurization dataframe stitch-fix data-platform hamilton hamiltonian machine-learning
Language:Python 858
openvinotoolkit / model_server
A scalable inference server for models optimized with OpenVINO™
openvino inference ai edge cloud deep-learning serving dag kubernetes machine-learning model-serving genai
Language:C++ 787
go-spacemesh
spacemeshos / go-spacemesh
Go Implementation of the Spacemesh protocol full node. 💾⏰💪
golang p2p blockchain consensus cryptocurrencies proof-of-work proof-of-space proof-of-space-time decentralization go blockmesh dag spacemesh
Language:Go 774
antvis / XFlow
React component for building interactive diagrams.
dag flowchart react x6
Language:TypeScript 645
deepcs233 / jieba_fast
Use C Api and Swig to Speed up jieba 高效的中文分词库
dag jieba python swig viterbi-hmm
Language:Python 639
erdogant / bnlearn
Python package for Causal Discovery by learning the graphical structure of Bayesian networks. Structure Learning, Parameter Learning, Inferences, Sampling methods.
bayesian-inference structure-learning sampling-methods dag directed-acyclic-graph parameter-learning causal-models causality-analysis causal-networks causal machine-learning python
Language:Jupyter Notebook 556
MixinNetwork / mixin
🚀 The Mixin-BFT-DAG network reference implementation.
blockchain bitcoin dag bft lightning-network mixin mixinnetwork
Language:Go 522
ytyht226 / taskflow
taskflow是一款轻量、简单易用、可灵活扩展的通用任务编排框架，基于有向无环图(DAG)的方式实现，框架提供了组件复用、同步/异步编排、条件判断、分支选择等能力，可以根据不同的业务场景对任意的业务流程进行编排
concurrent-programming dag java multi-threading multithreading parallel taskflow threadpool workflow
Language:Java 509
dagrs-dev / dagrs
A DAG engine build by Rust
dag rust hacktoberfest async concurrent fbp flow-based-programming parallel task tokio
Language:Rust 455
r-causal / ggdag
:arrow_lower_left: :arrow_lower_right: An R package for working with causal directed acyclic graphs (DAGs)
causal-inference dag ggplot-extension r rstats
Language:R 452
byteball / obyte-gui-wallet
Unstoppable money https://obyte.org
blockchain cryptocurrency cryptocurrency-wallet dag decentralization directed-acyclic-graph distributed-ledger-technology dlt obyte
Language:JavaScript 428
pipefunc
pipefunc / pipefunc
Lightweight fast function pipeline (DAG) creation in pure Python for scientific (HPC) workflows 🕸️🧪
dag hpc parallel-computing pipeline-framework pipelines reproducible-research slurm workflow-engine
Language:Python 427
murongqimiao / DAG-diagram
VUE plugin for DAG graph , u can use it 4 flow diagram or machine learning visualization
dag diagram visualization vue
Language:Vue 404
weibocom / rill-flow
Rill Flow is a high-performance, scalable workflow orchestration engine for distributed workloads and LLMs
agent ai backend-as-a-service cloud-native dag job-scheduler llm orchestration task-scheduler workflow
Language:Java 403
flowerfine / scaleph
Open data platform based on Kubernetes. Scaleph supports SeaTunnel、Flink and Doris backended by SeaTunnel on Flink engine、Flink Kubernetes Operator and Doris operator.
data-platform dataops dag seatunnel flink flink-sql flink-sql-gateway flink-kubernetes flink-kubernetes-operator doris doris-operator doris-manager
Language:Java 392
airflow-plugins / Example-Airflow-DAGs
Example DAGs using hooks and operators from Airflow Plugins
apache-airflow dag etl airflow-plugins selenium google-analytics facebook-ads sftp marketo salesforce hubspot marketo-sdk imap airflow mongodb mailgun singer dbt
Language:Python 347
XDagger / xdag
XDAG (Dagger Coin) Official Main Repository. XDAG is a novel DAG based cryptocurrency.
xdag dag cryptocurrency bitcoin blockchain blockchain-technology dagger xdagger
Language:C 327