data-ops

There are 12 repositories under data-ops topic.

PrefectHQ / prefect
Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
automation data data-engineering data-ops data-science infrastructure ml-ops observability orchestration pipeline prefect python workflow workflow-engine
Language:Python 15437
Avaiga / taipy
Turns Data and AI algorithms into production-ready web applications in no time.
automation data-engineering data-ops data-visualization datascience developer-tools mlops orchestration pipeline pipelines python taipy-gui workflow taipy-core hacktoberfest hacktoberfest2023 data-integration job-scheduler scenario scenario-analysis
Language:Python 9911
connect
redpanda-data / connect
Fancy stream processing made operationally mundane
message-queue stream-processing streaming-data message-bus logs stream-processor cqrs event-sourcing go golang kafka amqp rabbitmq nats etl data-engineering data-ops
Language:Go 8006
MarquezProject / marquez
Collect, aggregate, and visualize a data ecosystem's metadata
data-lineage data-discovery data-governance data-provenance metadata-service data-dictionary marquez metadata data-ecosystem-metadata data-ops
Language:Java 1693
active_workflow
automaticmode / active_workflow
Polyglot workflows without leaving the comfort of your technology stack.
orchestration-framework scheduling workflow event-driven ifttt scheduler services-platform data-engineering data-ops activeworkflow agents self-hosted
Language:Ruby 815
weboptout
alexjc / weboptout
Opt-Out tool to check Copyright reservations in a way that even machines can understand.
opt-out command-line-tool robots-txt webscraping terms-of-service ml-pipeline data-ops copyright
Language:Python 192
snowflakedb / snowflake-cli
Snowflake CLI is an open-source command-line tool explicitly designed for developer-centric workloads in addition to SQL operations.
cli data-ops devops-tools snowflake sql
Language:Python 166
datachecks
waterdipai / datachecks
Open Source Data Quality Monitoring.
data-engineering data-validation dataops dataquality metrics mlops postgresql python data-governance data-observability database postgres sql data-quality-monitor data-quality-monitoring data-ops elasticsearch mysql etl monitoring
Language:Python 135
dqo
dqops / dqo
Data Quality and Observability platform for the whole data lifecycle, from profiling new data sources to full automation with Data Observability. Configure data quality checks from the UI or in YAML files, let DQOps run the data quality checks daily to detect data quality issues.
data-ops data-quality data-quality-checks data-quality-measurement data-quality-monitoring data-quality-report monitoring data-observability data-profiling
Language:Java 79
aspuru-guzik-group / funsies
funsies is a lightweight workflow engine 🔧
workflow-engine python infrastructure automation data-ops data-engineering redis hashtree
Language:Python 38
NucleusEngineering / hack-your-pipe
Efficient streaming data ingestion, transformation & activation
cloud-run dataflow pubsub vertex-ai data-engineering streaming-data data-ops ml-ops apache-beam google-cloud-platform
Language:Python 28
duyet / grant-rs
Manage Redshift/Postgres privileges in GitOps style written in Rust
rust redshift postgres gitops data-engineering data-ops hacktoberfest
Language:Rust 27
stairlight
tosh2230 / stairlight
A data lineage tool detects table dependencies from rendered SQL statements.
sql data-lineage redash dbt bigquery data-engineering gcs s3 data-catalog data-governance data-ops data-discovery data-management lineage
Language:Python 27
hundredlabs / console
Open source data infrastructure platform. Designed for developers, built for speed.
spark kafka hadoop hadoop-hdfs data-ops
Language:TypeScript 22
glentner / dataphile
Data analytics library for Python and suite of open source, command line based data ops tools.
python data-science data-analysis scientific-computing data-ops
Language:Python 12
GuinsooLab / sheenflow
A next-generation open source orchestration platform for the development, production, and observation of data assets.
dag orchestration-framework schedule scheduled-tasks taskflow workflow workflows airflow data-ops guinsoolab infrastructure observability pipeline workflow-engine orchestration orchestration-engine
Language:Python 11
Snowflake-Labs / snowflake-cli-action
Github Action enabling easy use of Snowflake CLI in your CI/CD workflows
actions data-ops devops-tools snowflake sql
Language:Shell 9
marwan116 / supreme-task
A prefect extension that builds on top of the task decorator to reduce negative engineering!
data-ops data-science infrastructure ml-ops orchestration prefect python workflow
Language:Python 5
ysv-rs / ysv
ysv: clean and transform CSV data along your rules encoded in YAML format, lightning fast
csv-converter rust-lang yaml-configuration data-science data-mining data-ops
Language:Rust 4
Quantyca / content-qshare
Raccogliamo qui tutti i link alle risorse menzionate durante i nostri QShare
data-engineering data-science machine-learning deep-learning artificial-intelligence data-governance data-ops ml-ops dev-ops data-visualisation software-engineering
3
kirnosenko / Repositorch
Repositorch is a VCS repository analysis engine written in C#.
mining-software-repositories git git-analysis statistics dev-ops data-ops
Language:C# 1
aacraf / projecteADSDB
simple data platform to study which country indicators are relevant for olympics sports outcome
data-discovery data-ops data-platform-foundations data-science
Language:HTML 0
test
marisogo / test
Turns Data and AI algorithms into full web applications in no time.
automation data-engineering data-ops data-visualization mlops python taipy-core taipy-gui developer-tools pipeline workflow datascience orchestration pipelines
0
retargetly / ds-search-engine
data-ops
Language:Dockerfile 0
usedatabrew / open_ai_benthos_processor
Open AI processor for Benthos
benthos benthos-plugin golang-library openai etl etl-pipeline data-engineering data-ops event-sourcing stream-processor streaming-data
Language:Go 0