There are 47 repositories under data-integration topic.
Apache Airflow - A platform to programmatically author, schedule, and monitor workflows
The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
An orchestration platform for the development, production, and observation of data assets.
SeaTunnel is a next-generation super high-performance, distributed, massive data integration tool.
Infinitely scalable, event-driven, language-agnostic orchestration and scheduling platform to manage millions of workflows declaratively in code.
The open source high performance ELT framework powered by Apache Arrow
Privacy and Security focused Segment-alternative, in Golang and React
Jitsu is an open-source Segment alternative. Fully-scriptable data ingestion engine for modern data teams. Set-up a real-time data pipeline in minutes, not days
Community-curated list of software packages and data resources for single-cell, including RNA-seq, ATAC-seq, etc.
Lean and mean distributed stream processing system written in rust and web assembly. Alternative to Kafka + Flink in one.
Apache DevLake is an open-source dev data platform to ingest, analyze, and visualize the fragmented data from DevOps tools, extracting insights for engineering excellence, developer experience, and community growth.
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
A lightweight opinionated ETL framework, halfway between plain scripts and Apache Airflow
BitSail is a distributed high-performance data integration engine which supports batch, streaming and incremental scenarios. BitSail is widely used to synchronize hundreds of trillions of data every day.
Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data. Currently, the following data connectors are available worldwide: a) High-resolution demographics data b) Point of Interests from Open Street Map c) Google Popular Times
Database replication platform that leverages change data capture. Stream production data from databases to your data warehouse (Snowflake, BigQuery, Redshift) in real-time.
汇总Apache Hudi相关资料
Fast, sensitive and accurate integration of single-cell data with Harmony
SeaTunnel is a distributed, high-performance data integration platform for the synchronization and transformation of massive data (offline & real-time).
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
Categorical Query Language IDE
Self-contained distributed software platform for building stateful, massively real-time streaming applications in Rust.
Dataplane is an Airflow inspired unified data platform with additional data mesh and RPA capability to automate, schedule and design data pipelines and workflows. Dataplane is written in Golang with a React front end.
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
🎼 Integrate multiple high-dimensional datasets with fuzzy k-means and locally linear adjustments.