bigdata

There are 287 repositories under bigdata topic.

TDengine
taosdata / TDengine
TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.
iot bigdata time-series database industrial-iot connected-vehicles monitoring tsdb tdengine sql time-series-database scalability cluster metrics financial-analysis cloud-native distributed
Language:C 23076
apache / shardingsphere
Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
database distributed-database distributed-sql-database distributed-transactions sql shard database-cluster mysql postgresql database-plus encrypt dba rdbms oltp bigdata hacktoberfest
Language:Java 19659
heibaiying / BigData-Notes
大数据入门指南 :star:
azkaban big-data bigdata flume hadoop hbase hdfs hive kafka mapreduce phoenix scala spark sqoop storm yarn zookeeper
Language:Java 15571
oxnr / awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
awesome awesome-list bigdata data data-analytics data-science data-stream data-visualization data-warehouse database distributed-database series-database stream-processing streaming-data visualize-data
13021
juicefs
juicedata / juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
bigdata cloud-native distributed-systems filesystem go golang hdfs object-storage posix redis s3 storage
Language:Go 10292
DataExpert-io / data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
apachespark awesome bigdata data dataengineering sql
9920
wangzhiwubigdata / God-Of-BigData
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
flink spark hadoop hdfs hive hbase kafka zookeeper bigdata flume azkaban
9521
vaexio / vaex
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
bigdata data-science dataframe hdf5 machine-learning machinelearning memory-mapped-file pyarrow python tabular-data visualization
Language:Python 8233
databend
datafuselabs / databend
𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
ai bigdata database rust serverless snowflake
Language:Rust 7531
apache / hudi
Upserts, Deletes And Incremental Processing on Big Data.
apacheflink apachehudi apachespark bigdata data-integration datalake hudi incremental-processing stream-processing
Language:Java 5245
volcano-sh / volcano
A Cloud Native Batch System (Project under CNCF)
batch-systems kubernetes golang hpc bigdata machine-learning gene
Language:Go 3958
DTStack / chunjun
A data integration framework
flink bigdata data-integration framework java
Language:Java 3956
liyupi / sql-generator
🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~
bigdata hive javascript json spark sql mysql typescript vite vue ant-design monaco-editor vue3
Language:Vue 3401
apache / avro
Apache Avro is a data serialization system.
csharp ruby cplusplus python php java c avro bigdata dotnet perl rust
Language:Java 2840
douban / dpark
Python clone of Spark, a MapReduce alike framework in Python
bigdata dpark mapreduce python spark stream-processing
Language:Python 2691
MoRan1607 / BigDataGuide
大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
hadoop zookeeper hive flume kafka hbase spark flink scala bigdata javase
Language:Java 2584
griddb / griddb
GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.
bigdata database fast griddb iot newsql nosql sql time-series timeseries
Language:C++ 2349
DTStack / flinkStreamSQL
基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法
bigdata flink sql stream
Language:Java 2017
dotnet / spark
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
analytics apache-spark azure bigdata csharp databricks dotnet dotnet-core dotnet-standard emr fsharp hdinsight machine-learning microsoft spark spark-sql spark-streaming streaming tpcds tpch
Language:C# 2012
shzlw / poli
An easy-to-use BI server built for SQL lovers. Power data analysis in SQL and gain faster business insights.
business-intelligence sql dashboard data-visualization sql-editor jdbc bigdata reporting spring-boot reactjs
Language:Java 1958
iGaoWei / BigDataView
100+套大数据可视化炫酷大屏Html5模板；包含行业：社区、物业、政务、交通、金融银行等，全网最新、最多，最全、最酷、最炫大数据可视化模板。陆续更新中
bigdata bigdataviewer echarts html-template viewmodel
Language:JavaScript 1869
byzer-org / byzer-lang
Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.
machine-learning mlsql bigdata sql-like-dsl
Language:Scala 1825
Netflix / genie
Distributed Big Data Orchestration Service
big-data bigdata orchestration configuration configuration-management java spring-boot distributed-systems netflixoss cloud netflix-oss microservice microservices
Language:Java 1700
jadianes / spark-py-notebooks
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
big-data bigdata data-analysis data-science ipython ipython-notebook machine-learning mllib notebook pyspark python spark
Language:Jupyter Notebook 1623
YoongiKim / AutoCrawler
Google, Naver multiprocess image web crawler (Selenium)
crawler selenium deep-learning image-crawler chromedriver bigdata google multiprocess thread customizable python
Language:Python 1580
water8394 / BigData-Interview
:dart: :star2:[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
bigdata spark kafka hbase flink hadoop hdfs mapreduce yarn interview interview-questions
1559
hi-primus / optimus
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
spark pyspark data-wrangling bigdata big-data-cleaning data-science data-cleansing data-cleaner data-transformation machine-learning data-profiling data-extraction data-exploration data-analysis data-preparation cudf dask dask-cudf data-cleaning
Language:Python 1465
tensorbase / tensorbase
TensorBase is a new big data warehousing with modern efforts.
analytics bigdata data data-infrastructure data-warehouse database engineering high-performance infrastructure modern rust rust-lang warehouse
Language:Rust 1436
collabH / bigdata-growth
大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。
flink kafka hive mapreduce spark olap kudu hadoop hbase debezium hdfs bigdata hudi bigdatalearning
Language:Shell 1391
odd-platform
opendatadiscovery / odd-platform
First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.
oss data-platform metadata metadata-management data-pipelines data-engineering observability data-catalog datacatalog data-discovery data-lineage bigdata alerting lineage data-profiling data-exploration data-governance data-quality data-science data-observability
Language:Java 1171
kubernetes-retired / kube-batch
A batch scheduler of kubernetes for high performance workload, e.g. AI/ML, BigData, HPC
kubernetes k8s-sig-scheduling machine-learning bigdata hpc
Language:Go 1072
josonle / Coding-Now
学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等
ebook-collection notes coding java hadoop-hive spark linux bigdata
Language:Python 994
zeromicro / cds
Data syncing in golang for ClickHouse.
go golang clickhouse bigdata kafka-consumer
Language:Go 965
microsoft / Mobius
C# and F# language binding and extensions to Apache Spark
spark apache-spark rdd dataframe dstream dataset streaming csharp mobius kafka-streaming spark-streaming fsharp bigdata mapreduce eventhubs near-real-time
Language:C# 942
pingcap / tispark
TiSpark is built for running Apache Spark on top of TiDB/TiKV
bigdata spark tidb tikv
Language:Scala 880
apache / incubator-livy
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
livy bigdata spark apachelivy
Language:Scala 873

bigdata

taosdata / TDengine

apache / shardingsphere

heibaiying / BigData-Notes

oxnr / awesome-bigdata

juicedata / juicefs

DataExpert-io / data-engineer-handbook

wangzhiwubigdata / God-Of-BigData

vaexio / vaex

datafuselabs / databend

apache / hudi

volcano-sh / volcano

DTStack / chunjun

liyupi / sql-generator

apache / avro

douban / dpark

MoRan1607 / BigDataGuide

griddb / griddb

DTStack / flinkStreamSQL

dotnet / spark

shzlw / poli

iGaoWei / BigDataView

byzer-org / byzer-lang

Netflix / genie

jadianes / spark-py-notebooks

YoongiKim / AutoCrawler

water8394 / BigData-Interview

hi-primus / optimus

tensorbase / tensorbase

collabH / bigdata-growth

opendatadiscovery / odd-platform

kubernetes-retired / kube-batch

josonle / Coding-Now

zeromicro / cds

microsoft / Mobius

pingcap / tispark

apache / incubator-livy