bigdata

There are 285 repositories under bigdata topic.

TDengine
taosdata / TDengine
TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.
bigdata cloud-native cluster connected-vehicles database distributed financial-analysis industrial-iot iot metrics monitoring scalability sql tdengine time-series time-series-database tsdb
Language:C 22784
apache / shardingsphere
Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
bigdata database database-cluster database-plus dba distributed-database distributed-sql-database distributed-transactions encrypt hacktoberfest mysql oltp postgresql rdbms shard sql
Language:Java 19422
heibaiying / BigData-Notes
大数据入门指南 :star:
azkaban big-data bigdata flume hadoop hbase hdfs hive kafka mapreduce phoenix scala spark sqoop storm yarn zookeeper
Language:Java 15264
newTendermint / awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
awesome awesome-list bigdata data data-analytics data-science data-stream data-visualization data-warehouse database distributed-database series-database stream-processing streaming-data visualize-data
12784
juicefs
juicedata / juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
filesystem cloud-native golang redis distributed-systems storage object-storage posix hdfs s3 bigdata go
Language:Go 9781
wangzhiwubigdata / God-Of-BigData
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
azkaban bigdata flink flume hadoop hbase hdfs hive kafka spark zookeeper
9266
DataExpert-io / data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
apachespark awesome bigdata data dataengineering sql
8536
vaexio / vaex
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
bigdata data-science dataframe hdf5 machine-learning machinelearning memory-mapped-file pyarrow python tabular-data visualization
Language:Python 8173
databend
datafuselabs / databend
𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
ai bigdata database rust serverless snowflake
Language:Rust 7173
apache / hudi
Upserts, Deletes And Incremental Processing on Big Data.
apacheflink apachehudi apachespark bigdata data-integration datalake hudi incremental-processing stream-processing
Language:Java 5053
DTStack / chunjun
A data integration framework
flink bigdata data-integration framework java
Language:Java 3890
volcano-sh / volcano
A Cloud Native Batch System (Project under CNCF)
batch-systems kubernetes golang hpc bigdata machine-learning gene
Language:Go 3749
liyupi / sql-generator
🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~
bigdata hive javascript json spark sql mysql typescript vite vue ant-design monaco-editor vue3
Language:Vue 3375
apache / avro
Apache Avro is a data serialization system.
csharp ruby cplusplus python php java c avro bigdata dotnet perl rust
Language:Java 2756
douban / dpark
Python clone of Spark, a MapReduce alike framework in Python
bigdata dpark mapreduce python spark stream-processing
Language:Python 2693
MoRan1607 / BigDataGuide
大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
hadoop zookeeper hive flume kafka hbase spark flink scala bigdata javase
Language:Java 2482
griddb / griddb
GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.
bigdata database fast griddb iot newsql nosql sql time-series timeseries
Language:C++ 2306
DTStack / flinkStreamSQL
基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法
flink sql stream bigdata
Language:Java 2007
dotnet / spark
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
spark csharp dotnet analytics bigdata spark-streaming spark-sql machine-learning fsharp dotnet-core dotnet-standard streaming apache-spark tpcds tpch azure hdinsight databricks emr microsoft
Language:C# 1995
shzlw / poli
An easy-to-use BI server built for SQL lovers. Power data analysis in SQL and gain faster business insights.
business-intelligence sql dashboard data-visualization sql-editor jdbc bigdata reporting spring-boot reactjs
Language:Java 1951
byzer-org / byzer-lang
Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.
machine-learning mlsql bigdata sql-like-dsl
Language:Scala 1815
Netflix / genie
Distributed Big Data Orchestration Service
big-data bigdata orchestration configuration configuration-management java spring-boot distributed-systems netflixoss cloud netflix-oss microservice microservices
Language:Java 1679
iGaoWei / BigDataView
100+套大数据可视化炫酷大屏Html5模板；包含行业：社区、物业、政务、交通、金融银行等，全网最新、最多，最全、最酷、最炫大数据可视化模板。陆续更新中
bigdata bigdataviewer echarts html-template viewmodel
Language:JavaScript 1644
jadianes / spark-py-notebooks
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
spark python pyspark data-analysis mllib ipython-notebook notebook ipython data-science machine-learning big-data bigdata
Language:Jupyter Notebook 1607
will-che / BigData-Interview
:dart: :star2:[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
bigdata spark kafka hbase flink hadoop hdfs mapreduce yarn interview interview-questions
1529
YoongiKim / AutoCrawler
Google, Naver multiprocess image web crawler (Selenium)
crawler selenium deep-learning image-crawler chromedriver bigdata google multiprocess thread customizable python
Language:Python 1506
hi-primus / optimus
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
big-data-cleaning bigdata cudf dask dask-cudf data-analysis data-cleaner data-cleaning data-cleansing data-exploration data-extraction data-preparation data-profiling data-science data-transformation data-wrangling machine-learning pyspark spark
Language:Python 1442
tensorbase / tensorbase
TensorBase is a new big data warehousing with modern efforts.
analytics bigdata data data-infrastructure data-warehouse database engineering high-performance infrastructure modern rust rust-lang warehouse
Language:Rust 1429
collabH / bigdata-growth
大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。
flink kafka hive mapreduce spark olap kudu hadoop hbase debezium hdfs bigdata hudi bigdatalearning
Language:Shell 1256
odd-platform
opendatadiscovery / odd-platform
First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.
oss data-platform metadata metadata-management data-pipelines data-engineering observability data-catalog datacatalog data-discovery data-lineage bigdata alerting lineage data-profiling data-exploration data-governance data-quality data-science data-observability
Language:Java 1108
kubernetes-retired / kube-batch
A batch scheduler of kubernetes for high performance workload, e.g. AI/ML, BigData, HPC
kubernetes k8s-sig-scheduling machine-learning bigdata hpc
Language:Go 1069
josonle / Coding-Now
学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等
ebook-collection notes coding java hadoop-hive spark linux bigdata
Language:Python 992
zeromicro / cds
Data syncing in golang for ClickHouse.
bigdata clickhouse go golang kafka-consumer
Language:Go 953
microsoft / Mobius
C# and F# language binding and extensions to Apache Spark
spark apache-spark rdd dataframe dstream dataset streaming csharp mobius kafka-streaming spark-streaming fsharp bigdata mapreduce eventhubs near-real-time
Language:C# 937
pingcap / tispark
TiSpark is built for running Apache Spark on top of TiDB/TiKV
bigdata spark tidb tikv
Language:Scala 878
apache / incubator-livy
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
livy bigdata spark apachelivy
Language:Scala 851

bigdata

taosdata / TDengine

apache / shardingsphere

heibaiying / BigData-Notes

newTendermint / awesome-bigdata

juicedata / juicefs

wangzhiwubigdata / God-Of-BigData

DataExpert-io / data-engineer-handbook

vaexio / vaex

datafuselabs / databend

apache / hudi

DTStack / chunjun

volcano-sh / volcano

liyupi / sql-generator

apache / avro

douban / dpark

MoRan1607 / BigDataGuide

griddb / griddb

DTStack / flinkStreamSQL

dotnet / spark

shzlw / poli

byzer-org / byzer-lang

Netflix / genie

iGaoWei / BigDataView

jadianes / spark-py-notebooks

will-che / BigData-Interview

YoongiKim / AutoCrawler

hi-primus / optimus

tensorbase / tensorbase

collabH / bigdata-growth

opendatadiscovery / odd-platform

kubernetes-retired / kube-batch

josonle / Coding-Now

zeromicro / cds

microsoft / Mobius

pingcap / tispark

apache / incubator-livy