bigdata

There are 286 repositories under bigdata topic.

TDengine
taosdata / TDengine
TDengine is an open source, high-performance, cloud native time-series database optimized for Internet of Things (IoT), Connected Cars, Industrial IoT and DevOps.
iot bigdata time-series database industrial-iot connected-vehicles monitoring tsdb tdengine sql time-series-database scalability cluster metrics financial-analysis cloud-native distributed
Language:C 22864
apache / shardingsphere
Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.
database distributed-database distributed-sql-database distributed-transactions sql shard database-cluster mysql postgresql database-plus encrypt dba rdbms oltp bigdata hacktoberfest
Language:Java 19470
heibaiying / BigData-Notes
大数据入门指南 :star:
hadoop hdfs yarn mapreduce hive spark storm hbase scala kafka zookeeper flume azkaban sqoop phoenix bigdata big-data
Language:Java 15336
oxnr / awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
awesome bigdata data-warehouse data-analytics data-visualization data-stream database data-science data awesome-list distributed-database series-database stream-processing streaming-data visualize-data
12834
juicefs
juicedata / juicefs
JuiceFS is a distributed POSIX file system built on top of Redis and S3.
filesystem cloud-native golang redis distributed-systems storage object-storage posix hdfs s3 bigdata go
Language:Go 9859
wangzhiwubigdata / God-Of-BigData
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
azkaban bigdata flink flume hadoop hbase hdfs hive kafka spark zookeeper
9339
DataExpert-io / data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
apachespark awesome bigdata data dataengineering sql
8684
vaexio / vaex
Out-of-Core hybrid Apache Arrow/NumPy DataFrame for Python, ML, visualization and exploration of big tabular data at a billion rows per second 🚀
bigdata data-science dataframe hdf5 machine-learning machinelearning memory-mapped-file pyarrow python tabular-data visualization
Language:Python 8180
databend
datafuselabs / databend
𝗗𝗮𝘁𝗮, 𝗔𝗻𝗮𝗹𝘆𝘁𝗶𝗰𝘀 & 𝗔𝗜. Modern alternative to Snowflake. Cost-effective and simple for massive-scale analytics. https://databend.com
ai bigdata database rust serverless snowflake
Language:Rust 7275
apache / hudi
Upserts, Deletes And Incremental Processing on Big Data.
hudi apachehudi datalake bigdata apachespark incremental-processing stream-processing data-integration apacheflink
Language:Java 5111
DTStack / chunjun
A data integration framework
flink bigdata data-integration framework java
Language:Java 3908
volcano-sh / volcano
A Cloud Native Batch System (Project under CNCF)
batch-systems bigdata gene golang hpc kubernetes machine-learning
Language:Go 3802
liyupi / sql-generator
🔨 用 JSON 来生成结构化的 SQL 语句，基于 Vue3 + TypeScript + Vite + Ant Design + MonacoEditor 实现，项目简单（重逻辑轻页面）、适合练手~
bigdata hive javascript json spark sql mysql typescript vite vue ant-design monaco-editor vue3
Language:Vue 3382
apache / avro
Apache Avro is a data serialization system.
avro bigdata c cplusplus csharp dotnet java perl php python ruby rust
Language:Java 2778
douban / dpark
Python clone of Spark, a MapReduce alike framework in Python
bigdata mapreduce dpark stream-processing spark python
Language:Python 2692
MoRan1607 / BigDataGuide
大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
hadoop zookeeper hive flume kafka hbase spark flink scala bigdata javase
Language:Java 2511
griddb / griddb
GridDB is a next-generation open source database that makes time series IoT and big data fast,and easy.
database griddb nosql iot fast time-series newsql sql bigdata timeseries
Language:C++ 2321
DTStack / flinkStreamSQL
基于开源的flink，对其实时sql进行扩展；主要实现了流与维表的join，支持原生flink SQL所有的语法
flink sql stream bigdata
Language:Java 2011
dotnet / spark
.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers.
spark csharp dotnet analytics bigdata spark-streaming spark-sql machine-learning fsharp dotnet-core dotnet-standard streaming apache-spark tpcds tpch azure hdinsight databricks emr microsoft
Language:C# 2000
shzlw / poli
An easy-to-use BI server built for SQL lovers. Power data analysis in SQL and gain faster business insights.
business-intelligence sql dashboard data-visualization sql-editor jdbc bigdata reporting spring-boot reactjs
Language:Java 1951
byzer-org / byzer-lang
Byzer (former MLSQL): A low-code open-source programming language for data pipeline, analytics and AI.
machine-learning mlsql bigdata sql-like-dsl
Language:Scala 1817
iGaoWei / BigDataView
100+套大数据可视化炫酷大屏Html5模板；包含行业：社区、物业、政务、交通、金融银行等，全网最新、最多，最全、最酷、最炫大数据可视化模板。陆续更新中
bigdata bigdataviewer echarts html-template viewmodel
Language:JavaScript 1716
Netflix / genie
Distributed Big Data Orchestration Service
big-data bigdata orchestration configuration configuration-management java spring-boot distributed-systems netflixoss cloud netflix-oss microservice microservices
Language:Java 1685
jadianes / spark-py-notebooks
Apache Spark & Python (pySpark) tutorials for Big Data Analysis and Machine Learning as IPython / Jupyter notebooks
spark python pyspark data-analysis mllib ipython-notebook notebook ipython data-science machine-learning big-data bigdata
Language:Jupyter Notebook 1610
YoongiKim / AutoCrawler
Google, Naver multiprocess image web crawler (Selenium)
crawler selenium deep-learning image-crawler chromedriver bigdata google multiprocess thread customizable python
Language:Python 1551
will-che / BigData-Interview
:dart: :star2:[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
bigdata spark kafka hbase flink hadoop hdfs mapreduce yarn interview interview-questions
1541
hi-primus / optimus
:truck: Agile Data Preparation Workflows made easy with Pandas, Dask, cuDF, Dask-cuDF, Vaex and PySpark
spark pyspark data-wrangling bigdata big-data-cleaning data-science data-cleansing data-cleaner data-transformation machine-learning data-profiling data-extraction data-exploration data-analysis data-preparation cudf dask dask-cudf data-cleaning
Language:Python 1447
tensorbase / tensorbase
TensorBase is a new big data warehousing with modern efforts.
rust bigdata database analytics modern infrastructure data data-infrastructure high-performance engineering rust-lang warehouse data-warehouse
Language:Rust 1431
collabH / bigdata-growth
大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。
flink kafka hive mapreduce spark olap kudu hadoop hbase debezium hdfs bigdata hudi bigdatalearning
Language:Shell 1294
odd-platform
opendatadiscovery / odd-platform
First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.
oss data-platform metadata metadata-management data-pipelines data-engineering observability data-catalog datacatalog data-discovery data-lineage bigdata alerting lineage data-profiling data-exploration data-governance data-quality data-science data-observability
Language:Java 1122
kubernetes-retired / kube-batch
A batch scheduler of kubernetes for high performance workload, e.g. AI/ML, BigData, HPC
bigdata hpc k8s-sig-scheduling kubernetes machine-learning
Language:Go 1069
josonle / Coding-Now
学习记录的一些笔记，以及所看得一些电子书eBooks、视频资源和平常收纳的一些自己认为比较好的博客、网站、工具。涉及大数据几大组件、Python机器学习和数据分析、Linux、操作系统、算法、网络等
ebook-collection notes coding java hadoop-hive spark linux bigdata
Language:Python 994
zeromicro / cds
Data syncing in golang for ClickHouse.
go golang clickhouse bigdata kafka-consumer
Language:Go 956
microsoft / Mobius
C# and F# language binding and extensions to Apache Spark
apache-spark bigdata csharp dataframe dataset dstream eventhubs fsharp kafka-streaming mapreduce mobius near-real-time rdd spark spark-streaming streaming
Language:C# 939
pingcap / tispark
TiSpark is built for running Apache Spark on top of TiDB/TiKV
bigdata spark tidb tikv
Language:Scala 877
apache / incubator-livy
Apache Livy is an open source REST interface for interacting with Apache Spark from anywhere.
livy bigdata spark apachelivy
Language:Scala 858

bigdata

taosdata / TDengine

apache / shardingsphere

heibaiying / BigData-Notes

oxnr / awesome-bigdata

juicedata / juicefs

wangzhiwubigdata / God-Of-BigData

DataExpert-io / data-engineer-handbook

vaexio / vaex

datafuselabs / databend

apache / hudi

DTStack / chunjun

volcano-sh / volcano

liyupi / sql-generator

apache / avro

douban / dpark

MoRan1607 / BigDataGuide

griddb / griddb

DTStack / flinkStreamSQL

dotnet / spark

shzlw / poli

byzer-org / byzer-lang

iGaoWei / BigDataView

Netflix / genie

jadianes / spark-py-notebooks

YoongiKim / AutoCrawler

will-che / BigData-Interview

hi-primus / optimus

tensorbase / tensorbase

collabH / bigdata-growth

opendatadiscovery / odd-platform

kubernetes-retired / kube-batch

josonle / Coding-Now

zeromicro / cds

microsoft / Mobius

pingcap / tispark

apache / incubator-livy