big-data

There are 350 repositories under big-data topic.

awesome-scalability
binhnguyennus / awesome-scalability
The Patterns of Scalable, Reliable, and Performant Large-Scale Systems
system-design backend scalability interview architecture devops design-patterns interview-questions awesome-list big-data awesome resources lists web-development programming system interview-practice computer-science distributed-systems machine-learning
45386
apache / spark
Apache Spark - A unified analytics engine for large-scale data processing
big-data java jdbc python r scala spark sql
Language:Scala 35915
ClickHouse
ClickHouse / ClickHouse
ClickHouse® is a free analytics DBMS for big data
dbms olap analytics sql distributed-database big-data mpp clickhouse hacktoberfest
Language:C++ 29070
donnemartin / data-science-ipython-notebooks
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
python machine-learning deep-learning data-science big-data aws tensorflow theano caffe scikit-learn kaggle spark mapreduce hadoop matplotlib pandas numpy scipy keras
Language:Python 25159
apache / flink
Apache Flink
big-data flink java python scala sql
Language:Java 21346
amark / gun
An open source cybersecurity protocol for syncing decentralized graph data.
machine-learning artificial-intelligence big-data blockchain p2p decentralized graph cryptography crypto offline-first realtime crdt protocol database end-to-end encryption dweb dapp web3 metaverse
Language:JavaScript 17213
presto
prestodb / presto
The official home of the Presto distributed SQL query engine for big data
java presto hive hadoop big-data sql
Language:Java 14759
heibaiying / BigData-Notes
大数据入门指南 :star:
hadoop hdfs yarn mapreduce hive spark storm hbase scala kafka zookeeper flume azkaban sqoop phoenix bigdata big-data
Language:Java 13985
apache / predictionio
PredictionIO, a machine learning server for developers and ML engineers.
big-data predictionio scala
Language:Scala 12555
andkret / Cookbook
The Data Engineering Cookbook
best-practices big-data cookbook data-engineer data-engineering
11957
questdb
questdb / questdb
An open source time-series database for fast ingest and SQL queries
time-series low-latency database sql postgres grafana simd questdb tsdb java iot hacktoberfest postgresql cpp analytics time-series-database big-data financial-analysis
Language:Java 11523
yahoo / CMAK
CMAK is a tool for managing Apache Kafka clusters
kafka scala cluster-management big-data
Language:Scala 11386
nebula
vesoft-inc / nebula
A distributed, fast open-source graph database featuring horizontal scalability and high availability
graph-database distributed database graphdb raft cpp nebula-graph nebula graph nebulagraph big-data distributed-systems scalability
Language:C++ 9130
cython / cython
The most widely used Python to C compiler
python cython cpython cpython-extensions c cpp performance big-data
Language:Python 7979
trinodb / trino
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
java presto hive hadoop big-data sql prestodb database databases distributed-systems distributed-database data-science datalake jdbc query-engine trino analytics delta-lake iceberg
Language:Java 7913
catboost / catboost
A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.
machine-learning decision-trees gradient-boosting gbm gbdt python r kaggle gpu-computing catboost tutorial categorical-features gpu coreml data-science big-data cuda data-mining
Language:Python 7173
apache / beam
Apache Beam is a unified programming model for Batch and Streaming data processing.
python java big-data beam batch golang sql streaming
Language:Java 6907
apache / storm
Mirror of Apache Storm
storm java big-data
Language:Java 6454
h2oai / h2o-3
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
h2o machine-learning data-science deep-learning big-data ensemble-learning gbm random-forest naive-bayes pca opensource distributed java python r hadoop spark gpu automl h2o-automl
Language:Jupyter Notebook 6299
apache / zeppelin
Web-based notebook that enables data-driven, interactive data analytics and collaborative documents with SQL, Scala and more.
scala java big-data zeppelin javascript spark flink database nosql
Language:Java 6060
delta-io / delta
An open-source storage framework that enables building a Lakehouse architecture with compute engines including Spark, PrestoDB, Flink, Trino, and Hive and APIs
acid analytics big-data delta-lake spark
Language:Scala 6040
kafka-ui
provectus / kafka-ui
Open-Source Web UI for Apache Kafka Management
kafka-ui kafka-brokers kafka kafka-streams kafka-client opensource kafka-connect kafka-producer streams big-data apache-kafka cluster-management web-ui kafka-manager kafka-cluster streaming-data event-streaming hacktoberfest
Language:Java 5953
pachyderm / pachyderm
Data-Centric Pipelines and Data Versioning
go pachyderm docker analytics big-data containers distributed-systems kubernetes data-science data-analysis
Language:Go 5926
couchdb
apache / couchdb
Seamless multi-master syncing database with an intuitive HTTP/JSON API, designed for reliability
content network-server http cloud erlang javascript couchdb big-data network-client database
Language:Erlang 5699
arkime / arkime
Arkime (formerly Moloch) is an open source, large scale, full packet capturing, indexing, and database system.
network-monitoring pcap packet-capture nsm javascript c big-data security
Language:JavaScript 5645
hazelcast / hazelcast
Open-source distributed computation and storage platform. Real-time Stream Processing Unconference. Save Your Spot https://hazelcast.com/lp/unconference/
java hazelcast in-memory big-data scalability distributed caching hacktoberfest stream-processing low-latency distributed-computing distributed-systems
Language:Java 5386
apache / hive
Apache Hive
java hive database sql apache big-data hadoop
Language:Java 4840
tschellenbach / Stream-Framework
Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:
activity-stream cassandra feed news news-feed big-data redis activity-feed
Language:Python 4694
StarRocks / starrocks
StarRocks is a next-gen sub-second MPP database for full analytics scenarios, including multi-dimensional analytics, real-time analytics and ad-hoc query.
database olap sql analytics big-data realtime-database vectorized distributed-database data-analytics real-time-analytics mpp cbo join star-schema real-time-updates multi-table-relational-sql delta-lake hudi iceberg lakehouse
Language:Java 4641
apache / ignite
Apache Ignite
distributed-sql-database iot osgi network-client ignite data-management-platform big-data cloud database network-server hadoop sql cache in-memory-computing in-memory-database
Language:Java 4465
vespa
vespa-engine / vespa
The open big data serving engine. https://vespa.ai
vespa search-engine big-data ai serving serving-recommendation machine-learning server tensorflow java cpp vector-search
Language:Java 4433
feast-dev / feast
Feature Store for Machine Learning
big-data data-engineering data-quality data-science feature-store features machine-learning ml mlops python
Language:Python 4354
SynapseML
microsoft / SynapseML
Simple and Distributed Machine Learning
spark pyspark azure scala microsoft ml machine-learning databricks cognitive-services lightgbm http model-deployment deep-learning ai apache-spark data-science synapse big-data onnx opencv
Language:Scala 4295
tangbc / vue-virtual-scroll-list
⚡️A vue component support big amount data list with high render performance and efficient.
big-data infinite-scroll virtual-list
Language:JavaScript 3998
apache / calcite
Apache Calcite
geospatial calcite java big-data hadoop sql
Language:Java 3881
apache / arrow-datafusion
Apache Arrow DataFusion SQL Query Engine
arrow big-data dataframe datafusion olap python query-engine rust sql
Language:Rust 3714

Related Topics

dotnet pytorch laravel bitcoin library