hadoop

There are 193 repositories under hadoop topic.

donnemartin / data-science-ipython-notebooks
Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
aws big-data caffe data-science deep-learning hadoop kaggle keras machine-learning mapreduce matplotlib numpy pandas python scikit-learn scipy spark tensorflow theano
Language:Python 26542
spotify / luigi
Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
hadoop luigi orchestration-framework python scheduling
Language:Python 17380
Tencent / APIJSON
🏆 零代码、全功能、强安全 ORM 库 🚀 后端接口和文档零代码，前端(客户端) 定制返回 JSON 的数据和结构。 🏆 A JSON Transmission Protocol and an ORM Library 🚀 provides APIs and Docs without writing any code.
postgresql-database tidb sqlserver oracle clickhouse postgresql hive hadoop tdengine low-code lowcode nocode serverless baas crud milvus elasticsearch influxdb databricks snowflake
Language:Java 16697
presto
prestodb / presto
The official home of the Presto distributed SQL query engine for big data
java presto hive hadoop big-data sql data lakehouse query
Language:Java 15626
heibaiying / BigData-Notes
大数据入门指南 :star:
hadoop hdfs yarn mapreduce hive spark storm hbase scala kafka zookeeper flume azkaban sqoop phoenix bigdata big-data
Language:Java 15339
apache / hadoop
Apache Hadoop
hadoop
Language:Java 14396
deeplearning4j / deeplearning4j
Suite of tools for deploying and training deep learning models using the JVM. Highlights include model import for keras, tensorflow, and onnx/pytorch, a modular and tiny c++ library for running math code and a java based math library on top of the core c++ library. Also includes samediff: a pytorch/tensorflow like library for running deep learning using automatic differentiation.
java gpu deeplearning neural-nets deeplearning4j dl4j hadoop spark intellij artificial-intelligence python scala clojure linear-algebra matrix-library
Language:Java 13455
apache / doris
Apache Doris is an easy-to-use, high performance and unified analytics database.
olap database hadoop hive hudi iceberg real-time sql bigquery dbt delta-lake elt etl lakehouse query-engine redshift snowflake spark
Language:Java 11500
trinodb / trino
Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
analytics big-data data-science database databases datalake delta-lake distributed-database distributed-systems hadoop hive iceberg java jdbc presto prestodb query-engine sql trino
Language:Java 9640
wangzhiwubigdata / God-Of-BigData
专注大数据学习面试，大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...
flink spark hadoop hdfs hive hbase kafka zookeeper bigdata flume azkaban
9340
school-of-sre
linkedin / school-of-sre
At LinkedIn, we are using this curriculum for onboarding our entry-level talents into the SRE role.
sre linux networking git python mysql nosql hadoop system-design security
Language:HTML 7662
h2oai / h2o-3
H2O is an Open Source, Distributed, Fast & Scalable Machine Learning Platform: Deep Learning, Gradient Boosting (GBM) & XGBoost, Random Forest, Generalized Linear Modeling (GLM with Elastic Net), K-Means, PCA, Generalized Additive Models (GAM), RuleFit, Support Vector Machine (SVM), Stacked Ensembles, Automatic Machine Learning (AutoML), etc.
automl big-data data-science deep-learning distributed ensemble-learning gbm gpu h2o h2o-automl hadoop java machine-learning naive-bayes opensource pca python r random-forest spark
Language:Jupyter Notebook 6748
Alluxio / alluxio
Alluxio, data orchestration for analytics and machine learning in the cloud
alluxio memory-speed hadoop spark presto tensorflow data-analysis data-orchestration virtual-distributed-filesystem
Language:Java 6672
apache / hive
Apache Hive
java hive database sql apache big-data hadoop
Language:Java 5346
apache / ignite
Apache Ignite
distributed-sql-database iot osgi network-client ignite data-management-platform big-data cloud database network-server hadoop sql cache in-memory-computing in-memory-database
Language:Java 4697
apache / calcite
Apache Calcite
big-data calcite geospatial hadoop java sql
Language:Java 4386
tomwhite / hadoop-book
Example source code accompanying O'Reilly's "Hadoop: The Definitive Guide" by Tom White
book hadoop o-reilly
Language:Makefile 3489
WeBankFinTech / DataSphereStudio
DataSphereStudio is a one stop data application development& management portal, covering scenarios including data exchange, desensitization/cleansing, analysis/mining, quality measurement, visualization, and task scheduling.
workflow governance azkaban davinci linkis spark hive hadoop visualis zeppelin hue supperset tableau dataworks griffin kettle airflow flink dolphinscheduler atlas
Language:Java 2975
apache / nutch
Apache Nutch is an extensible and scalable web crawler
java nutch web-crawler crawling hadoop apache
Language:Java 2824
LuckyZXL2016 / Movie_Recommend
基于Spark的电影推荐系统，包含爬虫项目、web网站、后台管理系统以及spark推荐系统
spark-mllib spark-streaming ssm-maven scrapy scala hadoop nginx hive mysql
Language:Java 2715
MoRan1607 / BigDataGuide
大数据学习，从零开始学习大数据，包含大数据学习各阶段学习视频、面试资料
hadoop zookeeper hive flume kafka hbase spark flink scala bigdata javase
Language:Java 2511
HariSekhon / DevOps-Bash-tools
1000+ DevOps Bash Scripts - AWS, GCP, Kubernetes, Docker, CI/CD, APIs, SQL, PostgreSQL, MySQL, Hive, Impala, Kafka, Hadoop, Jenkins, GitHub, GitLab, BitBucket, Azure DevOps, TeamCity, Spotify, MP3, LDAP, Code/Build Linting, pkg mgmt for Linux, Mac, Python, Perl, Ruby, NodeJS, Golang, Advanced dotfiles: .bashrc, .vimrc, .gitconfig, .screenrc, tmux..
api aws bash ci cloudera devops docker gcp git github hacktoberfest hadoop jenkins kafka kubernetes linux mysql perl postgresql terraform
Language:Shell 2472
SZT-bigdata
geekyouth / SZT-bigdata
深圳地铁大数据客流分析系统🚇🚄🌟
kafka szt-bigdata flink spark scala elasticsearch kibana hive redis mongodb hadoop kylin mysql phoenix hbase zookeeper clickhouse docker cdh6 springboot
Language:Scala 2167
big-data-europe / docker-hadoop
Apache Hadoop docker image
hadoop-docker docker-hadoop hadoop hadoop-cluster docker
Language:Shell 2128
dahuoyzs / javapdf
🍣100本 Java电子书技术书籍PDF(以下载阅读为荣，以点赞收藏为耻)
hadoop java-pdf jvm mysql mysql-innodb
1987
apache / kyuubi
Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
spark hive sql thrift jdbc spark-sql data-lake hadoop kubernetes hacktoberfest
Language:Scala 1951
apache / drill
Apache Drill is a distributed MPP query layer for self describing data
java big-data drill sql hive hadoop jdbc parquet
Language:Java 1902
cdarlint / winutils
winutils.exe hadoop.dll and hdfs.dll binaries for hadoop windows
binaries hadoop winutils
Language:Shell 1782
gchq / Gaffer
A large-scale entity and relation database supporting aggregation of properties
accumulo graph graph-database hadoop big-data aggregation hbase parquet spark
Language:Java 1737
Qihoo360 / XLearning
AI on Hadoop
hadoop tensorflow caffe mxnet ai deeplearning machinelearning yarn
Language:Java 1732
moosefs / moosefs
MooseFS – Open Source, Petabyte, Fault-Tolerant, Highly Performing, Scalable Network Distributed File System (Software-Defined Storage)
software-defined-storage posix filesystem file-system distributed-file-system clustering distributed-storage distributed-computing fuse big-data snapshot storage-tiering high-availability scalability storage moosefs hadoop posix-compliant petabyte ditributed-systems
Language:C 1592
will-che / BigData-Interview
:dart: :star2:[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结
bigdata spark kafka hbase flink hadoop hdfs mapreduce yarn interview interview-questions
1545
apache / carbondata
High performance data store solution
scala java big-data carbondata data-format apache spark hadoop
Language:Scala 1419
collabH / bigdata-growth
大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。
flink kafka hive mapreduce spark olap kudu hadoop hbase debezium hdfs bigdata hudi bigdatalearning
Language:Shell 1298
DTStack / Taier
Taier is a big data development platform for submission, scheduling, operation and maintenance, and indicator information display
distributed-schedule-system dag scheduler job-scheduler workflow-scheduling-system task-schedule cronjob-scheduler data-schedule flink spark hive hadoop azkaban chunjun
Language:Java 1281
HariSekhon / Dockerfiles
50+ DockerHub public images for Docker & Kubernetes - DevOps, CI/CD, GitHub Actions, CircleCI, Jenkins, TeamCity, Alpine, CentOS, Debian, Fedora, Ubuntu, Hadoop, Kafka, ZooKeeper, HBase, Cassandra, Solr, SolrCloud, Presto, Apache Drill, Nifi, Spark, Consul, Riak
hadoop hbase cassandra solr solrcloud kafka consul zookeeper apache-drill dockerhub docker rabbitmq-cluster nagios-plugins spark presto rabbitmq linux kubernetes devops hacktoberfest
Language:Shell 1228

hadoop

donnemartin / data-science-ipython-notebooks

spotify / luigi

Tencent / APIJSON

prestodb / presto

heibaiying / BigData-Notes

apache / hadoop

deeplearning4j / deeplearning4j

apache / doris

trinodb / trino

wangzhiwubigdata / God-Of-BigData

linkedin / school-of-sre

h2oai / h2o-3

Alluxio / alluxio

apache / hive

apache / ignite

apache / calcite

tomwhite / hadoop-book

WeBankFinTech / DataSphereStudio

apache / nutch

LuckyZXL2016 / Movie_Recommend

MoRan1607 / BigDataGuide

HariSekhon / DevOps-Bash-tools

geekyouth / SZT-bigdata

big-data-europe / docker-hadoop

dahuoyzs / javapdf

apache / kyuubi

apache / drill

cdarlint / winutils

gchq / Gaffer

Qihoo360 / XLearning

moosefs / moosefs

will-che / BigData-Interview

apache / carbondata

collabH / bigdata-growth

DTStack / Taier

HariSekhon / Dockerfiles