apachespark
There are 2 repositories under apachespark topic.
DataExpert-io / data-engineer-handbook
This is a repo with links to everything you'd ever want to learn about data engineering
- Language:Java 5129
- Language:Scala 98
- ApacheSpark
martandsingh / ApacheSpark
This repository will help you to learn about databricks concept with the help of examples. It will include all the important topics which we need in our real life experience as a data engineer. We will be using pyspark & sparksql for the development. At the end of the course we also cover few case studies.
Language:Python 82 funkyminds / cleanframes
type-class based data cleansing library for Apache Spark SQL
Language:Scala 81propelledanalytics / SparkSQL.jl
SparkSQL.jl enables Julia programs to work with Apache Spark data using just SQL.
Language:Julia 25josephmachado / docker_for_data_engineers
Code for blog at: https://www.startdataengineering.com/post/docker-for-de/
Language:C 22tspannhw / FLiPStackWeekly
FLaNK AI Weekly covering Apache NiFi, Apache Flink, Apache Kafka, Apache Spark, Apache Iceberg, Apache Ozone, Apache Pulsar, and more...
- Language:Jupyter Notebook 12
SandeepAswathnarayana / professional-certificate-programs
This repository contains all the projects and labs I worked on while pursuing professional certificate programs, specializations, and bootcamp. [Areas: Deep Learning, Machine Learning, Applied Data Science].
Language:Jupyter Notebook 10SmartDataAnalytics / MA-INF-4223-DBDA-Lab
Repository for Lab “Distributed Big Data Analytics” (MA-INF 4223), University of Bonn
Language:Jupyter Notebook 10datumbrain / gossub
Trigger spark-submit in Golang. A Go implementation of famous SparkLauncher.java.
Language:Go 7sfrechette / spark-jdbc-mssql
Connect to SQL Server using Apache Spark
Language:Scala 7CarolinaNicasio / APACHESPARK-PYSPARK-2023
PySpark es una biblioteca de procesamiento de datos distribuidos en Python que permite procesar grandes volúmenes de datos en clústeres utilizando el framework Apache Spark, ofreciendo un alto rendimiento y un conjunto de herramientas integradas para el análisis y manejo de datos a gran escala.
- Language:Java 6
- Language:Java 5
Examples usages for cleanframes library
Language:Scala 5sahith / Link-Prediction-for-Citation-Networks-using-Apache-Spark
Link Prediction is about predicting the future connections in a graph. In this project, Link Prediction is about predicting whether two authors will be collaborating for their future paper or not given the graph of authors who collaborated for atleast one paper together.
Language:Scala 5AbdelmajidLh / spark-functionality-repo
Ce dépôt GitHub contient un document détaillé sur les bases du langage Scala.
bartosz25 / data-ai-summit-2020
You will find here the demo codes for my Data+AI 2020 talk about customizing Apache Spark state store.
divithraju / divith-raju-Immigration-Data-Engineering
A Capstone Project that covers several aspects of Data Engineering (Data Exploration, Cleaning, Modeling, Pipelining, Processing)
Language:Jupyter Notebook 2mayankrawat / CSVJoin
Use this project to join data from multiple csv files. Currently in this project we support one to one and one to many join. Along with this you can find how to use kafka producer efficiently with spark.
Language:Java 2MRYingLEE / Apache_Spark_in_Google_Colab
This is a Jupyter Notebook to practice Apache Spark in Google Colab, especially for the exam CCA Spark and Hadoop Developer Exam (CCA175).
Language:Jupyter Notebook 2Arkaprabha-B / PySpark-GraphFrames
Implementation of GraphFrames using PySpark in Eclipse IDE
Cloudslab / dSpark
Source code for the work "dSpark: Deadline-Based Resource Allocation for Big Data Applications in Apache Spark" published in IEEE e-Science 2017
Language:Java 1sarathchandrikak / ETL-Bank-Transcation
Data Analysis of bank transaction data
Language:Jupyter Notebook 1syedsaadahmed / Hands-on-with-Apache-Spark
Working with Apache Spark, Creating some small tutorials and at last implemeting a small project
Thapep / ApacheSpark
Apache Spark project for Advanced Topics on Databases course
Language:Python 1ZeroTwoDataRW / DE-Stream-Project-Random-Generated-User-Data
An end-to-end data engineering pipeline that orchestrates data ingestion, processing, and storage using Apache Airflow, Python, Apache Kafka, Apache Zookeeper, Apache Spark, and Cassandra. All components are containerized with Docker for easy deployment and scalability.
Language:Python 1Az1m04 / Advance-Data-Science-with-IBM-Watson-Studio
This work on Python notebook .It shows how to calculate covariance and correlations using pyspark
Language:Jupyter Notebook 0mehrdadalmasi2020 / ApacheSpark_ApacheZeppelin_SQL_Shell
Run your first analysis project on Apache Zeppelin using Scala (Spark), Shell, and SQL
Language:Scala 0urvashiforreal / Retail-Data-Analysis
Developed a real-time streaming analytics pipeline using Apache Spark to calculate and store KPIs for e-commerce sales data, including total volume of sales, orders per minute, rate of return, and average transaction size. Used Spark Streaming to read data from Kafka, Spark SQL to calculate KPIs, and Spark DataFrame to write KPIs to JSON files.
Language:Python 0amit2014 / Advanced-Data-Science-with-IBM-Specialization
As a coursera certified specialization completer you will have a proven deep understanding on massive parallel data processing, data exploration and visualization, and advanced machine learning & deep learning. You'll understand the mathematical foundations behind all machine learning & deep learning algorithms. You can apply knowledge in practical use cases, justify architectural decisions, understand the characteristics of different algorithms, frameworks & technologies & how they impact model performance & scalability. If you choose to take this specialization and earn the Coursera specialization certificate, you will also earn an IBM digital badge. To find out more about IBM digital badges follow the link ibm.biz/badging.
The rapid pace of innovation in Artificial Intelligence (AI) is creating enormous opportunity for transforming entire industries and our very existence. After competing this comprehensive 6 course Professional Certificate, you will get a practical understanding of Machine Learning and Deep Learning. You will master fundamental concepts of Machine Learning and Deep Learning, including supervised and unsupervised learning. You will utilize popular Machine Learning and Deep Learning libraries such as SciPy, ScikitLearn, Keras, PyTorch, and Tensorflow applied to industry problems involving object recognition and Computer Vision, image and video processing, text analytics, Natural Language Processing, recommender systems, and other types of classifiers. You will be able to scale Machine Learning on Big Data using Apache Spark. You will build, train, and deploy different types of Deep Architectures, including Convolutional Networks, Recurrent Networks, and Autoencoders. By the end of this Professional Certificate, you will have completed several projects showcasing your proficiency in Machine Learning and Deep Learning, and become armed with skills for a career as an AI Engineer.
- Language:Jupyter Notebook
YFC-ophey / big-data-group-project
US superstore opening analysis
