spark-rdd

There are 0 repository under spark-rdd topic.

• mahmoudparsian / pyspark-tutorial

  PySpark-Tutorial provides basic algorithms using PySpark

  big-databig-data-analyticsdata-algorithmsdataframespysparkpyspark-sqlpyspark-tutorialranking-functionsrddssparkspark-dataframesspark-rdd
  Language:Jupyter Notebook 1259

• mahmoudparsian / big-data-mapreduce-course

  Big Data Modeling, MapReduce, Spark, PySpark @ Santa Clara University

  pyspark-algorithms-bookmapreducesanta-clara-universitypysparkdata-algorithmsdata-transformationdata-partitionpartitioning-algorithmsalgorithmsmapreduce-pythonmapreduce-algorithmapache-hadoopapache-sparkbig-datadata-analysisdata-engineeringglossarymonoidspark-dataframesspark-rdd
  Language:HTML 161

• Thomas-George-T / Movies-Analytics-in-Spark-and-Scala

  Data cleaning, pre-processing, and Analytics on a million movies using Spark and Scala.

  sparkspark-sqlrddshell-scriptbig-datadataframesmovielens-data-analysisbig-data-analyticsbig-data-projectsspark-programsscalamovielens-datasetanalyticscase-studyspark-dataframesspark-rddhadoopspark-scalamovies
  Language:Scala 94

• yennanliu / spark-etl-pipeline

  Various data stream/batch process demo with Apache Scala Spark 🚀

  sparkdockerspark-streamingspark-batchspark-sqlspark-rddspark-dataframesscaladockerfilepipelinetwittersbtsbt-pluginsbt-assemblystream-processingapache-spark
  Language:Scala 11

• Ren294 / Log-Analysis-Project

  This project builds a scalable log analytics pipeline use Lambda architecture for real-time and batch processing of NASA server logs.

  apache-kafkaapache-nifiapache-sparkbig-databig-data-analyticscassandracassandra-driverdata-engineeringdata-sciencegrafanahadoophadoop-hdfshivepowerbispark-rddspark-sqlspark-streaming
  Language:Python 8

• nipunmanral / Community-Detection-In-Graphs

  Implementation of Girvan-Newman Algorithm to detect communities in graphs using Yelp dataset

  data-mininggirvan-newmancommunity-detectionbetweennesssocial-graphspark-rddmap-reduceyelp-datasetbreadth-first-search
  Language:Python 5

• MaxineXiong / Item-based-collaborative-filtering

  This project utilizes PySpark DataFrames and PySpark RDD to implement item-based collaborative filtering. By calculating cosine similarity scores or identifying movies with the highest number of shared viewers, the system recommends 10 similar movies for a given target movie that aligns users’ preferences.

  apache-sparkcollaborative-filteringmovie-recommendationpysparkpythonsparkspark-dataframesspark-rdd
  Language:Jupyter Notebook 4

• adityajn105 / Apache-Spark-Tutorials

  Apache spark is a big data analysis framework.

  pysparkspark-mlbigdatasparkspark-tutorialsspark-rdd
  Language:Jupyter Notebook 2

• MaxineXiong / Degrees-of-Separation-with-Breadth-first-Search

  This project utilizes PySpark RDD and the Breadth-first Search (BFS) algorithm to find the shortest path and degrees of separation between two given Marvel superheroes based on based on their appearances together in the same comic books, empowering users to discover connections between their favourite superheroes in the Marvel universe.

  apache-sparkpysparkpythonspark-rddsparkbfs-algorithmbreadth-first-searchdegrees-of-separationmarvel-characters
  Language:Jupyter Notebook 2

• manojpawar94 / Spark-Scala-Examples

  I have implemented the sample programs using apache spark. The programs have developed on the concepts of Spark RDD and Spark SQL Dataframe.

  apache-sparksparkspark-rddspark-sql
  Language:Scala 1

• mohammad-safari / spark-hadoop-exercise

  spark hadoop exercise of cloud computing course - aut 1402-1403 fall

  big-datahadoophadoop-yarnhdfsmapreducesparkspark-dataframesspark-rddspark-sql
  Language:Jupyter Notebook 1

• nikhilkumawat03 / Extracting-Relevant-Document

  Projects contains based on Big Data

  hadoopmapreducespark-sqlspark-rddjava-8
  Language:Java 1

• ricardoariasalazar / Flights-Delay

  In this project, we use Spark to visualize, manipulate, model and stream historical flight-delays data using Spark RDD, Spark SQL and Kafka

  pysparkspark-sqlspark-rddbig-data-analyticskafka-streams
  Language:Jupyter Notebook 1

• ShreeshaN / SparkBigDataTutorials

  Demonstration of basic data transformations using Spark RDD and Spark DataFrame in Scala

  sparkspark-sqlspark-sql-udfspark-rddspark-scalascala-sbt
  Language:Scala 1

• smusab9152 / Pyspark_programs_and_projects

  Collection of PySpark programs and projects demonstrating the use of Apache Spark's Python API for big data processing and analysis. It includes practical implementations such as logistic regression classification, data analysis on the Iris dataset, and basic PySpark operations like temperature conversion.

  apache-sparkbig-databig-data-analyticsdata-engineeringdistributed-computingetlpysparkspark-dataframesspark-rddspark-sql
  Language:Jupyter Notebook 1

• BhargaviNadendla / Twitter-Analysis-on-BigData

  apache-sparkspark-rddspark-sqltweepytwitter-streaming-api
  Language:Roff 0

• demanejar / spark-rdd

  Spark RDD basic

  projectsparkspark-rdd
  Language:Java 0

• firedent / Data-curation-and-indexing-with-ElasticSearch

  This program will process legal report via Stanford CoreNLP and index them in ElasticSearch

  elasticsearchjsonscalaspark-rddxml
  Language:Scala 0
• Big-Data-Analytics

  madhurimarawat / Big-Data-Analytics

  This repository demonstrates big data processing, visualization, and machine learning using tools such as Hadoop, Spark, Kafka, and Python.

  big-databig-data-analyticsbig-data-analytics-techniqueshadoop-hdfshadoop-installationhadoop-mapreducepythonapache-kafkaapache-sparkdata-preprocessing-and-cleaningdata-visualizationhadoop-hivehiveqlspark-graphxspark-mllibspark-mllib-libraryspark-rddspark-streamingdata-stratification
  Language:Jupyter Notebook 0

• on2e / ntua-atdb

  Advanced Topics in Databases course project - NTUA ECE - 2022-23

  advanced-databaseapache-hadoopapache-sparkntua-ecepysparkspark-dataframesspark-rdd
  Language:Python 0

• RiccardoRevalor / Spark

  Spark exercises

  pysparksparkspark-rddspark-sql
  Language:Jupyter Notebook 0

• alimiheb / ChicagoEnergyUsage-SparkAnalysis

  The Chicago Energy Usage Analysis project aims to explore energy consumption patterns in Chicago using big data techniques. Leveraging Apache Spark, it processes a dataset of 67051 records to provide actionable insights for urban planning and energy efficiency initiatives.

  apache-sparkbig-datachicago-data-portaldata-analysisintellijjavamavenspark-rddspark-sql
  Language:Java

• contactsunny / spring-spark-s3-file-read

  A POC written in Java using the Spring framework, which uses Apache Spark to read a file from Amazon S3 FS and counts the number of lines in the file.

  sparkapache-sparkspark-s3rdd-s3spark-rdd-s3spark-rddjavaspring-bootspringthetechcheckpoc
  Language:Java

• Maybrittnelson / spark-parquet-thrift-example

  Example Spark project using Parquet as a columnar store with Thrift objects.

  mavenparquetspark-rddthrift
  Language:Scala

• melissakou / Notes-Big-Data-Essentials

  A notes for Coursera Course: Big Data Essentials - HDFS, MapReduce and Spark RDD

  coursera-coursehdfsmapreducereading-notesspark-rdd

• vaibhav50596 / DeerfootTrailAnalysis

  The goal is to train a linear regression model to predict Deerfoot commute times given weather and accident conditions using Spark RDD and MLlib

  sparkspark-mllibspark-rdd
  Language:Jupyter Notebook