spark-streaming-kafka

There are 2 repositories under spark-streaming-kafka topic.

• zqhxuyuan / kafka-book

  《Kafka技术内幕》代码

  kafkastreamsspark-streaming-kafkaflink-kafkastorm-kafkabeam-kafkaawesome-kafka
  Language:Java 194

• EthicalML / kafka-spark-streaming-zeppelin-docker

  One click deploy docker-compose with Kafka, Spark Streaming, Zeppelin UI and Monitoring (Grafana + Kafka Manager)

  kafkasparkstreamingzeppelindockerdocker-composekafka-spark-streamingspark-streaming-kafkaspark-kafkakafka-sparkkafka-zeppelinspark-zeppelin
  119

• tmcgrath / spark-scala

  Spark with Scala example projects

  sparkscalaspark-streaming-kafkaspark-cassandra
  Language:Scala 34

• artem0 / kafka-scala-api

  Samples for using Kafka within Spark Streaming and Akka Actors, Akka Streams

  kafkasparkspark-streamingakka-streamsscalaakka-actorsakkamessage-queuespark-streaming-kafkaspark-structured-streamingdockerdistributed-systems
  Language:Scala 9

• manasbundele / big-data-projects

  These are a select few projects related to Big Data Analytics and Management. The projects listed are a combination of both small and big projects but interesting ones.

  scalapythonsparkspark-streaming-kafkahadoop-mapreduce
  Language:HTML 9

• rafaelvp-db / databricks-end-to-end-streaming

  End-to-end Kafka Streaming Examples on Databricks with Evolving Avro Schemas.

  avrodatabrickskafkasparkspark-streamingspark-streaming-kafkastreaming
  Language:Scala 9

• arpendu11 / graph-based-data-lake

  An ETL application which is written in Quarkus, Spark SQL Streaming, Neo4j and various types of Databases and stores. It also covers the devops frameworks like Jenkins CI/CD, docker and Kubernetes.

  quarkus-mavenquarkus-resteasyquarkus-kafkaspark-sqlspark-streaming-kafkareactive-kafka-programmingquarkus-neo4jetl-applicationkubernetesdockerneo4j-graphneo4j-search
  Language:Java 7

• AbdullahMu / Data-Streaming-Nanodegree-Project_02-Evaluate-Human-Balance-with-Spark-Streaming

  Design data streaming architecture and API for a real-life application called the Step Trending Electronic Data Interface (STEDI). It is a working application used to assess fall risk for seniors. When a senior takes a test, they are scored using an index which reflects the likelihood of falling, and potentially sustaining an injury in the course of walking. STEDI uses a Redis datastore for risk score and other data. The Data Science team has completed a working graph for population risk at a STEDI clinic. The problem is the data is not populated yet. You will work with Kafka Connect Redis Source events and Business Events to create a Kafka topic containing anonymized risk scores of seniors in the clinic.

  data-streamingkafkaspark-streamingapache-sparkapache-kafkaapispark-streaming-kafkaredisbase64
  Language:Python 3

• cevatarmutlu / kafka_spark_streaming

  sparkspark-streamingspark-sqlspark-streaming-kafkapostgresqldocker-composepandas-dataframereal-time-analyticsflaskflask-sqlalchemyrecommendation-systembatch-processingturkish
  Language:Python 3

• dharaneeshvrd / spark-examples

  Spark Examples

  sparkspark-sqlspark-examplepysparkspark-streamingspark-structured-streamingspark-streaming-kafka
  Language:Python 2

• haozhang-x / log-analysis-spark

  Structured Streaming Log Analysis

  sparkspark-sqlspark-structured-streamingspark-streaming-kafkakafka
  Language:Scala 2

• okulkarni-weekendr / rtb-using-kafka

  streaming-datakafkaspark-streaming-kafkakafka-streams
  Language:Scala 2

• roksolana-d / spark-streaming-examples

  Research on legacy and structured streaming with Spark

  scalaspark-streamingspark-structured-streamingspark-streaming-kafkakafka
  Language:Scala 2

• chiayongjian / twitter-kafka-sparkstreaming

  A working example of Twitter -> Kafka -> Spark Streaming integration by a beginner

  aws-s3aws-emrsparkspark-streamingkafkaspark-streaming-kafkapythontweepykafka-pythonjupyter-notebookamazon-web-servicesbig-datatwittertwitter-api
  Language:Jupyter Notebook 1

• faizpuad / DataEngineeringProject-DocumentStreamingWithData

  The core objective of this project is to build an end-to-end data streaming pipeline that processes this dataset in real-time. By leveraging modern data engineering tools and techniques, we aim to connect, buffer, process, store, and visualize streaming data. This allows for better understanding of data flows, handling of large-scale real-time data

  data-engineeringdocument-streamingfastapikafkamonspark-streaming-kafkastreamlit
  Language:Jupyter Notebook 1

• froblesmartin / BachFinalProject

  Project to compare Apache Spark Streaming vs Apache Flink.

  spark-streamingflinkflink-stream-processingflink-kafkaspark-streaming-kafkabig-datastreamingbenchmarkbenchmarkingsparkapachekafka
  Language:Java 1

• ludengke95 / spark-streaming-kafka-template

  SparkStreaming新手友好向模板，简化SparkStreaming开发

  sparksparkstreamingspark-streaming-kafkatemplate
  Language:Java 1

• martinywwan / spark-kafka-streaming

  Near real-time streaming using Apache Spark and Apache Kafka

  kafkaapache-kafkaapache-sparkspark-streaming-kafkaspark-streamingscala
  Language:Scala 1

• pereldegla / twitter-trend-sentiment-analysis-world-cup-use-case

  How to get closer to the audience using Twitter: an use case following the France football team run during the 2022 World Cup

  databricks-notebooksfootballkafkapython3sparkspark-streamingspark-streaming-kafkadata-engineeringpipeline
  Language:Jupyter Notebook 1

• Ragadeepthi / Machine-Learning-on-Bigdata--Loading-Data-using-Kafka-and-Flume

  Playstore apps rating analysis - Machine Learning on Bigdata- Loading streaming Data using Kafka and Flume

  pysparkpyspark-notebookpyspark-activitysparkspark-streamingspark-sqlspark-structured-streamingspark-streaming-kafkaflumeflume-agentspark-flumesqlmysqlxml
  Language:Jupyter Notebook 1

• rajeshsantha / MonitoredStructuredStreaming

  Repository for Spark structured streaming use case implementations.

  spark-structured-streamingspark-streaming-kafkaspark-streamingscalaspark-dataframeskafkaapache-spark
  Language:Scala 1

• sergei-grigorev / spark-streaming-project

  In-Stream final project

  spark-streamingspark-sqlsparkfunspark-structured-streamingspark-streaming-kafka
  Language:Scala 1

• silverstone1903 / stream-101

  Intro to streaming data with Kafka, Spark and AWS Glue

  awsaws-gluespark-streamingspark-streaming-kafka
  Language:Python 1

• vchoudhari45 / spark-kafka-integration

  spark-kafka-integration

  spark-streaming-kafkaapache-kafkasparkkafka-consumerkafkaspark-kafka-integrationspark-kafka
  Language:Scala 1

• viyadb / viyadb-spark

  Data processing ang ingestion backend for ViyaDB based on Spark streaming

  sparkspark-streamingspark-streaming-kafkaviyadb
  Language:Scala 1

• daniel-pape / nyc-ml-app

  Streaming ML with NYC taxi data

  kafkascalasparkspark-streaming-kafka
  Language:Scala 0

• fadhilyori / kaspacore

  Mata Elang | Data Preprocessing using Scala and Spark

  scalasparkspark-streamingspark-streaming-kafkapostgresql
  Language:Scala 0

• hieuung / Streaming-Kafka

  Using various data processing tool for real time data pipeline with Kafka

  kafkakafka-consumerkafka-producerspark-streamingspark-streaming-kafkaapache-beamapache-flinkapache-spark
  Language:Python 0

• michelheil / BigData

  Projects related to Big Data technologies

  apache-hbaseapache-kafkaapache-sparkjavamqttscalaspark-streamingspark-streaming-kafkaspark-structured-streaming
  Language:Java 0

• monyedavid / substance-effects-on-reflexes

  substance effects on reflexes

  sparkspark-streaming-kafkaakka-httpscala
  Language:Scala 0

• trendyol-data-eng-summer-intern-2019 / recom-engine-streaming

  Streaming component of the project, which is written with Spark Streaming.

  sparkspark-streamingspark-streaming-kafkaspark-mlspark-mllibcollaborative-filteringspark-mongodb-connectorscalasbt-assembly
  Language:Scala 0

• abhay6694 / PySpark-Component

  Collection of spark-components functions for big-data processing

  pyspark-pythonbig-data-analyticsspark-streaming-kafkaspark-mllibkafka-connect
  Language:Jupyter Notebook

• chandreshsutariya / bigdata---train-analysis

  batch processing and realtime tains(railway) data analysis to help Station Masters refreshing each 20 seconds

  hadokafkapysparksparkspark-streaming-kafka
  Language:Python

• deepcloudlabs / dcl700-2021-jun-21

  DCL-700: Big Data Essentials

  big-datahadoophadoop-mapreducehdfshivesparkspark-streamingspark-streaming-kafkaspark-sql
  Language:JavaScript

• Faisal-AlDhuwayhi / Evaluate-Human-Balance-with-Spark-Streaming

  Design a data streaming pipeline around Apache Spark, Kafka, and Redis for a real-time application

  data-streamingkafkapysparkreal-timeredissparkspark-streaming-kafkaspark-structured-streamingstream-processing
  Language:Python

• rotemfogel / spark-streaming-app

  Spark Streaming Playground

  sparkspark-streamingspark-streaming-kafka
  Language:Scala