real-time-data-pipeline

There are 0 repository under real-time-data-pipeline topic.

• kishlayjeet / Stock-Market-Real-Time-Data-Pipeline-with-Apache-Kafka-and-Cassandra

  A end-to-end real-time stock market data pipeline with Python, AWS EC2, Apache Kafka, and Cassandra Data is processed on AWS EC2 with Apache Kafka and stored in a local Cassandra database.

  apache-kafkacassandrakafkapipelinepythonstock-marketstock-market-data-pipelineawsaws-ec2data-engineeringdata-engineering-pipelinedata-pipelineec2ec2-instanceetletl-pipelinekafka-streamingkafka-streamsreal-timereal-time-data-pipeline
  Language:Python 17

• aymane-maghouti / Real-Time-Data-Pipeline-Using-Kafka

  This project implements a real-time data pipeline using Apache Kafka, Python's psutil library for metric collection, and SQL Server for data storage. The pipeline collects metrics data from the local computer, processes it through Kafka brokers, and loads it into a SQL Server database. Additionally, a real-time dashboard is created using Power BI.

  apache-kafkadata-collectiondata-streamingdata-visualizationpowerbipythonreal-timereal-time-data-pipeline
  Language:Python 3

• thecodemancer / Apache-Beam

  🔥👨‍💻 Build Big data pipelines with Apache Beam in any language and run it via Spark, Flink, GCP (Google Cloud Dataflow).

  apache-beamdata-engineeringdata-pipelinedata-pipelinespythonreal-time-data-pipelinestreaming-data-processing
  Language:Jupyter Notebook 1