streaming-data

There are 29 repositories under streaming-data topic.

oxnr / awesome-bigdata
A curated list of awesome big data frameworks, ressources and other awesomeness.
awesome awesome-list bigdata data data-analytics data-science data-stream data-visualization data-warehouse database distributed-database series-database stream-processing streaming-data visualize-data
13004
kafka-ui
provectus / kafka-ui
Open-Source Web UI for Apache Kafka Management
kafka-ui kafka-brokers kafka kafka-streams kafka-client opensource kafka-connect kafka-producer streams big-data apache-kafka cluster-management web-ui kafka-manager kafka-cluster streaming-data event-streaming hacktoberfest
Language:Java 9093
miller
johnkerl / miller
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
command-line command-line-tools csv csv-format data-cleaning data-processing data-reduction data-regression devops devops-tools json json-data miller statistical-analysis statistics streaming-algorithms streaming-data tabular-data tsv unix-toolkit
Language:Go 8726
connect
redpanda-data / connect
Fancy stream processing made operationally mundane
amqp cqrs data-engineering data-ops etl event-sourcing go golang kafka logs message-bus message-queue nats rabbitmq stream-processing stream-processor streaming-data
Language:Go 8007
materialize
MaterializeInc / materialize
The data warehouse for operational workloads.
data-warehouse database distributed-systems kafka materialized-view operational-data-warehouse postgresql postgresql-dialect rust sql stream-processing streaming streaming-data
Language:Rust 5676
river
online-ml / river
🌊 Online machine learning in Python
concept-drift data-science incremental-learning machine-learning online-learning online-machine-learning online-statistics python real-time-processing stream-processing streaming streaming-data
Language:Python 4898
readysettech / readyset
Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.
backend cache caching caching-proxy databases mysql mysql-database postgres postgresql postgresql-database rust rust-lang sql streaming-data
Language:Rust 4068
piskvorky / smart_open
Utils for streaming large files (S3, HDFS, gzip, bz2...)
python s3 hdfs webhdfs boto streaming file streaming-data gzip-stream bz2 hacktoberfest
Language:Python 3125
fluvio
infinyon / fluvio
Lean and mean distributed stream processing system written in rust and web assembly.
cloud-native data-flow data-integration data-pipelines distributed-systems event-driven-architecture real-time rust serverless stateful stream-processing stream-processing-engine streaming streaming-data streaming-data-pipelines streaming-data-processing webassembly
Language:Rust 2798
memgraph
memgraph / memgraph
Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.
cypher graph graph-algorithms graph-analysis graph-database kafka kafka-streams nosql opencypher stream-processing streaming-data
Language:C++ 2244
pravega / pravega
Pravega - Streaming as a new software defined storage primitive
streaming streaming-data distributed-storage real-time-data data-ingestion
Language:Java 1980
go-streams
reugn / go-streams
A lightweight stream processing library for Go
stream-processing pipeline etl kafka data-stream kafka-streams streams redis pulsar data-pipeline streaming-data stream-processor aerospike websocket nats-streaming streaming-api windowing throttling low-code workflow
Language:Go 1805
bytewax / bytewax
Python Stream Processing
data-engineering data-processing data-science dataflow machine-learning python rust stream-processing streaming-data
Language:Python 1398
microsoft / Trill
Trill is a single-node query processor for temporal or streaming data.
streaming-data temporal-data
Language:C# 1241
python-streamz / streamz
Real-time stream processing for python
async python real-time streaming-data
Language:Python 1231
quix-streams
quixio / quix-streams
100% Python stream processing with Streaming DataFrames
data-engineering data-intensive-applications data-science event-driven-architecture kafka machine-learning python real-time-data-processing stream-processing stream-processor streaming-data streaming-data-pipelines streaming-data-processing time-series-data
Language:Python 1030
zpl-c / zpl
📐 Pushing the boundaries of simplicity
zpl helper header-only c cpp json5-parser cross-platform math tar threading thread-pool coroutines cli hashing memory-allocation memory-management timer time streaming-data csv-parser
Language:C 967
OnlineStats.jl
joshday / OnlineStats.jl
⚡ Single-pass algorithms for statistics
julia online-algorithms statistics big-data onlinestats stochastic-approximation streaming-data julia-language julialang
Language:Julia 828
scikit-multiflow / scikit-multiflow
A machine learning package for streaming data in Python. The other ancestor of River.
scikit scikit-learn stream streaming-data machine-learning meka moa
Language:Python 751
hstreamdb / hstream
HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications.
streaming database iot stream-processing haskell real-time sql hstreamdb scale distributed-systems distributed-database materialized-view kafka financial-analysis realtime-database iot-database data-processing streaming-data streaming-database
Language:Haskell 699
infoslack / awesome-kafka
A list about Apache Kafka
kafka streaming-data data-pipeline stream-processing apache-kafka apache-spark kafka-streams infrastructure data-processing
572
streamdal / streamdal
Code-Native Data Privacy
data-contracts deno docker event-driven go nodejs observability python rust streaming-data tail-f message-queues astrojs reactjs javascript wasi wasm
Language:TypeScript 563
Stratio / sparta
Real Time Analytics and Data Pipelines based on Spark Streaming
streaming-data scala stratio spark streaming spark-streaming olap kafka hdfs workflow sparta analytics real-time sparksql stratio-sparta lambda triggers
Language:Scala 524
kLabUM / rrcf
🌲 Implementation of the Robust Random Cut Forest algorithm for anomaly detection on streams
outliers detect-outliers tree random-forest machine-learning anomaly-detection streaming-data python robust-random-cut-forest
Language:Python 488
swimos / swim
Full stack application platform for building stateful microservices, streaming APIs, and real-time UIs
stateful distributed-systems asynchronous-programming non-blocking-io websockets streaming-data actor-model microservices-architecture streaming-api decentralized-applications serverless serverless-framework real-time web-agent
Language:Java 477
guillermo-navas-palencia / optbinning
Optimal binning: monotonic binning with constraints. Support batch & stream optimal binning. Scorecard modelling and counterfactual explanations.
python binning woe optimization woebinning stream streaming-data batch-processing scorecard credit-scoring mdlp counterfactual-explanations
Language:Python 435
kafka-ui
kafbat / kafka-ui
Open-Source Web UI for managing Apache Kafka clusters
apache-kafka big-data cluster-management event-streaming hacktoberfest kafka kafka-brokers kafka-client kafka-cluster kafka-connect kafka-manager kafka-producer kafka-streams kafka-ui opensource streaming-data streams web-ui foss
Language:Java 417
radiantly / you-cant-download-this-image
Downloading images from the web is as easy as right clicking them and selecting "Save image as..", right? Well, not anymore xD
rust http streaming-data
Language:C++ 341
cloudflow
lightbend / cloudflow
Cloudflow enables users to quickly develop, orchestrate, and operate distributed streaming applications on Kubernetes.
cloudflow streaming-applications kubernetes streaming-data akka flink spark streaming-runtimes microservices-architectures
Language:Scala 322
microsoft / data-accelerator
Data Accelerator for Apache Spark simplifies onboarding to Streaming of Big Data. It offers a rich, easy to use experience to help with creation, editing and management of Spark jobs on Azure HDInsights or Databricks while enabling the full power of the Spark engine.
spark spark-streaming spark-sql sparksql streaming-data streaming servicefabric nodejs docker hdinsight cosmosdb react azure apache-spark iothub eventhub big-data iot kafka kafka-streams
Language:C# 294
Chulong-Li / Real-time-Sentiment-Tracking-on-Twitter-for-Brand-Improvement-and-Trend-Recognition
A real-time interactive web app based on data pipelines using streaming Twitter data, automated sentiment analysis, and MySQL&PostgreSQL database (Deployed on Heroku)
data-analysis streaming-data twitter-sentiment-analysis brand-improvement stream-processing tweets heroku-server topic-tracking dashboard twitter dash plotly
Language:Jupyter Notebook 276
bbejeck / kafka-streams-in-action
Source code for the Kafka Streams in Action Book
kafka kafkastreams streaming stream-processing streaming-data
Language:Java 260
goodboy / tractor
A distributed, structured concurrent runtime for Python (and friends)
actor-model rpc distributed-systems multiprocessing multicore-programming trio async-await structured-concurrency streaming-data
Language:Python 258
keithknott26 / datadash
Visualize and graph data in the terminal
terminal-based terminal-ui graph graphing graphing-application tabular-data csv tsv streaming-data chart charting go golang
Language:Go 254
selimfirat / pysad
Streaming Anomaly Detection Framework in Python (Outlier Detection for Streaming Data)
anomaly-detection outlier-detection incremental-learning streaming-data anomaly python machine-learning outliers unsupervised-learning real-time intrusion-detection fraud-detection
Language:Python 225
Western-OC2-Lab / PWPAE-Concept-Drift-Detection-and-Adaptation
Data stream analytics: Implement online learning methods to address concept drift and model drift in data streams using the River library. Code for the paper entitled "PWPAE: An Ensemble Framework for Concept Drift Adaptation in IoT Data Streams" published in IEEE GlobeCom 2021.
machine-learning anomaly-detection iot data-stream drift online-learning real-time-analytics concept-drift change-detector intrusion-detection-system cicids2017 drift-detection iot-data-analytics lightgbm adaptive-learning ensemble-learning streaming-data
Language:Jupyter Notebook 198