stream-processing

There are 56 repositories under stream-processing topic.

• google / mediapipe

  Cross-platform, customizable ML solutions for live and streaming media.

  mediapipec-plus-pluscomputer-visiondeep-learningandroidvideo-processingaudio-processingmobile-developmentmachine-learninginferencegraph-frameworkgraph-basedcalculatorframeworkpipeline-frameworkstream-processingperception
  Language:C++ 21014
• flink-learning

  zhisheng17 / flink-learning

  flink learning blog. http://www.54tianzhisheng.cn/ 含 Flink 入门、概念、原理、实战、性能调优、源码解析等内容。涉及 Flink Connector、Metrics、Library、DataStream API、Table API & SQL 等内容的学习案例，还有 Flink 落地应用的大型项目案例（PVUV、日志存储、百亿数据实时去重、监控告警）分享。欢迎大家支持我的专栏《大数据实时计算引擎 Flink 实战与性能优化》

  flinkkafkaelasticsearchsparkredismysqlrocketmqhbaserabbitmqstream-processingstreamingclickhouselokiinfluxdbopentsdb
  Language:Java 13286
• vector

  vectordotdev / vector

  A high-performance observability data pipeline.

  routerlogsmetricsrustobservabilityforwardervectorparsereventsstream-processingpipeline
  Language:Rust 13036

• newTendermint / awesome-bigdata

  A curated list of awesome big data frameworks, ressources and other awesomeness.

  awesomebigdatadata-warehousedata-analyticsdata-visualizationdata-streamdatabasedata-sciencedataawesome-listdistributed-databaseseries-databasestream-processingstreaming-datavisualize-data
  11810

• madd86 / awesome-system-design

  A curated list of awesome System Design (A.K.A. Distributed Systems) resources.

  distributed-systemshadoop-ecosysteminterviewmessage-brokermicroservicesmicroservices-architecturenosqlrelational-databasestream-processing
  6649

• robinhood / faust

  Python Stream Processing

  asynciodistributed-systemskafkakafka-streamspythonstream-processing
  Language:Python 6521
• benthos

  benthosdev / benthos

  Fancy stream processing made operationally mundane

  message-queuestream-processingstreaming-datamessage-buslogsstream-processorcqrsevent-sourcinggogolangkafkaamqprabbitmqnatsetldata-engineeringdata-ops
  Language:Go 5860
• watermill

  ThreeDotsLabs / watermill

  Building event-driven applications the easy way in Go.

  watermillevent-drivenevent-sourcingsagaseventsstream-processingreactivegogolangrabbitmqkafkacqrsnats
  Language:Go 5451

• confluentinc / ksql

  The database purpose-built for stream processing applications.

  stream-processingreal-timesqlinteractivekafkakafka-connectmaterialized-viewsksqldbksqldb-documentationksqldb-tutorialsstreaming-queriesevent-streaming-database
  Language:Java 5447

• hazelcast / hazelcast

  Open-source distributed computation and storage platform. Real-time Stream Processing Unconference. Save Your Spot https://hazelcast.com/lp/unconference/

  javahazelcastin-memorybig-datascalabilitydistributedcachinghacktoberfeststream-processinglow-latencydistributed-computingdistributed-systems
  Language:Java 5288
• materialize

  MaterializeInc / materialize

  Materialize is a fast, distributed SQL database built on streaming internals.

  databasedistributed-systemskafkamaterialized-viewpostgresqlpostgresql-dialectrustsqlstream-processingstreaming
  Language:Rust 4919

• fluent / fluent-bit

  Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows

  fluentdcloggingdata-collectorfluent-bitforwardercloudnativestream-processingsql-queriesmetricstraceslogsopentelemetryprometheus
  Language:C 4409
• river

  online-ml / river

  🌊 Online machine learning in Python

  incremental-learningmachine-learningpythononline-learningonline-statisticsdata-sciencestreamingonline-machine-learningstreaming-dataconcept-driftreal-time-processingstream-processing
  Language:Python 4150
• risingwave

  risingwavelabs / risingwave

  RisingWave: A Distributed SQL Database for Stream Processing

  databasestream-processingcloud-nativesqldistributed-databaserustserverlesspostgresql
  Language:Rust 4136
• danfojs

  javascriptdata / danfojs

  Danfo.js is an open source, JavaScript library providing high performance, intuitive, and easy to use data structures for manipulating and processing structured data.

  data-analyticsdata-sciencepandasdata-manipulationtensorsdataframejavascriptdata-analysisdanfojsstream-datastream-processingtensorflowtableplotting-charts
  Language:TypeScript 4127

• apache / hudi

  Upserts, Deletes And Incremental Processing on Big Data.

  hudiapachehudidatalakebigdataapachesparkincremental-processingstream-processingdata-integrationapacheflink
  Language:Java 4050

• douban / dpark

  Python clone of Spark, a MapReduce alike framework in Python

  bigdatadparkmapreducepythonsparkstream-processing
  Language:Python 2694

• pipelinedb / pipelinedb

  High-performance time-series aggregation for PostgreSQL

  realtimestream-processinganalyticssqlpushpostgresqlpipelinedbtime-seriesaggregation
  Language:C 2547

• manuzhang / awesome-streaming

  a curated list of awesome streaming frameworks, applications, etc

  awesome-liststream-processingawesomelist
  2224

• francoispqt / gojay

  high performance JSON encoder/decoder with stream API for Golang

  gogolangdecoderencoderperfomancejsonstream-decoderstream-processing
  Language:Go 2066

• nerevu / riko

  A Python stream processing engine modeled after Yahoo! Pipes

  stream-processingetlparallelismrssdataasynchronouslibraryclifunctional-programmingfeatured
  Language:Python 1594

• WallarooLabs / wally

  Distributed Stream Processing

  stream-processingstream-processorstream-processing-enginepony-languagepythonlinuxgolanggoframeworkapi
  Language:Pony 1476

• siddhi-io / siddhi

  Stream Processing and Complex Event Processing Engine

  streaming-data-processingcepstream-processingcomplex-event-processingonline-learningcloud-nativecloudlibrarycncf
  Language:Java 1426
• yomo

  yomorun / yomo

  🦖 Serverless Streaming Framework for low-latency geo-distributed system and realtime collaboration application, running atop QUIC protocol, as Metaverse infrastructure, engaging 5G technology.

  5gdistributed-cloudedge-aiedge-computingedge-meshfunctional-reactive-programminggeo-distributedgeodistributedsystemsiotlow-latencymetaversemetaverse-infrastructurenetworkingquicrealtimerealtime-collaboserverlessstream-processing
  Language:Go 1397
• go-streams

  reugn / go-streams

  A lightweight stream processing library for Go

  stream-processingpipelineetlkafkadata-streamkafka-streamsstreamsredispulsardata-pipelinestreaming-datastream-processoraerospikewebsocketnats-streamingstreaming-apiwindowingthrottlinglow-codeworkflow
  Language:Go 1369
• memgraph

  memgraph / memgraph

  Open-source graph database, built for real-time streaming data, compatible with Neo4j.

  graph-algorithmsgraph-databasegraph-analysisstream-processingstreaming-datakafka-streamskafkacyphergraphnosqlopencypher
  Language:C++ 1230
• hazelcast-jet

  hazelcast / hazelcast-jet

  Distributed Stream and Batch Processing

  javabig-datastream-processingbatch-processingevent-processinglow-latencykafkacdchacktoberfest
  Language:Java 1049

• edp963 / wormhole

  Wormhole is a SPaaS (Stream Processing as a Service) Platform

  spark-streamingstream-processingwormhole
  Language:JavaScript 970

• spring-cloud / spring-cloud-dataflow

  A microservices-based Streaming and Batch data processing in Cloud Foundry and Kubernetes

  microservices-architectureorchestrationstream-processingbatch-processingpredictive-analyticscloud-nativedatapipelines
  Language:Java 968

• clj-commons / manifold

  A compatibility layer for event-driven abstractions

  stream-processingfutureclojureasynchacktoberfeststream
  Language:Clojure 960

• spring-cloud / spring-cloud-stream

  Framework for building Event-Driven Microservices

  microservicesmessagingevent-driven-microservicesstream-processingreactivecloud-native
  Language:Java 884

• halaxa / json-machine

  Efficient, easy-to-use, and fast PHP JSON stream parser

  phpstream-processingjson-parserjson-iteratorjson-streamparsing
  Language:PHP 834

• nodefluent / kafka-streams

  equivalent to kafka-streams :octopus: for nodejs :sparkles::turtle::rocket::sparkles:

  kafkastreamsstream-processingkafka-streamsnodenodejsbig-data
  Language:TypeScript 801

• vladimirvivien / automi

  A stream processing API for Go (alpha)

  data-streamgogolangstream-processingstreaming-api
  Language:Go 782

• uhop / stream-json

  The micro-library of Node.js stream components for creating custom JSON processing pipelines with a minimal memory footprint. It can parse JSON files far exceeding available memory streaming individual primitives using a SAX-inspired API.

  streaming-jsonjavascript-objectsparse-json-filesparserstream-componentsstream-processing
  Language:JavaScript 770

• gearpump / gearpump

  Lightweight real-time big data streaming engine over Akka

  akkastream-processingbigdatascala
  Language:Scala 765