mkuthan / stream-processing

Learn how to develop and test stateful streaming and batch data pipelines

Home Page:http://mkuthan.github.io/tags/#stream-processing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Learn How To Develop And Test Stateful Streaming Data Pipelines

CI codecov

Shared modules:

  • stream-processing-shared - shared utilities for developing stateful streaming data pipelines
  • stream-processing-infrastructure - infrastructure layer with IOs for BigQuery, Pubsub and Cloud Storage
  • stream-processing-test - shared utilities for testing stateful streaming data pipelines

Use cases:

  • toll-application, toll-domain, toll-infrastructure - sample application for toll data processing, see blog post
  • word-count - fixed window example, see blog post
  • session-window - session window example, see blog post

About

Learn how to develop and test stateful streaming and batch data pipelines

http://mkuthan.github.io/tags/#stream-processing

License:Apache License 2.0


Languages

Language:Scala 95.4%Language:HCL 3.3%Language:Shell 1.3%