CuriousCurmudgeon / messaging_with_limits_talk

Presentation initially given at StirTrek 2024 about GetThru's concurrent, multi-stage data processing pipeline written in Elixir.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Messaging With Limits: Concurrent, Multi-Stage Data Processing in the Real World

Our team at GetThru found ourselves with a scaling problem. Working with our existing infrastructure for sending text messages was becoming increasingly complex. Years of bewildering external requirements had led us to the point where the existing system was both incorrect and difficult to reason about. In this session, we'll go over a case study of our new concurrent, multi-stage data processing pipeline. Starting from the fundamentals, we'll see how our team efficiently handled our existing external requirements and built in flexibility for the future. Finally, we'll look at how we did all of this in our existing monolith, avoiding the complexity of microservices.

Presentation History

Stir Trek 2024

2024-05-03
https://stirtrek.com/speakers/2024/Brian-Meeker.html

About

Presentation initially given at StirTrek 2024 about GetThru's concurrent, multi-stage data processing pipeline written in Elixir.