dashbitco / broadway

Concurrent and multi-stage data ingestion and data processing with Elixir

Home Page:https://elixir-broadway.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Telemetry events for producers?

samullen opened this issue · comments

To get a better idea of how many processors I should be running against the producers in our various Broadway apps, I wanted to compare durations between them (producers:processors). Unfortunately, I don't see producer events listed in either the documentation or the code. Is this an oversight in the Broadway library, or is the onus on the "off-Broadway" Producer libraries such as BroadwaySQS? If it's the latter, do those libraries need to follow a naming convention for events?

Each producer is too distinct for us to provide a template. For example, pulling vs push, prefetch vs ack-based, etc. So you should request your producers for specific events. They should probably be namespaced under the producer name. Pull requests will likely be welcome. :)