Infinite loop in case of ProvisionedThroughputExceededException from Kinesis
thiago-sylvain opened this issue · comments
Hi, thanks for your amazing package!
We use it as a consumer for Kinesis.
We use the suggested configuration with multiconsumer:
multi = MultiConsumer(people_consumer, places_consumer)
multi.run()
At some point, our Kinesis service raises a ProvisionedThroughputExceededException
, multiple times a row, and it ends up with the _get_records
method of the kinesis based consumer returning None.
This further leads to a StopIteration but coupled with the indefinite consumer, it ends up in an infinite loop.
For the moment we overrode the _get_records
to generate a specific exception and crash for good, but there must be a cleaner solution to avoid entering this infinite loop.
Thank you very much for your advice!