Client reconnect on CBORDecodingError creating infinite loop reading frames
joelghill opened this issue · comments
I'm reading from the firehouse using the synchronous client and I am encountering an error when processing a frame. A CBORDecodingError is raised and the logic when handling this error appears to be to reconnect to the feed. The client continues to read from the feed from the initial cursor and shortly after encounters the same unprocessable frame.
The cursor that the error occurs at is just after 93278360
I'm not sure what the solution should be. Should the client reconnect just after the problematic cursor and continue reading the feed, or is there a deeper issue in processing the data that needs to be resolved here?
What a good catch! Thank you! Yeah, it requires deep research into CBOR decoding that is provided by 3d part lib, but for now, let's ignore frames that could not be decoded. The fix will be included in the next release (v0.0.19)
@joelghill released. pls verify pip install -U atproto
Looks like it's working, thank you!
all issues are coming from a library that I chose as WS client... i migrated to stable one and all issues are gone! #101 i'll merge it soon
@joelghill let me know if you can install this dev build with the new WS lib to your feed and run it for a few hours. Nothing changed for public API. Only my internals of firehose client impl
I'll take a look sometime this evening!
Still playing around with it on my end, but so far I haven't seen any exceptions and it's spinning up far fewer threads in the background. Seems much more stable!
Also , the fan on my laptop is much quieter, so that's also a good sign, haha!
I also expect that it manages threads much better. But I didn't notice a big performance boost or something like that. That's cool that now we can peacefully decode cbor without frame looses
Thank you for your tests! I'll include this PR in the next release
I stay connected to the firehose for 7 hours without any errors. Looks soo stable
released in v0.0.20