Restarting Alert Stream on Connection Loss or Timeout
shaibt opened this issue · comments
Hi,
Was having a lot of problems with reconnecting cameras' alert streams after network disconnects or camera restarts and started looking into the watchdog mechanism.
Problem is the current implementation of the watchdog mechanism:
- Will only reset the connection if the stream iterator is still active, meaning the camera is still sending some data over the connection but not parsed as Video Loss KA message
- OR, will reset the connection if the
for line in stream.iter_lines():loop is broken - which from experience never happens on connection loss or break. Or might happen after a very long time.
It would be preferable to use the python requests timeout param such as:
stream = self.hik_request.get(url, stream=True, timeout=60.0)
and then capture the resulting exception:
except requests.exceptions.ConnectionError as err:
The nice thing about timeout is that its also a read timeout - for stream requests, the number of seconds that the client will wait between bytes sent from the server. See more here
(also see this stack overflow question)
I've been testing the above change in my fork and it looks to be more robust than the watchdog (which I think is now redundant).
Thanks for digging into this. I only recently realized (thanks to a recent PR) that the timeout being set on the requests session object is doing nothing. With that knowledge the reconnect issues certainly make a lot more sense.
I see in your fork you added the timeout parameter to the init function, do you have a use case for this? I'd rather just set constants for connection and read timeout (5 and 60) and pass them into the appropriate functions as needed. I think these values are fairly safe and it's unlikely they would need tweaked higher.
In my use case I have different sensitivities to camera reconnect times so I initially added to init() so will be configurable but on second thought its not that important and setting constants is good enough
Suggestions have been implemented. I'm leaving the watchdog for now, but may remove it later. As you've mentioned with the timeouts properly in place it really shouldn't be needed.