silentbugs / hon-client-scraper

Messy python script to quickly fetch hon client versions

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

[Feature request] Sleep then retry on TimeoutError

mrhappyasthma opened this issue · comments

Whenever downloading a large version (e.g. x86_64 v4.8.6), I see a handful of timeouts for some files. It may be nice to have the script automatically retry (maybe up to 3 times or some reasonable amount?) whenever a timeout occurs.

('Connection aborted.', TimeoutError(10060, 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond', None, 10060, None))

Approximately how many timeouts did you get for this version? I have added the skipping part for this reason in order to re-download any files that timed out previously, but I'd consider retrying on timeout.

Did you notice if the file started downloading and timed out during download, or did the initial request for the file timeout? On the former case we have a big issue and we probably need to figure out a way to checksum the files in order to choose whether we need to re-download or not.

It seems resolved for me when applying this PR: #8

Fixed with the merge of PR #8