0-byte !tmp files from downloading Sankaku
Romangelo opened this issue · comments
Using Full Batch Mode, downloading from Sankaku will produce a lot of 0-byte .!tmp files in the download folder. It's like the app tried to download images but failed, then the images were left in temp state.
If I look into the log file and go to the URLs for those images, they're fully accessible.
I notice a lot of issue when it try to download the full image from https://v.sankakucomplex.com host. I try to use the browser and it just keep aborted by the server.
So it's an issue on Sankaku's side? Right now a lot of images have been skipped by the downloader.
We can do nothing unless Sankaku fix their problems, right?
So it's an issue on Sankaku's side? Right now a lot of images have been skipped by the downloader.
We can do nothing unless Sankaku fix their problems, right?
I can confirm some of the Sankaku file server is down couple of days ago for around a week or so.
@Nandaka also a note, keeping the tmp file is not all bad idea for everyone, as for me i take it as quick lookup of failed downloads and will check on it manually. atm I need to browse through thousands of lines of logs to find out missed downloads.
Recently sankaku has many files just failed to download, had to manually pull from source or keep a log of failed files and wait for some time to try again.
@Nandaka also a note, keeping the tmp file is not all bad idea for everyone, as for me i take it as quick lookup of failed downloads and will check on it manually. atm I need to browse through thousands of lines of logs to find out missed downloads.
Recently sankaku has many files just failed to download, had to manually pull from source or keep a log of failed files and wait for some time to try again.
Right now, my solution is setting the retry count to 20. By doing this, most of the files are being downloaded successfully. It may take a lot longer to download a batch when Sankaku is semi-down but the error rates are low and I barely have to manually download any. If there's a few errors that went beyond 20 retries, it'd be very very few and I can just search for "20 of 20" in the log files.
@Nandaka also a note, keeping the tmp file is not all bad idea for everyone, as for me i take it as quick lookup of failed downloads and will check on it manually. atm I need to browse through thousands of lines of logs to find out missed downloads.
Recently sankaku has many files just failed to download, had to manually pull from source or keep a log of failed files and wait for some time to try again.Right now, my solution is setting the retry count to 20. By doing this, most of the files are being downloaded successfully. It may take a lot longer to download a batch when Sankaku is semi-down but the error rates are low and I barely have to manually download any. If there's a few errors that went beyond 20 retries, it'd be very very few and I can just search for "20 of 20" in the log files.
yup same here, with notepad++ for better performance.
retry count 20 is very slow if have to deal with the infamous error: (429) Please slow down. What is your delay time? Does set it lower than 3min cause any issue? (e.g. spamming thr server)
@Nandaka also a note, keeping the tmp file is not all bad idea for everyone, as for me i take it as quick lookup of failed downloads and will check on it manually. atm I need to browse through thousands of lines of logs to find out missed downloads.
Recently sankaku has many files just failed to download, had to manually pull from source or keep a log of failed files and wait for some time to try again.Right now, my solution is setting the retry count to 20. By doing this, most of the files are being downloaded successfully. It may take a lot longer to download a batch when Sankaku is semi-down but the error rates are low and I barely have to manually download any. If there's a few errors that went beyond 20 retries, it'd be very very few and I can just search for "20 of 20" in the log files.
yup same here, with notepad++ for better performance. retry count 20 is very slow if have to deal with the infamous error: (429) Please slow down. What is your delay time? Does set it lower than 3min cause any issue? (e.g. spamming thr server)
I set it to 120s. Sometimes I tried 90 but it didn't seem to work well. Idk... Sankaku will probably put your IP on the blacklist longer if you keep spamming too soon, so 120s seems to be the best for me.
@Nandaka also a note, keeping the tmp file is not all bad idea for everyone, as for me i take it as quick lookup of failed downloads and will check on it manually. atm I need to browse through thousands of lines of logs to find out missed downloads.
Recently sankaku has many files just failed to download, had to manually pull from source or keep a log of failed files and wait for some time to try again.Right now, my solution is setting the retry count to 20. By doing this, most of the files are being downloaded successfully. It may take a lot longer to download a batch when Sankaku is semi-down but the error rates are low and I barely have to manually download any. If there's a few errors that went beyond 20 retries, it'd be very very few and I can just search for "20 of 20" in the log files.
yup same here, with notepad++ for better performance. retry count 20 is very slow if have to deal with the infamous error: (429) Please slow down. What is your delay time? Does set it lower than 3min cause any issue? (e.g. spamming thr server)
I set it to 120s. Sometimes I tried 90 but it didn't seem to work well. Idk... Sankaku will probably put your IP on the blacklist longer if you keep spamming too soon, so 120s seems to be the best for me.
hmm about the same, i set 180s and 10 retries.
haha, got so many (504) Gateway Timeout and (502) Bad Gateway that it is a guranteed 20 retries. Hope there is an alternet setting to sepreate download error types and wait time. e.g. if error code in 1st set of delay timer then else 2nd set of delay timer.
welp, for now just leave it overnight.
(429) Please slow down
120s for me
Hope there is an alternet setting to sepreate download error types and wait time.
If I have time, but Pull Request is also appreciated :D