Cadair / parfive

An asyncio based parallel file downloader for Python 3.8+

Home Page:https://parfive.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Inform user if >0 downloads fail

dstansby opened this issue · comments

I was recently downloading a bunch of files (parfive is awesome!), and parfive stopped downloading after 12/40 files, and the output in my terminal was this:

Downloading 40 files
Files Downloaded:  30%|█████████████████████████████████████████████                                                                                                         | 12/40 [00:13<00:31,  1.14s/file]

I got very confused by this, and didn't understand why parfive was stopping - if I ran the download again on the same 40 files, it would just download 12 more that it hadn't downloaded before.

Eventually I thought to check if any download errors were present, and found lots of

"Waiting for ('220',) but got 421 [' There are too many connections from your internet address.'

which explains what happened, but it took me a while to get there. I think it would be helpful if parfive issued some sort of warning if any of the downloads failed, perhaps something like "At least one download failed, see Result.errors for a list of the download errors".

Seems like a good idea.

Were the files you were downloading http or ftp?

Seems like a good idea.

Were the files you were downloading http or ftp?

ftp

You should only have 5 open connections, so it's very concerning that you get to 12 downloads and then it throws you a "too many connections" error 🤔

I think it's trying to open 5, but only manages to open 4, and after 3 times opening 4 connections the server gets a bit annoyed.

Fixed in #76