Cadair / parfive

An asyncio based parallel file downloader for Python 3.8+

Home Page:https://parfive.readthedocs.io/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Queue Full Error with 1.2.0rc1

Cadair opened this issue · comments

../../.tox/py38-online/lib/python3.8/site-packages/sunpy/net/fido_factory.py:369: in fetch
    results = downloader.download()
../../.tox/py38-online/lib/python3.8/site-packages/parfive/downloader.py:273: in download
    return self._run_in_loop(self.run_download(timeouts))
../../.tox/py38-online/lib/python3.8/site-packages/parfive/downloader.py:193: in _run_in_loop
    return asyncio.run(coro)
/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/asyncio/runners.py:43: in run
    return loop.run_until_complete(main)
/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
../../.tox/py38-online/lib/python3.8/site-packages/parfive/downloader.py:226: in run_download
    done.update(await self._run_http_download(main_pb, timeouts))
../../.tox/py38-online/lib/python3.8/site-packages/parfive/downloader.py:324: in _run_http_download
    self.http_tokens.generate_queue(maxsize=self.max_conn),
../../.tox/py38-online/lib/python3.8/site-packages/parfive/utils.py:171: in generate_queue
    queue.put_nowait(item)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <Queue at 0x7fd43f044160 maxsize=4 _queue=[<parfive.utils.Token object at 0x7fd43efaff70>n = 1, <parfive.utils.Token o...= 2, <parfive.utils.Token object at 0x7fd43efafbe0>n = 3, <parfive.utils.Token object at 0x7fd43efafa30>n = 4] tasks=4>
item = <parfive.utils.Token object at 0x7fd43efafd00>n = 5

    def put_nowait(self, item):
        """Put an item into the queue without blocking.
    
        If no free slot is immediately available, raise QueueFull.
        """
        if self.full():
>           raise QueueFull
E           asyncio.queues.QueueFull

/opt/hostedtoolcache/Python/3.8.5/x64/lib/python3.8/asyncio/queues.py:148: QueueFull

https://dev.azure.com/sunpy/sunpy/_build/results?buildId=9500&view=logs&j=a7b3aa55-7d57-562f-3433-7f6b2d4252da&t=c806977a-3b27-51ee-41ed-46d1d189e419&l=960

@dstansby I am not entirely sure I understand how your PR introduced this. 😕

We should add a test tho.

Don't suppose there's a self contained example for this? Taking a look now