Hangs
schollii opened this issue · comments
When I run the example (Ubuntu 18, python 3.6) it just hangs forever. Whenever I ctrl-c the process, it shows the same output:
me:~ $ python3 -m aws_list_all query --service ec2 --region us-east-1
^CTraceback (most recent call last):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 746, in next
item = self._items.popleft()
IndexError: pop from an empty deque
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.6/dist-packages/aws_list_all/__main__.py", line 109, in <module>
main()
File "/usr/local/lib/python3.6/dist-packages/aws_list_all/__main__.py", line 87, in main
do_query(services, args.region, args.operation)
File "/usr/local/lib/python3.6/dist-packages/aws_list_all/query.py", line 91, in do_query
for result in ThreadPool(32).imap_unordered(acquire_listing, to_run):
File "/usr/lib/python3.6/multiprocessing/pool.py", line 750, in next
self._cond.wait(timeout)
File "/usr/lib/python3.6/threading.py", line 295, in wait
waiter.acquire()
KeyboardInterrupt
me:~ $
Direct aws CLI commands like aws s3 ls
work fine.
+1
Hmm. Unfortunately, I cannot reproduce the issue here.
"IndexError: pop from an empty deque" is just an issue with Ctrl-C'ing into the multiprocessing.
I've now added extra verbosity to the execution, and if you add "-vvv" to the run it should print out the full traceback of any exceptions. That would very much help in finding any bugs!
I have the same problem using python 3.6.7, but it worked with 3.7.3, it seems a bug in python multiprocessing, workaround is here: felixhuttmann@81e45eb