A lot of threads are created in Docker and /tmp/ files aren't get removed themself
dmazurok opened this issue · comments
Describe the bug
A clear and concise description of what the bug is.
I use a docker, I tried several images, including Debian 10, Ubuntu 20.04.
Whenever I call phonemizer (I use espeak-ng backend), it creates a tmp file in /tmp, and that file is never being removed. I gotta do it manually. e.g. /tmp/tmpmm4wvh2a/
Second and more serious issue - it creates 5 thread per call. These threads are called python3 if I call in just in python, or a script name where I call it.
These threads do not take place when I use phonemizer not in docker in the same OS, so they may get removed by OS. But, the files are still exist.
Here's what lsof shows on one of the threds:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python3 2918 root cwd DIR 0,315 4096 357361 /opt/tts/core
python3 2918 root rtd DIR 0,315 4096 360129 /
python3 2918 root txt REG 0,315 5494584 193103 /usr/bin/python3.8
python3 2918 root mem REG 8,64 193103 /usr/bin/python3.8 (path dev=0,315)
python3 2918 root mem REG 8,64 377749 /tmp/tmp41tqi6z1/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 377740 /tmp/tmp7zi8wqgs/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 377735 /tmp/tmp8vm012_s/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 360303 /tmp/tmpui4t1me4/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 377734 /tmp/tmp9zx0ki47/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 377732 /tmp/tmpqxs_x1jl/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 377730 /tmp/tmp802s1e9n/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 377728 /tmp/tmptpnxq0zk/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 377726 /tmp/tmpcq6gwn70/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 377719 /tmp/tmpv6xc13jn/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 377707 /tmp/tmpyxyz0lwq/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 377695 /tmp/tmp7cdn55gh/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 377683 /tmp/tmpbhwa9dyo/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 377671 /tmp/tmpvkal1687/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 377659 /tmp/tmpfi80wx96/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 377648 /tmp/tmpwxwkegtl/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 377646 /tmp/tmpm8uwl_ut/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 377635 /tmp/tmphj8h5yze/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 377279 /tmp/tmp6asia1xb/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 360519 /tmp/tmpmm4wvh2a/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 360507 /tmp/tmpihws9if5/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 360489 /tmp/tmp3359d4en/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 360481 /tmp/tmpybm7adk4/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 360474 /tmp/tmp0s4sdb_n/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 360468 /tmp/tmpc6cg042a/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 360466 /tmp/tmpfuef0oji/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 360464 /tmp/tmpbbxg5bgy/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 193634 /usr/lib/python3.8/lib-dynload/mmap.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 174088 /usr/lib/x86_64-linux-gnu/libuuid.so.1.3.0 (path dev=0,315)
python3 2918 root mem REG 8,64 193621 /usr/lib/python3.8/lib-dynload/_multiprocessing.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417965 /usr/local/lib/python3.8/dist-packages/numpy/random/_generator.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417954 /usr/local/lib/python3.8/dist-packages/numpy/random/_sfc64.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417952 /usr/local/lib/python3.8/dist-packages/numpy/random/_pcg64.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417951 /usr/local/lib/python3.8/dist-packages/numpy/random/_philox.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417961 /usr/local/lib/python3.8/dist-packages/numpy/random/_mt19937.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417962 /usr/local/lib/python3.8/dist-packages/numpy/random/_bounded_integers.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417945 /usr/local/lib/python3.8/dist-packages/numpy/random/_common.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417958 /usr/local/lib/python3.8/dist-packages/numpy/random/bit_generator.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417950 /usr/local/lib/python3.8/dist-packages/numpy/random/mtrand.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417388 /usr/local/lib/python3.8/dist-packages/numpy/fft/_pocketfft_internal.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 418033 /usr/local/lib/python3.8/dist-packages/numpy/linalg/_umath_linalg.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 173980 /usr/lib/x86_64-linux-gnu/libffi.so.7.1.0 (path dev=0,315)
python3 2918 root mem REG 8,64 193611 /usr/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root mem REG 8,64 417759 /usr/local/lib/python3.8/dist-packages/numpy/core/_multiarray_tests.cpython-38-x86_64-linux-gnu.so (path dev=0,315)
python3 2918 root DEL REG 8,64 360461 /tmp/tmphd32b0wg/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 360459 /tmp/tmpjp0ubv4j/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 360457 /tmp/tmplroa0vey/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 360455 /tmp/tmpy3igbxq2/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 360453 /tmp/tmpxolo2lvl/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 360451 /tmp/tmpcm9do6_y/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 360449 /tmp/tmp5ghf0t49/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 360447 /tmp/tmp7g2optrf/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 360445 /tmp/tmpdsuq507r/libespeak-ng.so.1.1.49
python3 2918 root mem REG 8,64 360443 /tmp/tmps_pqu5ct/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root mem REG 8,64 360441 /tmp/tmp_vivj8pn/libespeak-ng.so.1.1.49 (path dev=0,315)
python3 2918 root DEL REG 8,64 360439 /tmp/tmpslgwsv6b/libespeak-ng.so.1.1.49
python3 2918 root DEL REG 8,64 360437 /tmp/tmpcts29en3/libespeak-ng.so.1.1.49
and I see the threads in top -H only, as top shows only 1 process.
here's part of top -H output when I called phonemizer multiple times in script p.py:
3357 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.30 python3 p.py
3385 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3387 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3389 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3391 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3394 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3396 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3398 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3400 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3403 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3405 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3407 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3409 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3412 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3414 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3416 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3418 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3421 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3423 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3425 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3427 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3430 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3432 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3434 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3436 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3439 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3441 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3443 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3445 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3448 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3450 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3452 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3454 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3457 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3459 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3461 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3463 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3466 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3468 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3470 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3472 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3475 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3477 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3479 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3481 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3484 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3486 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
3488 root 20 0 4772812 739324 185004 S 0.0 4.5 0:00.00 python3 p.py
Phonemizer version
v3.2.1
System
Debian 10, Ubuntu 20.04
To reproduce
phonemize("hello", separator=seperator, strip=False, njobs=1, backend='espeak', language="en", preserve_punctuation=False, language_switch='remove-flags', punctuation_marks=';:,.!?—-', preserve_empty_lines=False)
Expected behavior
The threads that are created by phonemizer (related to espeak most as it seems) must be killed in Docker as well.
The files in /tmp that are created by phonemizer must get removed automatically.
Additional context
Add any other context about the problem here.