failed to execute youtube-dl: signal: killed

Question

failed to execute youtube-dl: signal: killed

op69qs opened this issue 4 years ago · comments

it's run at arm platform, It will fail when the video is too long.

ffmpeg version 3.2.14-1~deb9u1 Copyright (c) 2000-2019 the FFmpeg developers built with gcc 6.3.0 (Debian 6.3.0-18+deb9u1) 20170516 configuration: --prefix=/usr --extra-version='1~deb9u1' --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libebur128 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared libavutil 55. 34.101 / 55. 34.101 libavcodec 57. 64.101 / 57. 64.101 libavformat 57. 56.101 / 57. 56.101 libavdevice 57. 1.100 / 57. 1.100 libavfilter 6. 65.100 / 6. 65.100 libavresample 3. 1. 0 / 3. 1. 0 libswscale 4. 2.100 / 4. 2.100 libswresample 2. 3.100 / 2. 3.100 libpostproc 54. 1.100 / 54. 1.100

youtube-dl --version
2020.06.06
time="2020-06-18T16:44:29+08:00" level=error msg="youtube-dl error: /tmp/podsync-281075233/IwXWPP-xmMI.%(ext)s" error="failed to execute youtube-dl: signal: killed" time="2020-06-18T16:44:29+08:00" level=error msg="[youtube] IwXWPP-xmMI: Downloading webpage\n[download] Destination: /tmp/podsync-281075233/IwXWPP-xmMI.webm\n\r[download] 0.0% of 30.47MiB at 28.15KiB/s ETA 18:36\r[download] 0.0% of 30.47MiB at 78.45KiB/s ETA 06:39\r[download] 0.0% of 30.47MiB at 171.14KiB/s ETA 03:03\r[download] 0.0% of 30.47MiB at 342.58KiB/s ETA 01:31\r[download] 0.1% of 30.47MiB at 168.20KiB/s ETA 03:05\r[download] 0.2% of 30.47MiB at 215.95KiB/s ETA 02:24\r[download] 0.4% of 30.47MiB at 278.38KiB/s ETA 01:51\r[download] 0.8% of 30.47MiB at 428.23KiB/s ETA 01:12\r[download] 1.6% of 30.47MiB at 674.28KiB/s ETA 00:45\r[download] 3.3% of 30.47MiB at 1.08MiB/s ETA 00:27\r[download] 6.6% of 30.47MiB at 1.86MiB/s ETA 00:15\r[download] 13.1% of 30.47MiB at 2.17MiB/s ETA 00:12\r[download] 21.7% of 30.47MiB at 2.79MiB/s ETA 00:08\r[download] 32.1% of 30.47MiB at 2.88MiB/s ETA 00:07\r[download] 32.1% of 30.47MiB at 22.50KiB/s ETA 15:42\r[download] 32.1% of 30.47MiB at 66.43KiB/s ETA 05:19\r[download] 32.1% of 30.47MiB at 152.50KiB/s ETA 02:19\r[download] 32.2% of 30.47MiB at 321.41KiB/s ETA 01:05\r[download] 32.2% of 30.47MiB at 171.28KiB/s ETA 02:03\r[download] 32.3% of 30.47MiB at 199.03KiB/s ETA 01:46\r[download] 32.5% of 30.47MiB at 284.35KiB/s ETA 01:14\r[download] 32.9% of 30.47MiB at 425.56KiB/s ETA 00:49\r[download] 33.8% of 30.47MiB at 674.31KiB/s ETA 00:30\r[download] 35.4% of 30.47MiB at 1.09MiB/s ETA 00:18\r[download] 38.7% of 30.47MiB at 1.84MiB/s ETA 00:10\r[download] 45.2% of 30.47MiB at 2.19MiB/s ETA 00:07\r[download] 54.2% of 30.47MiB at 2.64MiB/s ETA 00:05\r[download] 64.0% of 30.47MiB at 2.93MiB/s ETA 00:03\r[download] 64.0% of 30.47MiB at 24.05KiB/s ETA 07:49\r[download] 64.0% of 30.47MiB at 67.90KiB/s ETA 02:46\r[download] 64.0% of 30.47MiB at 149.02KiB/s ETA 01:15\r[download] 64.0% of 30.47MiB at 301.63KiB/s ETA 00:37\r[download] 64.1% of 30.47MiB at 172.86KiB/s ETA 01:04\r[download] 64.2% of 30.47MiB at 214.11KiB/s ETA 00:52\r[download] 64.4% of 30.47MiB at 285.08KiB/s ETA 00:38\r[download] 64.8% of 30.47MiB at 436.31KiB/s ETA 00:25\r[download] 65.6% of 30.47MiB at 691.43KiB/s ETA 00:15\r[download] 67.3% of 30.47MiB at 1.11MiB/s ETA 00:08\r[download] 70.5% of 30.47MiB at 1.90MiB/s ETA 00:04\r[download] 77.1% of 30.47MiB at 2.26MiB/s ETA 00:03\r[download] 86.2% of 30.47MiB at 2.72MiB/s ETA 00:01\r[download] 95.2% of 30.47MiB at 2.94MiB/s ETA 00:00\r[download] 95.2% of 30.47MiB at 22.98KiB/s ETA 01:05\r[download] 95.2% of 30.47MiB at 67.85KiB/s ETA 00:22\r[download] 95.2% of 30.47MiB at 155.79KiB/s ETA 00:09\r[download] 95.2% of 30.47MiB at 328.38KiB/s ETA 00:04\r[download] 95.3% of 30.47MiB at 168.49KiB/s ETA 00:08\r[download] 95.4% of 30.47MiB at 219.78KiB/s ETA 00:06\r[download] 95.6% of 30.47MiB at 274.48KiB/s ETA 00:05\r[download] 96.0% of 30.47MiB at 425.19KiB/s ETA 00:02\r[download] 96.8% of 30.47MiB at 675.19KiB/s ETA 00:01\r[download] 98.5% of 30.47MiB at 1.10MiB/s ETA 00:00\r[download] 100.0% of 30.47MiB at 1.48MiB/s ETA 00:00\r[download] 100% of 30.47MiB in 00:18\n[ffmpeg] Destination: /tmp/podsync-281075233/IwXWPP-xmMI.mp3\n"

op69qs · Answer 1 · Fri Jun 19 2020 18:07:28 GMT+0800 (China Standard Time)

I find out the ffmpeg still running after youtube-dl killed .

Dewsmen · Answer 2 · Tue Jun 30 2020 02:21:52 GMT+0800 (China Standard Time)

Hi. I have the same issue on Raspberry Pi with Docker.

It looks like the issue is in DownloadTimeout = 10 * time.Minute (src).

In my case error timing is exectly 10 minutes (check the seconds):
time="2020-06-29T17:55:29Z" level=info msg="! downloading episode https://youtube.com/watch?v=XXXXXXX" episode_id=XXXXXXX index=0
time="2020-06-29T18:05:29Z" level=error msg="youtube-dl error: /tmp/podsync-078749168/XXXXXXX.%(ext)s" error="failed to execute youtube-dl: signal: killed"
or
time="2020-06-29T17:39:41Z" level=info msg="! downloading episode https://youtube.com/watch?v=YYYYYYY" episode_id=YYYYYYY index=0
time="2020-06-29T17:49:41Z" level=error msg="youtube-dl error: /tmp/podsync-233063467/YYYYYYY.%(ext)s" error="failed to execute youtube-dl: signal: killed"

If we could set it in config would be great.

op69qs · Answer 3 · Fri Jul 03 2020 10:40:19 GMT+0800 (China Standard Time)

I don't find this timeout config in go exec........

Dewsmen · Answer 4 · Wed Jul 08 2020 06:14:27 GMT+0800 (China Standard Time)

I built binary with 1h timeout (src)

const (
	DownloadTimeout = 60 * time.Minute
	UpdatePeriod    = 24 * time.Hour
)

and it fixed the issue:

time="2020-07-07T20:39:26Z" level=info msg="! downloading episode https://youtube.com/watch?v=XXXXXXX" episode_id=XXXXXXX index=0
time="2020-07-07T21:35:23Z" level=info msg="successfully downloaded file \"XXXXXXXX\"" episode_id=XXXXXXX index=0
time="2020-07-07T21:35:34Z" level=info msg="downloaded 1 episode(s)".

Also youtube-dl does 2 jobs:

Download
Extract audio (in my case it is audio, and that's why @op69qs saw ffmpeg running after the youtube-dl was killed - ffmpeg is not even it's child process)

but the timeout is common for both, so if you have slow internet(I'm now on 3G ~ 3-5Mbps) + weak CPU(Raspberry Pi Model B) the full process could take more then 10 minutes.

As a conclusion: looks like it makes sense to have an ability to extend this timeout via config and to use youtube-dl --socket-timeout (issue link) argument to restrict just download time.

Regards.

Dewsmen · Answer 5 · Wed Jul 08 2020 14:58:43 GMT+0800 (China Standard Time)

I don't find this timeout config in go exec........

@op69qs
It passes via Context:

...
	ctx, cancel := context.WithTimeout(ctx, DownloadTimeout)
	defer cancel()

	cmd := exec.CommandContext(ctx, dl.path, args...)
...

op69qs · Answer 6 · Wed Jul 08 2020 16:06:39 GMT+0800 (China Standard Time)

Thanks a log, How can I miss that........😂