Almost freezes with extreme RAM and CPU spikes on trying to download embedded images
AlexBocken opened this issue · comments
Avoid duplicates
- Bug is not already reported in another issue
Describe the bug
When downloading linked files is enabled and an image is embedded into a page using data:image/png;base64 moodle-dl seems to attempt to download this file. This results in huge RAM Spikes (in my case +7GB) and CPU usage (100% usage) while also seemingly having to rescan all files?
You can see a screenshot of such a culprit below:
moodle-dl get's stuck on a page that looks like this. You can see that the image is embedded on the right.
Steps to reproduce the issue
Requires HTML to have embedded images.
- Enable downloading linked files
- Attempt a download
Terminal Protocol: Kind of confused what a terminal protocol is. Running zsh in st if that helps.
You can find a partial log file attached, I exited moodle after a while as it would take hours to finish.
MoodleDownloader.log
The url is in the the form of <moodle-url>/mod/page/view.php?id=<id> if that helps
Technical details
OS: Arch Linux 6.1.12
Moodle-DL Version: Most recent -git AUR version: (v2.2.2.4.r1.g9f65e57-1)
Shell: zsh
Terminal: st (personal build)
Expected behavior
Either 1. extract these images into a separate file or 2. skip them.
Possible Fix
Simply skip these files? Otherwise an explicit check for embedded files and a subsequent extraction into a separate file could prove to be a quick fix maybe.
Can you send me the whole embeded image tag (via mail or discord or here) so I can test it with my local moodle?
👌 Check your email inbox
Feel free to open a new issue, if the issue persists in the new release.
Had to delete my sql db but now everything works smoothly. Thanks for the quick work!