_FAILED_ folders not being deleted properly?

Question

_FAILED_ folders not being deleted properly?

engels74 opened this issue 3 months ago · comments

engels74 commented 3 months ago

SABnzbd version

4.2.3 [e7e06de]

Operating system

Ubuntu 22.04.4 LTS (VM through UnraidOS)

Using Docker image

hotio

Description

Hello 👍

I used SABnzbd for many years, so first of I just wanted to say a massive thank you for maintaining and improving this app over the years ❤️ . Very grateful.

My setup is the usual "arr" setup. Adding series to Sonarr -> Prowlarr searches, returns downloads to Sonarr -> SABnzbd downloads -> Sonarr imports -> Available on my personal Media Server. Sonarr is set to both "Remove Completed" and "Remove Failed" in the SABnzbd settings.

Anyways, I've had an odd experience today. I had a massive list of episodes from a single TV series, that was in my SABnzbd queue. All the downloads failed, I'm guessing either it was taken down from Usenet or some unpacking error.

But after the downloads failed, and Sonarr had removed the failed downloads from its history, all of the folders were kept in the "completed" download directory. In my case, and since it's Sonarr, it's /data/mount/stage/.usenet/completed/sonarr. Like 100GBs of FAILED folders, with two files inside: One .mkv file of 0 bytes, and a .zip archive of 5-10GB. All of the folders had one .mkv and a multiple-GB .zip file in there.

My cleanup list in SABnzbd doesn't contain "zip", as I'm not sure if that would affect the unpacking process?

Current SABnzbd settings:
I redacted the parts from my sabnzbd.ini file, and it's available here:
https://logs.notifiarr.com/?21193d7bd72458d1=#F4BEedxx582BMY3mCRAoqbWRc859yA8tUDKPGkEn16PE

If that doesn't work out, here's some screenshots of each of my SABnzbd settings pages:

The rest I haven't changed, so not attaching screenshots of that

Docker Compose (hotio):

name: download-tools
services:
  sabnzbd:
    container_name: sabnzbd
    image: ghcr.io/hotio/sabnzbd
    environment:
      - PUID=1000
      - PGID=1000
      - UMASK=002
      - TZ=Europe/Berlin
      - TP_HOTIO=true
      - TP_THEME=dark
    volumes:
      - /opt/docker-all/downloadTools/sabnzbd:/config
      - /data:/data
      - /data/scripts/themepark/98-themepark-sabnzbd:/etc/cont-init.d/98-themepark
    restart: unless-stopped

Logs of the affected series: (look for My.Favorite.Series or MY FAVORITE SERIES)
Link: (huge file)
https://logs.notifiarr.com/?6c6ec2d15047da7d#FcmZ7E4ggB3JrNEAZLficEztAsykCaUrK7hR8cPdGNxC

It says this:

2024-04-22 02:50:42,899::INFO::[nzbstuff:1806] [N/A] Purging data for job My.Favorite.Series.S07E10.Multi.1080p.BluRay.DTS-HDMA (delete_all_data=False)

for each folder, but all of the _FAILED_ folders were still there, including the large .zip file and the empty .mkv file.
It was specifically all of the My.Favorite.Series.SXXEXX.Multi.1080p.BluRay.DTS-HDMA that was "left behind", or just not deleted / removed completely.

Question:
What have I done wrong? I haven't really experienced it ever quite like that, having hundreds of GB being left behind in the _FAILED_ folders. Should I set up my "Cleanup List" to include .zip files, or is it somewhere else it fails? It usually deletes _FAILED_ folders just fine, with maybe a few weird ones once in a while, not being deleted.

thezoggy · Answer 1 · Tue Apr 23 2024 03:58:25 GMT+0800 (China Standard Time)

remove completed/failed from those apps is mainly about removing it from the history db (or now archiving it from history on sab 4.3.x+).

sab does not remove files for jobs in the complete folder (regardless of status) and this is by design.

engels74 · Answer 2 · Tue Apr 23 2024 04:31:25 GMT+0800 (China Standard Time)

I see. Is there any way to achieve this? E.g. adding zip to the cleanup list? Or a simple bash script looking for _FAILED_ folders or something? :)

Or is it something that Sonarr would have to do / add as a feature?

Sander · Answer 3 · Tue Apr 23 2024 04:44:12 GMT+0800 (China Standard Time)

How about deleting all dirs older than 3 days, and run it daily from crontab?

find /path/to/Downloads/incomplete/* -type d -ctime +3 -exec rm -rf {} \;
Note: not tested

engels74 · Answer 4 · Tue Apr 23 2024 05:06:42 GMT+0800 (China Standard Time)

Yeah, I'll create some simple bash script for it. I was just wondering whether SABnzbd was supposed to delete them or not.

thezoggy · Answer 5 · Tue Apr 23 2024 06:39:38 GMT+0800 (China Standard Time)

also just so you are aware, you can set your logging level to DEBUG to show a lot more about what sab is doing. click the wrench on home page and set logging level there from INFO to DEBUG.

Safihre · Answer 6 · Tue Apr 23 2024 14:58:06 GMT+0800 (China Standard Time)

We should investigate why it resulted in __FAILED__, because this only happens in unexpected conditions. Otherwise it would have cleaned up.
Can you maybe set indeed the logging to Debug?
Then, run one of the episodes again. Check what is the error in the logs, why it can't unpack those zip files. Maybe it's password protected? Because it sounds like either protected or a scam download, with the strange mkv of 0 bytes and the zip file.
In any case, the History should have also told you that.

engels74 · Answer 7 · Tue Apr 23 2024 16:45:13 GMT+0800 (China Standard Time)

I'll see if I can reproduce it later with debugging enabled. Then I'll reopen the issue, once I've gathered logs and so on.

engels74 · Answer 8 · Wed Apr 24 2024 01:30:26 GMT+0800 (China Standard Time)

Debug log
It's massive:
https://logs.notifiarr.com/?bbac0ba687e4e607=#DK5XZJBbggN6ryuuyKDdYtgXMfUX39pmTdSDtjnrqnuX

Should be reasonable sanitized for IPs and API keys.

Look like a wrong pass of most of them, and then the S05E01 has:

Error "Incomplete sequence of joinable files" while running file_join on My.Favorite.Series.S05E01.Multi.1080p.BluRay.DTS-HDMA

Other information
The tree output of the /complete/sonarr folder:

user@server:/data/mount/stage/.usenet/complete/sonarr$ tree
.
|-- _FAILED_My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA
|   |-- My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA.mkv
|   `-- a8f307577225da392697d2d32960e12d.zip
|-- _FAILED_My.Favorite.Series.S05E03.Multi.1080p.BluRay.DTS-HDMA
|   |-- My.Favorite.Series.S05E03.Multi.1080p.BluRay.DTS-HDMA.mkv
|   `-- a9148f16417489106aa099b95cd4c5da.zip
`-- _FAILED_My.Favorite.Series.S05E04.Multi.1080p.BluRay.DTS-HDMA
    |-- 43d93c4ed4fb7e42a4fa7f9691a40f25.zip
    `-- My.Favorite.Series.S05E04.Multi.1080p.BluRay.DTS-HDMA.mkv

The output of ls -alh from some of the folders:

user@server:/data/mount/stage/.usenet/complete/sonarr$ cd _FAILED_My.Favorite.Series.S05E01.Multi.1080p.BluRay.DTS-HDMA/
user@server:/data/mount/stage/.usenet/complete/sonarr/_FAILED_My.Favorite.Series.S05E01.Multi.1080p.BluRay.DTS-HDMA$ ls -alh
total 9.9G
drwxrwxr-x 1 user user  148 Apr 23 19:16 .
drwxrwxrwx 1 user user  472 Apr 23 19:16 ..
-rw-rw-r-- 1 user user 9.7G Apr 23 19:16 45358d5a6b201f6d29a34f946a614aa1.zip
-rw-rw-r-- 1 user user 280M Apr 23 19:16 45358d5a6b201f6d29a34f946a614aa1.zip.1
user@server:/data/mount/stage/.usenet/complete/sonarr/_FAILED_My.Favorite.Series.S05E01.Multi.1080p.BluRay.DTS-HDMA$ cd ..
user@server:/data/mount/stage/.usenet/complete/sonarr$ tree
.
|-- _FAILED_My.Favorite.Series.S05E01.Multi.1080p.BluRay.DTS-HDMA
|   |-- 45358d5a6b201f6d29a34f946a614aa1.zip
|   `-- 45358d5a6b201f6d29a34f946a614aa1.zip.1
|-- _FAILED_My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA
|   |-- My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA.mkv
|   `-- a8f307577225da392697d2d32960e12d.zip
|-- _FAILED_My.Favorite.Series.S05E03.Multi.1080p.BluRay.DTS-HDMA
|   |-- My.Favorite.Series.S05E03.Multi.1080p.BluRay.DTS-HDMA.mkv
|   `-- a9148f16417489106aa099b95cd4c5da.zip
`-- _FAILED_My.Favorite.Series.S05E04.Multi.1080p.BluRay.DTS-HDMA
    |-- 43d93c4ed4fb7e42a4fa7f9691a40f25.zip
    `-- My.Favorite.Series.S05E04.Multi.1080p.BluRay.DTS-HDMA.mkv

4 directories, 8 files
user@server:/data/mount/stage/.usenet/complete/sonarr$ cd _FAILED_My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA/
user@server:/data/mount/stage/.usenet/complete/sonarr/_FAILED_My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA$ ls -alh
total 9.8G
drwxrwxr-x 1 user user  182 Apr 23 19:11 .
drwxrwxrwx 1 user user  472 Apr 23 19:16 ..
-rw-rw-r-- 1 user user    0 Mar 19  2022 My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA.mkv
-rw-rw-r-- 1 user user 9.8G Apr 23 19:11 a8f307577225da392697d2d32960e12d.zip
user@server:/data/mount/stage/.usenet/complete/sonarr/_FAILED_My.Favorite.Series.S05E02.Multi.1080p.BluRay.DTS-HDMA$

thezoggy · Answer 9 · Wed Apr 24 2024 03:07:42 GMT+0800 (China Standard Time)

What 7zip do you have install.. and version

7z --help

p7zip? p7zip-full? Or one of the many forks?

engels74 · Answer 10 · Wed Apr 24 2024 03:52:53 GMT+0800 (China Standard Time)

I have the one used in hotio's container I guess 👍

user@server:~$ docker exec sabnzbd 7z --help

7-Zip (z) 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20
 64-bit locale=en_US.UTF-8 Threads:22 OPEN_MAX:1048576

Sander · Answer 11 · Wed Apr 24 2024 03:58:42 GMT+0800 (China Standard Time)

Sidestep: can you manually unzip a8f307577225da392697d2d32960e12d.zip ? It asks for a password, so do you have that (for example in the NZB, or from your indexer)?

If there is no password, I guess that's an explanation for failing?

engels74 · Answer 12 · Wed Apr 24 2024 05:02:12 GMT+0800 (China Standard Time)

I don't have the password no, it was just grabbed from an NZB indexer

albino1 · Answer 13 · Wed Apr 24 2024 08:22:42 GMT+0800 (China Standard Time)

The password is probably in the NZB. Open the NZB in a text editor and at the top look for something like:

<head>
   <meta type="password">123456</meta>
</head>

engels74 · Answer 14 · Thu Apr 25 2024 19:55:35 GMT+0800 (China Standard Time)

I'm not sure where this issue is heading, honestly 😆 . I'm not asking for help with the specific failed NZBs. Sonarr/the arr-stack will figure that out automatically, when it fails anyways.

I was merely wondering if it was intentional that SABnzbd wasn't deleting the _FAILED_ folders. And it's seemingly is intentional, that it doesn't. So I basically got my answer; but I'm not sure if there's another purpose development-wise? 😄

Safihre · Answer 15 · Sun Apr 28 2024 03:37:24 GMT+0800 (China Standard Time)

I wanted to check what is going wrong with the filejoin, but I can see that file 45358d5a6b201f6d29a34f946a614aa1.zip.058 occurs twice in the NZB. Resulting in both a 45358d5a6b201f6d29a34f946a614aa1.zip.058 and 45358d5a6b201f6d29a34f946a614aa1.zip.1.058.
Which is then seen as 2 separate sets that need joining, which of course fails.
We could ignore the second failure, but it's just a messy NZB.. so don't think I want to do that.