Allow inputting an archive of nzb.gz files
Ravencentric opened this issue · comments
Description
Sab lets you upload an archive of .nzb files and it'll automatically add them all. This however doesn't work if you have an archive of .nzb.gz files.
This already exists, iirc it only will work if there only nzb files inside the gzip though
It does not. There are also no errors or feedback, just fails silently.
System:
Edition Windows 11 Pro
Version 23H2
Installed on 04-09-2023
OS build 22631.3085
Experience Windows Feature Experience Pack 1000.22684.1000.0
Sab:
Version: 4.2.2 [f730607]
Uptime: 15m
Config File: C:\Users\raven\AppData\Local\sabnzbd\sabnzbd.ini
Parameters: "C:\Program Files\SABnzbd\SABnzbd.exe"
Python Version: 3.12.1 (tags/v3.12.1:2305ca5, Dec 7 2023, 22:03:25) [MSC v.1937 64 bit (AMD64)] [cp1252]
OpenSSL: OpenSSL 3.0.11 19 Sep 2023
logs:
024-02-11 00:19:38,163::INFO::[nzbparser:87] Attempting to add Big Buck Bunny (.nzb.gz).zip
2024-02-11 00:19:38,171::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968824'}
2024-02-11 00:19:38,172::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968825'}
2024-02-11 00:19:39,049::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707589738129'}
2024-02-11 00:19:39,050::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707589738130'}
2024-02-11 00:19:39,186::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968827'}
2024-02-11 00:19:39,187::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968826'}
2024-02-11 00:19:40,207::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968828'}
2024-02-11 00:19:40,207::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968829'}
2024-02-11 00:19:41,035::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707589738132'}
2024-02-11 00:19:41,035::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707589738131'}
2024-02-11 00:19:41,221::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968830'}
2024-02-11 00:19:41,222::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968831'}
2024-02-11 00:19:41,370::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'status', 'skip_dashboard': '0', 'calculate_performance': '0', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968832'}
2024-02-11 00:19:41,375::DEBUG::[getipaddress:102] Local IPv4 address = <REMOVED>
2024-02-11 00:19:41,377::DEBUG::[happyeyeballs:136] Available addresses for self-test.sabnzbd.org (port=443, IPv4-only): 3 IPv4 and 0 IPv6
2024-02-11 00:19:41,426::DEBUG::[happyeyeballs:81] Happy Eyeballs connected to 104.26.2.117 (self-test.sabnzbd.org) in 35ms
2024-02-11 00:19:41,426::INFO::[happyeyeballs:169] Quickest IP address for self-test.sabnzbd.org (port=443, IPv4-only): 104.26.2.117 (self-test.sabnzbd.org)
2024-02-11 00:19:41,431::DEBUG::[happyeyeballs:81] Happy Eyeballs connected to 104.26.3.117 (self-test.sabnzbd.org) in 53ms
2024-02-11 00:19:41,442::DEBUG::[happyeyeballs:81] Happy Eyeballs connected to 172.67.69.176 (self-test.sabnzbd.org) in 35ms
2024-02-11 00:19:41,543::DEBUG::[getipaddress:145] Public address IPv4-only = <REMOVED>
2024-02-11 00:19:41,544::DEBUG::[getipaddress:166] Local IPv6 address = <REMOVED>
2024-02-11 00:19:41,545::DEBUG::[getipaddress:89] DNS Lookup = True (in 0.00 seconds)
2024-02-11 00:19:42,047::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707589738133'}
2024-02-11 00:19:42,048::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707589738134'}
2024-02-11 00:19:42,249::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'history', 'failed_only': '0', 'start': '0', 'limit': '10', 'last_history_update': '1', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968834'}
2024-02-11 00:19:42,249::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'queue', 'start': '0', 'limit': '20', 'output': 'json', 'REMOVED': '<REMOVED>', '_': '1707590968833'}
2024-02-11 00:19:43,169::DEBUG::[interface:145] Request GET /api from 127.0.0.1 [Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/121.0.0.0 Safari/537.36] {'mode': 'showlog', 'REMOVED': '<REMOVED>'}
Samples:
Big Buck Bunny (.nzb).zip
Big Buck Bunny (.nzb.gz).zip
you can add raw nzb or a nzb thats been archived,
adding your zip of nzb worked just fine
your zip of .nzb.gz doesnt as thats not supported... as you have nzb inside a .gz inside a .zip. thats to many levels deep with how we would expect things.
nzb.gz was supported, so I assumed this a zip containing only .nzb.gz files would be too. Can consideration be given to adding support for this??
curious why the ask, is there a website your doing that results in an nested archive or is it just a byproduct of some personal workflow?
to add some notes here,
per site they use .gz to store .nzbs on the backend to reduce storage, and then pass gzip header so single .nzb is served up. but for multiple nzb they get tossed in zip and served up, hence why there is a nzb in a .gz inside the .zip.
do we really want to support the nested nature..
It looks like more refactoring may required then I expected, it doesn't look too bad but I think we'd want to avoid repeating and process_nzb_archive_file already repeats a lot of the single path.
Maybe see if the indexer is open to constructing a normal zip first?
It looks like more refactoring may required then I expected, it doesn't look too bad but I think we'd want to avoid repeating and
process_nzb_archive_filealready repeats a lot of the single path.Maybe see if the indexer is open to constructing a normal zip first?
i asked but they are not at this time. so far this is the only site ive ever heard doing this.. but also maybe others would if it didnt break stuff.. as storing the nzb gzipped does prob make sense for the site for storage reasons.
they serve up the .gz.nzb with gzip so its deflated and a normal .nzb is served up just like any other site. its only when they do the multi-nzb part where they just zip up the existing .gz.nzb. it sounds like it would be too much for them to deflate nzb to store then to zip then serve that up to the endpoint on demand
Indeed the only reason I sort of consider this is because there is quite some code duplication between the NZB and archived code paths. That's due to historical reasons.
It would make more sense to have the code only extract from supported archives and then hand over to 1 code path that adds the NZB.
it sounds like it would be too much for them to deflate nzb to store then to zip then serve that up to the endpoint on demand
In theory this wouldn't be necessary. Both GZip and Zip use DEFLATE compression, so they could just extract the compressed stream and stitch it into a Zip file without any de/compression.
I do a similar thing with xz and 7z (the LZMA2 compressed streams are pulled from source xz files and stitched into a 7z without de/compression).
This would require a certain level of knowledge of file formats to pull off though.
looks like they figured that out, site has been updated now so it serves up a zip of .nzb so works with existing setup like other sites now. so feel free to close this out unless we want to just do the code cleanup to leverage adding this feature in the end
Nah 😄 it's working fine for now.