Giganews: Received unknown status code 220 (and subsequent UnicodeDecodeError)
totally-a-human opened this issue · comments
SABnzbd version
4.2.3RC2 [dbff203]
Operating system
Debian 12
Using Docker image
None
Description
When downloading a file SABnzbd got an unknown status code 220 from Giganews. When SABnzbd tried to handle this error it also appears that it failed to decode a byte throwing an exception that paused the download queue.
The last lines from the log file are pasted here.
2024-03-02 10:31:59,671::INFO::[notifier:142] Sending notification: Warning - 34@news.giganews.com: Received unknown status code 220 for article f90b88cd3830a77ecc3212@ace34fb3af6a: 220 0 <f90b88cd3830a77ecc3212@ace34fb3af6a> (type=warning, job_cat=None)
2024-03-02 10:31:59,671::WARNING::[downloader:768] 34@news.giganews.com: Received unknown status code 220 for article f90b88cd3830a77ecc3212@ace34fb3af6a: 220 0 <f90b88cd3830a77ecc3212@ace34fb3af6a>
2024-03-02 10:31:59,673::INFO::[notifier:142] Sending notification: Error - Fatal error in Downloader (type=error, job_cat=None)
2024-03-02 10:31:59,672::ERROR::[downloader:697] Fatal error in Downloader
Traceback (most recent call last):
File "/usr/share/sabnzbdplus/sabnzbd/downloader.py", line 693, in process_nw_worker
self.process_nw(read_fds[nw_queue.get()])
File "/usr/share/sabnzbdplus/sabnzbd/downloader.py", line 774, in process_nw
nw.nntp_msg.splitlines()[0],
^^^^^^^^^^^
File "/usr/share/sabnzbdplus/sabnzbd/newswrapper.py", line 102, in nntp_msg
return ubtou(self.data[: self.data_position]).strip()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/share/sabnzbdplus/sabnzbd/encoding.py", line 41, in ubtou
return str_in.decode("utf-8")
^^^^^^^^^^^^^^^^^^^^^^
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe8 in position 986: invalid continuation byte
2024-03-02 10:31:59,674::INFO::[downloader:394] Pausing
2024-03-02 10:31:59,674::INFO::[notifier:142] Sending notification: SABnzbd - Paused (type=pause_resume, job_cat=None)
2024-03-02 10:31:59,674::INFO::[downloader:414] Forcing disconnect
@ShaneMcC I told you so 😅(about including the message resulting in problems)
Might have to get a Giganews account to Debug this one. Seems strange to get 220 here...
@Safihre Yes you did 😅 I still think there is value in having that response-code line for error messages though.
Maybe there is a better/safer way we can try and get the first (header) line of the message?
Though interesting that it looks like it worked enough to send the first message but then error-ed afterwards trying to do it a second time.
The decoding error seems to be random. So far I've gotten one message with Received unknown status code and three exceptions.
Just now I also happened to get a new exception I haven't seen before. Don't know if this is a different bug or if it's related. Still Giganews though.
2024-03-02 19:19:58,735::INFO::[notifier:142] Sending notification: Error - Fatal error in Downloader (type=error, job_cat=None)
2024-03-02 19:19:58,734::ERROR::[downloader:697] Fatal error in Downloader
Traceback (most recent call last):
File "/usr/share/sabnzbdplus/sabnzbd/downloader.py", line 693, in process_nw_worker
self.process_nw(read_fds[nw_queue.get()])
File "/usr/share/sabnzbdplus/sabnzbd/downloader.py", line 732, in process_nw
if nw.status_code != 222 and not done:
^^^^^^^^^^^^^^
File "/usr/share/sabnzbdplus/sabnzbd/newswrapper.py", line 98, in status_code
return int(self.data[:3])
^^^^^^^^^^^^^^^^^^
ValueError: invalid literal for int() with base 10: b'Mes'
I wonder if actually it's just trying to decode random data lines as headers?
It got the 220 (as a positive "here is the data" message because it's a 2XX) but sab is treating it as unknown and "bad" (and not expecting any further data), so then the next line of data is random binary data but sab is expecting a header again and is now trying to read it as a header response instead?
(I don't know enough about how sab's downloader works to investigate further, and that article id on my GN account replies as a 222 not a 220)
I wonder if actually it's just trying to decode random data lines as headers?
yup: 'utf-8' codec can't decode byte 0xe8 in position 986
position 986 is much more/further than the the one-line response (typical 80 chars?)
And: one-line response is pure ASCII, so no utf-8 decode failure possible.
So SAB is indeed slurping and trying to utf8-decode the binary data (which is yencoded)
This is indeed a bug..
220 is a valid code, that we can expect in case servers don't support the BODY command.
We always send BODY and only if the server responds with 500 (unknown command / syntax error) we switch to ARTICLE.
So we should have expected 220...
@totally-a-human could you try the new build here:
https://github.com/sabnzbd/sabnzbd/actions/runs/8125304149
I also fixed the decoding problem by removing the printing of the message.
@totally-a-human could you try the new build here: https://github.com/sabnzbd/sabnzbd/actions/runs/8125304149
I tried downloading my files again with that build and it appears to be working fine now! No warnings, errors or exceptions! Thanks!
I got a new warning showing up in the log and UI: Received unknown status code 503
Probably not much that can be done about that one except for ignoring it?
No indeed.. Server error.
Released as part of 4.2.3RC2.