Disk error on creating file, pausing downloads
Jonnehs opened this issue · comments
SABnzbd version
4.2.1
Operating system
docker compose on ubuntu
Using Docker image
linuxserver
Description
I'm just testing moving to containerised versions of my various apps etc and this is now running on an intel gen11 nuc under docker.
The storage is a simple CIFS share over 1gb link to a windows box.
Sabnzbd keeps pausing during some test downloads, claiming a disk error. Logs below.
I can find no issue with the storage, the only real issue is that its a little 1gb pipe on this test setup to the storage. I want to eventually get a 10gbe nas and docker host, present NFS storage or something but for now it'd be good to clear up this issue.
2024-01-13 12:10:48,030::ERROR::[assembler:108] Disk error on creating file /incomplete/EPISODEXXXXX.mkv-xpost/[N3wZ] \7LBF7G126780::[PRiVATE]-[WtFnZb]-[WtF[nZb].nfo]-[2+2] - "" yEnc 12 (1+1)
2024-01-13 12:10:48,030::INFO::[assembler:116] Traceback:
Traceback (most recent call last):
File "/app/sabnzbd/sabnzbd/assembler.py", line 81, in run
self.assemble(nzo, nzf, file_done)
File "/app/sabnzbd/sabnzbd/assembler.py", line 171, in assemble
with open(nzf.filepath, "ab", buffering=0) as fout:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 22] Invalid argument: '/incomplete/EPISODEXXXXX.mkv-xpost/[N3wZ] \7LBF7G126780\::[PRiVATE]-[WtFnZb]-[WtF[nZb].nfo]-[2+2] - "" yEnc 12 (1+1)'
2024-01-13 12:10:48,031::INFO::[downloader:407] Pausing
2024-01-13 12:10:48,031::INFO::[notifier:137] Sending notification: SABnzbd - Paused (type=pause_resume, job_cat=None)
2024-01-13 12:10:48,031::INFO::[downloader:427] Forcing disconnect
So far as I can tell the disk write speed is pretty poor. Is that the issue? I am going to try capping download speed a little below the disk write speed and see if it clears up, but any help would be appreciated.
Not a bug, but anyway:
SAbnzbd -> Wrench -> 1GB test download ... does that work?
(My guess: Yes)
the download seems to work for a while, its been going for a couple of hours and paused a few times
yes the test works, but again, not sure how to effectively test. Really I was just hoping to get more info about what sab thinks the disk issue is, or if there was any more effective troubleshooting to try
yes the test works,
Good. As I expected and predicted.
but again, not sure how to effectively test. Really I was just hoping to get more info about what sab thinks the disk issue is, or if there was any more effective troubleshooting to try
My guess: the cause is that you're running SABnzb on docker=Linux, saving to a Windows = CIFS share, and the download has all kind of weird, foribdden characters 'EPISODEXXXXX.mkv-xpost/[N3wZ] \7LBF7G126780\::[PRiVATE]-[WtFnZb]-[WtF[nZb].nfo]-[2+2] - "" yEnc 12 (1+1)'
Before we go further:
when you start SABnzbd, don't you get a warning about your Imcomplete directory?
or did you turn it off?
... is not writable with special character filenames. This can cause problems.
To prevent all helpful warnings, disable Special setting 'helpful_warnings'.
Oh, wait: you've put Incomplete on your remote Windows drive?! Then it's easy to solve: Don't!
Keep it inside your docker filesystem.
limited it to 30MB/s and it stopped again with the same error
2024-01-13 13:32:37,212::DEBUG::[assembler:80] Decoding part of /incomplete/EPISODEXXXXX.mkv-xpost/[N3wZ] \iZDIcQ126780::[PRiVATE]-[WtFnZb]-[WtF[nZb].nfo]-[2+2] - "" yEnc 12 (1+1)
2024-01-13 13:32:37,212::INFO::[notifier:137] Sending notification: Error - Disk error on creating file /incomplete/EPISODEXXXXX.mkv-xpost/[N3wZ] \iZDIcQ126780::[PRiVATE]-[WtFnZb]-[WtF[nZb].nfo]-[2+2] - "" yEnc 12 (1+1) (type=error, job_cat=None)
2024-01-13 13:32:37,212::ERROR::[assembler:108] Disk error on creating file /incomplete/EPISODEXXXXX.mkv-xpost/[N3wZ] \iZDIcQ126780::[PRiVATE]-[WtFnZb]-[WtF[nZb].nfo]-[2+2] - "" yEnc 12 (1+1)
2024-01-13 13:32:37,212::INFO::[assembler:116] Traceback:
Traceback (most recent call last):
File "/app/sabnzbd/sabnzbd/assembler.py", line 81, in run
self.assemble(nzo, nzf, file_done)
File "/app/sabnzbd/sabnzbd/assembler.py", line 171, in assemble
with open(nzf.filepath, "ab", buffering=0) as fout:
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 22] Invalid argument: '/incomplete/EPISODEXXXXX.mkv-xpost/[N3wZ] \iZDIcQ126780\::[PRiVATE]-[WtFnZb]-[WtF[nZb].nfo]-[2+2] - "" yEnc 12 (1+1)'
2024-01-13 13:32:37,212::INFO::[downloader:407] Pausing
2024-01-13 13:32:37,212::INFO::[notifier:137] Sending notification: SABnzbd - Paused (type=pause_resume, job_cat=None)
2024-01-13 13:32:37,212::INFO::[downloader:427] Forcing disconnect
I wonder if its the extraction process rather than the download causing this? Either way still no visible issues from the storage
Do you read and do what I adviced?
I've used remote storage for incomplete since forever, I suppose there is no reason to do so except that I find it needs a manual cleanup every now and again
Yes I do get that error, but its the same directory the previous windows hosted sabnzb install has been using (on a remote share) for years
Yes I do get that error, but its the same directory the previous windows hosted sabnzb install has been using (on a remote share) for years
Ah, and you decided to ignore that? Clear
I've nothing else to tell or advice you.
not sure if i've upset you or something but the attitude really doesn't help
I'm not convinced it can be characters in the download since it resumes happily and works to download and extract the files, surely if it was problematic characters it would fail entirely since they would always be present?
In Config Switches, enable Make Windows Compatible in the bottom.
You are writing to a Windows share, but SABnzbd doen 't know that so it's allowing characters in the filenames that are fine on Linux but not on Windows.
But I agree with Sander, you shouldn't be putting your Incomplete on a (Windows) share but somewhere locally. The mixing of OS systems will cause issues.
Indeed, did you not get a warning on startup about filenames? Maybe we didn't include some characters in the check.
I'm not convinced it can be characters in the download since it resumes happily and works to download and extract the files, surely if it was problematic characters it would fail entirely since they would always be present?
If that nfo file in the error you posted is the only file with problematic chars, the rest of job should still complete and extract. But the forward slashes in that filename are an obvious no-go on windoze, because those serve as directory separators on that operating system.
Thanks, when did this change? My old install presumably just assumed this setting somehow because it was installed on windows I guess?
I've set it and will see if it works.
I'm still getting the warning on startup even with it set,
I'm not convinced it can be characters in the download since it resumes happily and works to download and extract the files, surely if it was problematic characters it would fail entirely since they would always be present?
If that nfo file in the error you posted is the only file with problematic chars, the rest of job should still complete and extract. But the forward slashes in that filename are an obvious no-go on windoze, because those serve as directory separators on that operating system.
what happens when i resumed the download manually then, it just carried on and worked? Its downloading a load of releases from the game group so I guess a bunch of them have slashes, but it only seems to be stopping on some. Its not a very consistent failure if the characters are really what's causing it I guess, thats what my confusion was.
I've set the windows flag as suggested now, so will close this if it stops doing it.
Thanks all.
When you run it in Windows, we naturally apply the Windows based filtering.
On Linux mounts we have no clue what filesystem is actually behind it.
Maybe you just got lucky that the failed files were not important to the actual download, maybe just nfo files etc
working fine now for the rest of the queue so, that seems to have nailed it
glad we got to the bottom of it, thanks for the help
not sure if i've upset you
Yes. I don't like your attitude:
- you ignore SABnzbd's warnings
- you ignore my advice
or something but the attitude really doesn't help
At least we agree on that. Good!
@Jonnehs while I might had worded it differently, I do agree with Sander that SABnzbd told you there's a problem and you decided to ignore it. You didn't even mentioned the warning when creating the report here. This made it harder to help you with the right setting.
I don't really want to get into it to be honest, someone as angry as he is about a problem report obviously has more wrong in his life than me "ignoring" a warning.
To my mind I had not changed the storage from my previous install so the warning just confused me. I did not know releases sometimes had bad characters in, nor did I know the slashes in the logs were part of a file name. And the download continued to work and downloaded and then imported, which would have been impossible if it was a bad character issue with the actual video, so that threw me off.
It's easy for you guys to know, because you wrote the system. Attacking users who are just genuinely confused is a bit of a weird look.
The error and warning could both do with a bit more explanation, if the failure had said there were bad characters I would have been able to put this together myself. If the warning mentioned the "storage is windows" setting I might have known to look for it.
So thanks for your help, that's my feedback.
@Safihre had a thought;
You could have the warning trigger a question to the user as to what the underlying file system is, and then set the correct settings based on response or guide the user to the correct settings.
I understand you're recommending not to do what I am doing, and this is only a test, but the above would be a much better experience for someone like me who didn't know the setting existed.
its hard to make assumptions, people do all sorts of dumb things.
pretend user is running sab in docker on windows in wsl on a linux. then all their paths are actually remote shares to their mac pro. no logic on earth would get it "right". when you run mixed os and do things suboptimally.. there is going to be compromises.
but thats why we offer various options and settings so people can tweak if need be. we do make some crude assumptions like hey if host os is windows we assume windows will be in the filesystem chain and make things windows safe for files/folders. now technically user could be running windows and storing files on a linux remote.. then they didnt need the windows safe stuff but how are we to know.
generally people run into issue, look at options and tweak in response. sometimes its not clear that the issue is because they are using encryption, case sensitive filesystem, remote filesystem with multi-tiered setups, etc etc. also what works for incomplete may not be the same for complete.
windows being the lowest common denominator, we know if the os sab is running on then use those guards, we cant really assume this to be the case if its not visible to sab in the chain. and if we just always assume windows is going to be used, then no one would be happy with that assumption.
One more thing: Make Windows Compatible only fixed the files that SABnzbd tries to write. However, other tools like par2 and unrar also don't know that your file system is Windows even though you are on Linux. So if inside the "abc.rar" is actually a filename that contains an invalid character specifically for Windows, unrar will will fail to unpack. Unrar has no flag to make the filenames Windows Compatible, so you can't fix that.
Edit: on Windows unrar will sanitize the filenames automatically, it just doesn't have a flag to do that on other platforms.
So this this setup will create problems and that's what the warning is saying..