Increase `max_queued_disk_bytes` limit
Obegg opened this issue · comments
qBittorrent v4.6.0 - Windows x64 (lt20 qt6)
Qt: 6.4.3
Libtorrent: 2.0.9.0
Boost: 1.83.0
OpenSSL: 1.1.1w
zlib: 1.3
Edition Windows 10 Enterprise LTSC
Version 21H2
Installed on 22/10/2023
OS build 19044.3570
Experience Windows Feature Experience Pack 1000.19052.1000.0
There is a setting in qBittorrent called "Disk queue size" and from what I understand it's called max_queued_disk_bytes in libtorrent.
Currently the limit is 2147483647 / 1024 = 2097151 KiB = ~2 GiB as answered here.
My issue is apparently an uncommon issue where one of my hard drives from whatever reason read and writes from within qBittorrent is unstable, causing a total slow down in the entire download process in qBittorrent (yes, downloads in other drives will slow down to a halt), to clarify - it only happens with large amount of downloads, it doesn't happen when downloading a single file.
This issue appears only one out of the five hard drives I own, and a simple benchmark test using CrystalDiskMark shows the read and writes as advertised (above 100MBs), which I consider normal.
This specific hard drive is Transcend StoreJet 35T3 which I shucked (I did not tape anything to the conductors, I just formatted it once plugged in the machine) and the hard drive inside is ST3000DM007-1WY10G.
I've tried many solutions like: fresh install of Windows 10 with default qBittorrent settings, install qBittorrent with lt20 and non lt20 (yes, it happens with lt1.2), maximizing every setting inside qBittorrent to 2147483647, different SATA cable, enabling page file for this specific hard drive, disabling page file for this specific hard drive, test to RAM using memtest86 (pass 1 found 0 errors. I did not test pass 2, 3 and 4).
Checking using CrystalDiskInfo doesn't reveal any issues with any of the drives in the machine.
I have yet to know why this issue is specific to this hard drive, and trying to analyze this issue has become annoying, I have tried to open Resource Monitor at the time of the issue and found that the memory has many Hard Faults, which prompted me to try and enable/disable page file (because apparently that's how you fix it by simple Google Search).
To counter this issue (temporary or permanently) I suggest increasing the limit of max_queued_disk_bytes, I see it as a benefit from a couple a reasons, one being that RAM could be of use in machines with slow read or writes (without considering the reasons why), if a machine has 256GB of RAM there's currently no option to use all of it, second reason is that it will solve (as I said - temporary or permanently) my issue, third reason is that I don't see any reason why NOT to increase the limit, as I see it it will only be a good thing.
2 GiB is quite a lot for this disk buffer. Are you confident that increasing it would improve your disk performance?
Are you sure you're not hitting your random access read rate limit?
It's not so easy to change this to a wider variable (int64) or to make it be counting 16 kiB blocks instead of bytes. The latter would be an incompatible change.
Are you confident that increasing it would improve your disk performance?
I did not suggest this would improve disk performance, not at all.
I was suggesting increasing it because I did not want to suffer from slow download speeds when I reach 2GiB.
This would not improve disk performance.
Are you sure you're not hitting your random access read rate limit?
I have no idea, and I would love to know how I can check it in order to add more information about this issue to get it fixed.
This issue has been bothering me for quite a while when downloading many files all at once, so I would really appreciate it if you can guide me on what to do and steps to take in order to debug it properly for it to be fixed.
so, you want to download into RAM quickly, and then (slowly) write it out to disk, yeah?
You won't be able to use the data while it's in the disk write queue though. It could probably pass hash checks and be available to upload to other peers.
That's probably the extent of the utility of keeping the data in the write queue over keeping it on the other peers (downloading at the speed the drive supports).
so, you want to download into RAM quickly, and then (slowly) write it out to disk, yeah?
Yes, that's what I did want up till now, I no longer want that upon understanding your explanation.
So my question now is how do I solve that issue, what's the root cause of it and what's the solution?
On second thought I would like to increase the max_queued_disk_bytes, 2GiB is bothersome.
Will you be able to do it?