Never silently overwrite existing files
Changaco opened this issue · comments
qBittorrent should never overwrite existing files without explicit confirmation from the user. Moreover, it should always checksum them and keep them if they match the torrent.
second this
yes :)
I believe uTorrent puts torrents in a paused state with a message suggesting a recheck in cases like this.
The importance of this premise cannot be overstated. I love qBittorrent, but I must say, overwriting files without user confirmation is a major problem with this client. I've had to re-download several GB files as a result of this behavior.
If I start qBittorrent without my download drive connected, all completed torrents whose destinations are on that drive start in a "Paused" state, which is fine. The problem is when I notice this and then mount the target volume on which the downloaded files already exist, and then click "Play" for each torrent, and qBittorrent proceeds to pave-over the existing files without warning!
Is there some workaround for this, now that I know what's happening? Do I need to right-click and choose "Force recheck" in this scenario, instead of clicking "Play"?
Is there some workaround for this, now that I know what's happening? Do I need to right-click and choose "Force recheck" in this scenario, instead of clicking "Play"?
I am almost certain that doing a force recheck before clicking "play" will not overwrite the files.
For all the others. I agree that this is a serious problem and needs fixing. The sad part is that we don't have much time to devote. I suspect to fix this bug you need to touch the code in several places and probably add more code. This isn't a simple one-liner bug unfortunately.
For the time being, I suggest doing a force recheck when a torrent is paused because of an error.
Also, some situtations aren't handled correctly by libtorrent either. For example:
- You finish a torrent in path X.
- You re-add the torrent and set it to download in path Y this time.(let's just say you forgot that you already have it)
- The torrent downloads some data, let's say 13%, and you realize that you already have it in path X and you want to seed it instead.
- You right-click in the torrent and select "Set location.." where you enter the path X
- libtorrent will move all the files from Y to X without checking if they already exist in X, and it overwrites them. You most probably lose only a few files from a multifile torrent if you do a recheck afterwards.
- The sad thing is, that libtorrent doesn't provide API to fine tune the move....
I made a feature request upstream for the last "bug": https://code.google.com/p/libtorrent/issues/detail?id=473
Also appending label "waiting API implementation", because it partially involves this bug.
Just to confirm that uTorrent doesn't have this problem. qBittorrent caused me severe headaches because of this.
The issue is listed as "fixed" on the libtorrent side now (see link posted by sledgehammer999), any chance of incorporating this into qBittorrent? Overwriting of files was the reason for me to go back to µtorrent, as I'm not on a fast connection to be able to easily recover from losing gigabytes of data.
Problem is when you some times start the client after a crash some times the size is not recorder and data is re downloaded.
When you use the extension for part downloads sometimes the extension is not used and in force rechecks the files without the incomplete extension are overridden.
In competed files in a multi part download the extension is some times removed when a file is completed. Sometime the completed portion is not detected and re created with the extension for incomplete downloads.
This should be treated as a critical bug. What good is a torrent client that destroys data meant to be distributed? There isn't even an error or anything when adding a torrent that already finished downloading (for example, with a different client).
At least add a "one-liner" workaround which sets the torrent into error state if files with same name is detected in the directory where the completed files land. Pretty please?
Sticking with µtorrent for now, and god knows that's not optimal on linux...
Sticking with µtorrent for now, and god knows that's not optimal on linux...
uTorrent does NOT use the libtorrent or libtorrent-rasterbar BitTorrent protocol engine so is unaffected by the issue
Yes, obviously it doesn't use libtorrent, that's not the point here. The point is I have to resort to using a wine program because the rest of the linux clients either don't offer it's functionality or eat my files.
I wish I could fix that myself, but looking at the code I'm totally lost in it.
that's not the point here
But it IS the point, the 'bug', 'problem', 'issue' IS in the libtorrent library, so it is NOT something that can be 'fixed' in the source code of qBittorrent. No amount of complaining or cajoling (though bribery will be welcomed) here or at the support forum will get this 'fixed'.,It is plainly, simply and wholly outside the control of qBT developers.
I suggest you re-read the messages. The issue @sledgehammer999 has reported to the libtorrent devs has been fixed. There is no issue with libtorrent anymore, it was fixed. Libtorrent now offers functionality to check for existing files. So no, the paving over files issue is not outside of developer's control. Not anymore.
Just so you know, that feature was implemented in the libtorrent 1.0.x series which was recently released. I don't think any distro has packaged it yet. So even if I put code inside qbt to utilize that it would be meaningless for linux users atm.
I'm on arch, so I have to get qbt from AUR anyway, might as well pull libtorrent-git or libtorrent-rasterbar-svn for it as well. I think adding the code to qbt might also be a good incentive for distros to update libtorrent.
I'm on arch, so I have to get qbt from AUR anyway, might as well pull libtorrent-git or libtorrent-rasterbar-svn
You'll need to recompile against 1.0.x too...
AUR is arch's user repository, which hosts source packages to be built on target system. Means installing qbt from AUR will make me build libtorrent 1.0.x first, then build qbt against the built libtorrent version. The packaging rules have that covered.
By the way, I'm not sure how "separate" the RSS downloader is in this scenario, but the last time I lost data was because it matched an entry I already had downloaded with another client, and overwrote each file without going into the error state as it (apparently) does when you add a torrent manually.
I don't know if it helps, but I tried to sum up the checks that have to be implemented for the two scenarios:
- opening torrent (also for RSS!)
- check final dir
-> if yes, start recheck - check temp dir (if set)
-> if yes, start recheck
-> else allocate file (if set)
- check final dir
- force recheck
- check if non .!qb file is present
- if not, check if .!qb file present
- check the file
- check if file has to be moved if result = 100%
Yes. A flowchart helps when implementing the code. It helps to not forget things.
Was any work already done on this? If not would like to look into it a submit a pull request at some point
I am actively working on this. And probably no one else. So you can look into it if you want.
This is still a problem right? Lost a 200MB file yesterday.
Just lost over 10,000 files due to this error. Can't believe this hadn't been fixed yet.
qBit also overwrites any changes you've made to already downloaded files without notice. Ouch.
This just wiped out quite some files, qBittorrent moves existing files back to the main Downloads folder (from Downloads Finished). Some will properly recheck, but some are full of zeros and overwritten.
qBittorrent was started without the storage medium connected (dodgy Mellanox NIC drivers), but normally a quit and a restart fixes and reloads all torrents, well this time, it spend time writing zeros to files it seems.
I made a torrent and some files haven't been fully transmitted to the swarm and qbittorent overwrote them ;(
Situation: I had two folders on my hard drive, unsure which one was the completed one. I pointed the torrent on the uncompleted one by accident. It checked to around 10%.
Then I pointed it to the completed one. Guess what happened? Qbit overwrote the completed files with the 10% files.
This is absolutely horrendous and unacceptable. These are files that I've waited two years for a seeder for! It's sheer luck that I still have a backup of them on my seedbox.
It's unbelievable that this doesn't get fixed. This should be priority number 1. Whoever programmed this had zero respect or consideration for the final user's data.
Same issue as here: #8758
That issue was well-explained and actually got the "data-loss" tag to show how important it is but got closed because "muh duplicate" and this badly formulated very abstract issue stays up as the original.
Again, UN-FUCKING-BELIEVABLE.
Wait so this has been a issue for 7+ years......wtf.
Is this ever going to be fixed? You'd think this issue would be number 1 priority. Using an external HDD with Qbittorrent is pointless until it is.
Guess I am gonna return to my utorrent 2.2.1. Not able to sort files by folder, but able to automatically recheck finished downloads.
It is "hilarious" to stumble upon 8 year old problem. One critical problem that ruins entire amazing project. It's almost like building 5th gen fighter that is unable to carry missiles.
Well, this problem is pretty common. I personally encountered many torrents with this problem:
There's exists a lot of torrents which contains file/files with the same filename(s) and the same extension(s),
but content of the file(s) is different.
I'm asking for function which will check sizes of the file(s) in new torrent and existing one,
or check filenames + hash (or something else), and if its mismatch with already present file(s),
qbittorrent must open "adding-torrent window" with warning message:
File(s) with the same filename(s) is already present in the folder, sizes of file(s) are different.
@sledgehammer999 @glassez @Chocobo1 @FranciscoPombal, Please, move this issue higher in your priority-lists.
Just lost another files due to this issue. 😭
I want to overwrite the content in the specified existing directory with qbittorent, how can I do that?
@kenshinnmt commented on 2022. ápr. 15. 11:27 CEST:
I want to overwrite the content in the specified existing directory with qbittorent, how can I do that?
Matching files are automatically checked and overwritten as needed when you start a torrent.
@kenshinnmt commented on 2022. ápr. 15. 11:27 CEST:
I want to overwrite the content in the specified existing directory with qbittorent, how can I do that?
Matching files are automatically checked and overwritten as needed when you start a torrent.
In my case, qbittorent does not overwrite, I want to override to update content but can't.
When I start adding torrent files, the metadata shows the new content, but when it's done, the old content is still there, nothing has changed.
@kenshinnmt commented on 2022. ápr. 15. 11:58 CEST:
@kenshinnmt commented on 2022. ápr. 15. 11:27 CEST:
I want to overwrite the content in the specified existing directory with qbittorent, how can I do that?
Matching files are automatically checked and overwritten as needed when you start a torrent.
In my case, qbittorent does not overwrite, I want to override to update content but can't.
When I start adding torrent files, the metadata shows the new content, but when it's done, the old content is still there, nothing has changed.
Then there's nothing to overwrite because the files are different.
@kenshinnmt commented on 2022. ápr. 15. 11:58 CEST:
@kenshinnmt commented on 2022. ápr. 15. 11:27 CEST:
I want to overwrite the content in the specified existing directory with qbittorent, how can I do that?
Matching files are automatically checked and overwritten as needed when you start a torrent.
In my case, qbittorent does not overwrite, I want to override to update content but can't.
When I start adding torrent files, the metadata shows the new content, but when it's done, the old content is still there, nothing has changed.Then there's nothing to overwrite because the files are different.
In my case, the file content is different.
File A.exe, size 313kb, downloaded with qbittorent 1st time, everything works perfectly.
Then I edited the A.exe file mentioned above, and the new size is 115kb.
I create a new torrent file, the metadata before starting to download the file shows that the size of A.exe is 115kb, but after qbittorent has finished downloading, the torrent status is 100%, seeding, file A .exe is still 313kb, it's the old file.
@kenshinnmt
What do you mean "I create a new torrent file", how is creating a torrent file relevant? What the hell are you doing?
What do you mean "I create a new torrent file", how is creating a torrent file relevant? What the hell are you doing?
I create torrents to share my files with other remote computers.
As I said above, I used qbittorent to create torrent files, used qbittorent to download, and the first time everything worked fine.
The second time, I need to edit the content, but still the A.exe name, then continue to create the torrent file, and share it to the remote computer, this time qbittorent does not overwrite the A.exe file.
It really feels to me that you are missing the point of this ticket.
You might want to open a new ticket and plead your case separately
It really feels to me that you are missing the point of this ticket.
You might want to open a new ticket and plead your case separately
#16881
I tried to open a ticket at this link
I would like to see more similar cases about this error
Happy 10 year anniversary to this crippling issue, STILL not even being listed as being worked on!
...Wooooo!
........ 😑
it's not a bug, it's a feature 😹
re-downloading a bunch of stuff thanks to this.
Was using deluge and utorrent 2.2.1, never had this issue
Just started using qbittorrent, encountered this issue.
What is the problem actually, because there's exactly nothing useful in the opening post.
What is the problem actually
Please search for word "referenced" on this page and click the links. Numerous issues was closed in favor of this one.
Issue is present currently, i get a Torrent which shares the same path and most of the files. It is a collection which the owner of the torrent keeps adding files and updating the torrent instead re-sharing all the content in a new torrent.
Expected behavior is after finishing the checksum qBit can handle the files: Downloading the missing / failed ones and sharing the others.
Currently qBit marks the torrents as permanently "moving" and does not download / upload properly. This only happens with qBit as other torrent clients can handle this properly (by pausing one of the torrents or doing as expected)
Maybe you need to open a new issue for this @Arturoe1
Bumping this issue? Although this thread has been open for more than a decade there hasn't been a fix? Possibly a checksum associated with the torrent stored locally might pose as a simple workaround, especially for my situation.
I have set all torrents to download locally and moved to a local network storage location. This server is taken down moderately frequently for matinence and the machine running qbittorrent isn't always shut down with the server.
I can't believe that it's already been 11 years. Originally this issue drove me absolutely mad as one of the foreign trackers I was on would repeatedly offer torrents whose folders/files used existing names but different contents. I ended up using a different client at the time. Fortunately they have since stopped that ridiculous practice, but it's one reason why this could be a real issue in some rare scenarios.
@kesumin
This. I lost so many files recently because of this. Same setup, mapped to network storage. Qbitorrent didn't shut down with the server because it's on a separate machine. It looks like qbit deleted the files, and tried to redownload them. A lot of the trackers don't even actually work anymore. So I have no way of getting the files back.
Absolutely insane. This should be a top priority issue.
Bump. I still would like to see this feature implemented.
@seniorm0ment
You guys might need to rethink your setup. Writing the matching files in question is normal behavior.
@StrangePeanut Does this actually happen? Formerly I donwloaded stuff on storage that was removed the later re-added, but don't remember re-downloads, I did get data loss with the ".unwanted" folder clusterF.
How else would I set it up? The files are stored on my server.
If qbit realizes it can't find the storage path, then why does it try to redownload? Also it was redownloading when the server came back up, so the storage path was there. But it just started overwriting or something.
Do out expect me to have duplicates of all the files, just so one folder can be used exclusively for seeding? If it was 30gb that's one thing, but when you have terabytes of different stuff.
@seniorm0ment
Not knowing any details, you might be able to run QB run on the server and download it there directly. Access it via WebUI if nothing else.
Maybe don't set QB to move automatically the files there, move it manually, only if you really need it there. Such things.