[ENG-349] Archive/Zip
brxken128 opened this issue · comments
We need a job to archive file(s) and directories. Currently we aren't able to multi-select items within the explorer, so I'll design this job around one object or directory (like the others). There are some thoughts about implementing multi-select support for FS jobs below¹.
We can offer ZSTD and possibly a few other compression algorithms (and levels!), with the zip crate. I have experience with this crate already, and I feel as though it's better than the alternatives. There are a few quirks for Windows machines (mostly \ issues) that I'll have to ensure are thoroughly tested.
Eventually I feel as though we should design our own zip crate - it's not the easiest of tasks but we'd definitely benefit from it in the long-run.
We will also have to implement zip-slip protection for unzipping.
Additionally, we need to ensure that the supported compression algorithms won't cause any build issues, as I believe some of them are as annoying as FFmpeg - I'll need to double check this though.
¹ As a further note, multiple input/source object support probably be achieved by taking a Vec<{path_id}> and iterating through them to create steps for the job. We can use this for copy, erase, cut, delete and pretty much all actions due to the job system.
From SyncLinear.com | ENG-349
Agreed with @ericson.ds999, we don't need to broke this job to steps, it can be a 1 big bulky zip and unzip job that does it thing through the library. I don't think pausing it or remembering where it is makes sense here too. Let's not make this complicated and keep it as simple as possible. It just needs work.
I don't think that this job can be broken in steps at all. Steps are only needed when we can interrupt the job to be resumed later, which isn't the case. But the new task system enables periodically pauses with a mutable state between them. Without the restriction of being needed to be serializable. On a compression task we would be able to pause it in memory, but if the app was closed, it would need to be started from the beginning.
Oh sorry I missed that part, thanks @brxken128! Do you know about this @ericson.ds999?
and the current job system doesn't allow for a mutable object to be shared between job steps (which is what the zip crate would require)
Why is this blocked?
Yeah, we still need zip-slip protection as well as special path handling on Windows (most likely), and the current job system doesn't allow for a mutable object to be shared between job steps (which is what the zip crate would require).
As mentioned on Slack, plan is to use a already ready zip crate
Might not be blocked but not a priority atm.
I'm pretty sure this is still not possible given the current state of the job system. I will look further into it, but we need a mutable state which is not serialised. Maybe a serde(skip)will work, but the job will not be resumable at all.
I'm currently unable to continue with this as we're not able to mutably share objects between job steps, when the objects in question cannot be serialized/deserialized.