-chunk doesn't work with encryption
busyammonia opened this issue · comments
I want to create an encrypted multipart archive, but have trouble in doing so. It works fine without encryption.
[f@pc:~/Documents]$ zpaqfranz a "default_????.zpaq" ./kovcheg -m1 -chunk 1G -key
zpaqfranz v59.2q-JIT-L(2024-02-23)
franz:-method 1
franz:-chunk 1.000.000.000
Please do NOT start with '-' (minus)
Enter password :******************************************
franz:-key (hidden)
Creating default_0001.zpaq at offset 32 + 0
zpaqfranz error: 37992: cannot read salt
8.631 seconds (000:00:08) (with errors)
It is a bug, present only in the *nix version (on Windows it does works)
... work in progress...
Please check the attached pre-release
59_3a.zip
And yes, the fix is a...fflush 😄
Let me know if OK
Thank you
It does seem to work now, thank you.
[f@pc:~/Documents]$ /tmp/zpaqfranz a "test_????.zpaq" kovcheg/ -m1 -key -chunk 490M
zpaqfranz v59.3a-JIT-L(2024-02-25)
franz:-method 1
Please do NOT start with '-' (minus)
Enter password :******************************************
franz:-key (hidden)
franz:-chunk 490.000.000
Creating test_0001.zpaq at offset 32 + 0
Add 2024-02-26 10:03:39 19.273 4.707.961.405 ( 4.38 GB) 12T (5.076 dirs)
24.349 +added, 0 -removed.
INFO: The last chunk is: test_0005.zpaq
32 + (4.707.961.405 -> 3.136.879.839 -> [CKS #5] 1.614.066.183) = 1.614.066.215 [TOT] @ 89.91 MB/s
49.938 seconds (000:00:49) (all OK)
[f@pc:~/Documents]$ zpaqfranz t "test_????.zpaq"
zpaqfranz v59.2q-JIT-L(2024-02-23)
Archive seems encrypted (or corrupted)
Enter password :******************************************
test_????.zpaq:
1 versions, 24.349 files, 2.104.078.887 bytes (1.96 GB)
To be checked 4.707.961.405 in 19.273 files (12 threads)
7.15 stage time 12.78 no error detected (RAM ~192.78 MB), try CRC-32 (if any)
Checking 20.966 blocks with CRC-32 (4.695.889.249 not-0 bytes)
Block 00020K 4.32 GB
CRC-32 time 0.63s
Blocks 4.695.889.249 ( 20.966)
Zeros 12.072.156 ( 40) 0.011000 s
Total 4.707.961.405 speed 7.508.710.374/sec (6.99 GB/s)
GOOD : 00019273 of 00019273 (stored=decompressed)
VERDICT : OK (CRC-32 stored vs decompressed)
13.405 seconds (000:00:13) (all OK)
This is the fix... the fflush
With encryption zpaq write a 32 bytes-long "prequel" on the archive, the "salt"
Then, in another position of the source code, read back (from the filesystem, with a fopen/fread on the archive) those 32 bytes
When the archive ALREADY exists (aka: updating), no problem.
But when archive (the very first "chunk") does NOT (aka: creating), on some non-Windows systems the write (of the seed, the very first 32 bytes) is keeped in some operating system buffer, cache or whatever.
Therefore, when zpaq read back the salt... get nothing, because the file (in the filesystem) is 0 bytes long, and not 32 as expected
=>
fflushing enforce to transfer the 32 bytes to the file, making a 32-bytes-long file, that can be readed back
if (off==0 && myfwrite(salt, 1, 32, fp)!=32)
ioerr(thefilename.c_str());
fflush(fp); /// unix fix
Be careful with fixed-sized-multipart, it is a "cutting edge" feature (required a LOT of spaghetti-code)
Thanks for your report
Can you please make a new release with this fix so that Linux distros would upgrade it?
The source code already provides for the use of -chunk with encryption (the only function that cannot be used is -index, and I don't think it will be in the future)
If you are referring to the packages in the various distributions it is a very complex matter, it is not so easy to update them (it is not up to me).
I will try to contact the maintainers