Torrent download and Zstd compression
mgrankin opened this issue · comments
I repackaged dataset with zstd and uploaded it to academictorrents.com.
http://academictorrents.com/details/dba20c45d4d6fa6453a4e99d2f8a4817893cfb94
Also it is temporarily availible as a direct link here
http://fma.mine.toys/fma/checksums
http://fma.mine.toys/fma/fma_metadata.tar.zst
http://fma.mine.toys/fma/fma_small.tar.zst
http://fma.mine.toys/fma/fma_medium.tar.zst
http://fma.mine.toys/fma/fma_large.tar.zst
http://fma.mine.toys/fma/fma_full.tar.zst
Zstd is way faster than Zip to unpack. If you don't have "tar" with zstd support you can install if from conda.
conda install tar zstd
How to unpack
tar -xaf fma_small.tar.zst
How to pack
tar -caf fma_small.tar.zst fma_small/
Alternatively you can install binary and use zstd as an external command for tar.
sudo apt install zstd
tar -I zstd -xvf fma_small.tar.zst
tar -I zstd -cf fma_small.tar.zst fma_small/
If that is desirable and appropriate then I can make a PR with changes to README.
I just downloaded the fma_small dataset but an unexpected end of file error occur when I tried to unzip.
I tried with these commands:
brew install zstd
zstd -d fma_small.tar.zst
Has anyone managed to unzip?
Also, can you host a direct link again?
The links are up. I'll try to keep them up.
You shouldn't try to unpack files with zstd command. Use tar. Tar actually manages the file container and zstd is the compression algorithm.
I was able to uncompress the fma_small.tar.zst file differently. I added the solution as a comment on a StackOverflow question. https://stackoverflow.com/a/61800897/9571713
The torrent seems down. Academic torrents shows no seeds. Can you check the server?
Yeah, it's been cold for few days in Moscow (-20C) and my switch located on the balcony went out. I'll buy new in the next couple of days.
Ouch. Thanks @mgrankin. In the meantime, direct download from https://os.unil.cloud.switch.ch/fma should work. What are your reasons to prefer torrents over direct downloads? We considered to setup a seeding server at EPFL but finally didn't. We could consider it again (or find another alternative) if there are good arguments.
@stdio2016 links and torrent are up