Feature Request: automatic download based on OS and ARCH
NiceGuyIT opened this issue · comments
It would be nice if OS and ARCH could be pulled from the current OS and architecture and be "untagged" like the version. This would allow for automated download of an asset that matches the current computer.
OS
The OS name for macOS varies between apple, macos, darwin and osx. Maybe search for all those and untag it as {os}.
Windows varies between windows, win64, and win32.
Linux usually is just linux but I've seen linux64.
ARCH
The architecture is more varied due to the architecture naming scheme in the programming language used. I don't know if there's an easy solution for this.
Examples
Consider junegunn/fzf for example.
dra untag junegunn/fzf
Release tag is 0.42.0
? Pick the asset to untag ›
❯ fzf-0.42.0-darwin_amd64.zip
fzf-0.42.0-darwin_arm64.zip
fzf-0.42.0-freebsd_amd64.tar.gz
fzf-0.42.0-linux_amd64.tar.gz
fzf-0.42.0-linux_arm64.tar.gz
fzf-0.42.0-linux_armv5.tar.gz
fzf-0.42.0-linux_armv6.tar.gz
fzf-0.42.0-linux_armv7.tar.gz
fzf-0.42.0-linux_loong64.tar.gz
fzf-0.42.0-linux_ppc64le.tar.gz
fzf-0.42.0-linux_s390x.tar.gz
fzf-0.42.0-openbsd_amd64.tar.gz
fzf-0.42.0-windows_amd64.zip
fzf-0.42.0-windows_arm64.zip
fzf-0.42.0-windows_armv5.zip
fzf-0.42.0-windows_armv6.zip
fzf-0.42.0-windows_armv7.zip
fzf_0.42.0_checksums.txt
Source code (tar.gz)
Source code (zip)
Untagging on an M1/M2 Mac would choose darwin for the OS and arm64 for the ARCH, resulting in the following.
dra untag junegunn/fzf
Release tag is 0.42.0
✔ Pick the asset to untag · fzf-0.42.0-darwin_arm64.zip
fzf-{tag}-{os}_{arch}.zip
This command can be cross-platform to download the appropriate file for that OS/ARCH combination.
dra download --select 'fzf-{tag}-{os}_{arch}.zip' junegunn/fzf
Hi @NiceGuyIT!
I really like your idea, and it's very similar to something I was thinking about a while back.
My concern is that I'm not sure if there is a robust solution for this. Or at least nothing comes to mind right now.
Guessing the OS I think is kinda doable, but the architecture could be tricky to pull off.
One improvement could be to sort the list of assets in the download/untag select list by the OS name (and maybe the ARCH if possible).
That way you would find your "preferred" asset faster without scrolling too much.
Out of curiosity, how did you find out about dra?
I need a way to programmatically download files from GitHub but without using the gh CLI which requires a login. I want to learn Rust and discovered dra when Googling how to download JSON from GitHub's API. I wouldn't have discovered this repo otherwise! If you want more visibility 👀, perhaps add it to Awesome Rust or similar.
As for the implementation, I would start with a regex to identify the asset pattern. The components are name, version, os, arch and sometimes flavor. The separator is usually _, - or .. Once a pattern is identified, the components can be extracted. A map can be used to match the detected OS/ARCH with the assets. For the architecture, I don't think the same name is used in multiple languages, meaning "amd64" is synonymous with "x86_64" in all languages. (Is that true?)
This may not be a robust solution but it will work for most packages. There will always be edge cases that are hard or impossible to solve. For example, micro doesn't follow a standard naming convention and Alacritty doesn't produce a Linux binary.
If I was more experienced in Rust, I'd see about contributing. In the mean time, I'm using something like this. I ❤️ the project BTW! It greatly simplifies downloading assets.
# https://github.com/go-task/task
os=$(uname -s | tr '[:upper:]' '[:lower:]')
arch=$(uname -m | sed 's/x86_/amd/')
tmp_dir=$(mktemp --directory)
cd "${tmp_dir}" || exit 1
dra download --install --select "task_${os}_${arch}.tar.gz" go-task/task
chmod go-w task
sudo cp "${tmp_dir}/task" /usr/local/bin/task
cd / || exit 1
rm -r "${tmp_dir}"I'm really happy you like the project 🚀
Just to better understand your problem, is it right if I say that you would like to automatically download an asset for your current OS+ARCH?
Example 1
Repository: sharkdp/bat
OS: Linux x86_64
Assets:
bat-v0.24.0-arm-unknown-linux-gnueabihf.tar.gzbat-v0.24.0-x86_64-apple-darwin.tar.gzbat-v0.24.0-x86_64-unknown-linux-musl.tar.gz
You run dra download and the downloaded asset is bat-v0.24.0-x86_64-unknown-linux-musl.tar.gz
Example 2
Repository: sharkdp/bat
OS: macOS x86_64
Assets:
bat-v0.24.0-arm-unknown-linux-gnueabihf.tar.gzbat-v0.24.0-x86_64-apple-darwin.tar.gzbat-v0.24.0-x86_64-unknown-linux-musl.tar.gz
You run dra download and the downloaded asset is bat-v0.24.0-x86_64-apple-darwin.tar.gz
Example 3
Repository: sharkdp/bat
OS: Linux ARM v8
Assets:
bat-v0.24.0-arm-unknown-linux-gnueabihf.tar.gzbat-v0.24.0-x86_64-apple-darwin.tar.gzbat-v0.24.0-x86_64-unknown-linux-musl.tar.gz
You run dra download and no asset match your current OS+ARCH. The user must manually choose the asset to download or exit.
you would like to automatically download an asset for your current OS+ARCH?
Exactly!
Reasoning
I support Windows and want to have one script/program for all platforms. Most "automation" programs are designed with the cloud in mind and work on Linux and maybe macOS with a little bit of help. They frequently leave out Windows. Cross-platform scripting took a huge step forward with task and just. Just allows setting the shell and nushell is an excellent shell for all platforms. Many other tools are available as a single binary install making cross-platform support significantly easier.
Then comes the problem of bootstrapping. Bootstrapping will always be OS specific since a bare machine does not have the same tooling across all platforms. Which tool is the easiest to download and can be used to download the other tools? dra makes sense because it can download the other assets, but then I run into the problem above. If dra download --install can download and extract (--install) assets for the current OS+ARCH, it would greatly simplify the bootstrapping process.
Side note
This gist has many scripted variations for downloading from GitHub, one of which is below. It would be nice if dra could be released without the version in the filename and compression. This will help with bootstrapping but I realize this may not be desired. Feel free to ignore!
wget https://github.com/aquasecurity/tfsec/releases/latest/download/tfsec-linux-amd64Thank you for the detailed explanation. I was missing it to fully understand the actual problem, instead of the proposed solution.
I'm not sure when, but I can try to sketch something up in the following weeks.
I like this feature and was thinking about it for some time now.
About the side note:
I'm not going to release assets that are not inside archives. Sorry about that.
Maybe this script I use to bootstrap my Linux system that install dra can help you: https://github.com/devmatteini/dotfiles/blob/main/bootstrap/dependencies.sh#L12-L30
Hi @NiceGuyIT!
I finally had time to work on this feature, and I'm pretty happy about it.
If you want to try it and share feedback about it, here are the instructions (you will have to compile dra by yourself for now):
# you need rust installed
git clone https://github.com/devmatteini/dra && cd dra
make release
# you can use `-a` or `--automatic`
./target/release/dra download -a <REPO>There are still some things I'm working on, but I think I will create a new release pretty soon.
Woah!😲 This is awesome! dra can download most assets automatically. I'm using an M1 Mac and it could download the assets below. The ones that didn't work don't support aarch64.
Works
- starship/starship
- tomwright/dasel
- uutils/coreutils
- watchexec/watchexec
- wader/fq
- BurntSushi/ripgrep
- FiloSottile/age
- adnanh/webhook
- alacritty/alacritty
- atuinsh/atuin
- casey/just
- cloudflare/cfssl has multiple assets but only one was downloaded.
- denoland/deno
- docker/buildx
- docker/compose
- fujiapple852/trippy
- go-acme/lego
- go-task/task
- hairyhenderson/gomplate
- helix-editor/helix
- junegunn/fzf
- kopia/kopia
- mikefarah/yq
- mozilla/sops
- mr-karan/doggo
Doesn't work
- sharkdp/fd
- 01mf02/jaq
- evanw/esbuild
- ducaale/xh
- getzola/zola
- gohugoio/hugo
- ouch-org/ouch
- phiresky/ripgrep-all
- pkolaczk/fclones
@devmatteini As far as I'm concerned, this issue is resolved. I'm not closing it because you said there are a few things you're still working on.
New release is out 0.5.0 🎉