cargo-dist
cargo build
but For Building Final Distributable Artifacts and uploading them to an archive.
The Big Idea of cargo-dist is that we want to streamline all the steps of providing prebuilt binaries for a rust project. This includes:
- Generating your "Cut A Release" Github CI for you
- Picking good build flags for a "production" release
- Making zips and installers for the resulting binaries
- Generating a machine-readable manifest so other tools can understand the results
- Uploading all the resulting artifacts to a Github Releaseā¢ļø
Even though cargo-dist is primarily a tool for building and packaging applications (steps 2-4), we put a fair amount of effort into Generating Your CI Scripts For You because we want to be able to run things locally and know what the CI should do without actually running it. It also helps avoid needless vendor lock-in -- in an ideal world, migrating from Github to Gitlab or your own personal infra would be just one invocation of cargo-dist away!
That said, the current version is Very Very Unstable And Experimental and the extra conveniences currently only work with Github CI and Github Releasesā¢ļø!
- Way-Too-Quick Start
- Installation
- Setup
- Usage (CI)
- Usage (Local Builds)
- Concepts
- Build Flags
- Compatibility With Other Tools
- Contributing
Way-Too-Quick Start
# install tools
cargo install cargo-dist
# one-time setup
cargo dist init --ci=github
git add .
git commit -am "wow shiny new cargo-dist CI!"
# cut a release like you normally would
# <manually update the version of your crate, run tests, etc>
# then:
git commit -am "chore: Release version 0.1.0"
git tag v0.1.0
cargo publish
git push
git push --tags
That's gonna do a whole bunch of stuff you might not have expected, but if it all works you'll get a Github Releaseā¢ļø with built and zipped artifacts uploaded to it! Read the rest of the docs to learn more!
You may have noticed "cut a release" still has a lot of tedious work. That's because we recommend using cargo-release to streamline the last step:
# install tools
cargo install cargo-dist
cargo install cargo-release
# one-time setup
cargo dist init --ci=github
git add .
git commit -am "wow shiny new cargo-dist CI!"
# cut a release
cargo release 0.1.0
(I left off the --execute flag from cargo-release
so you won't actually break anything if you really did just copy paste that š)
Installation
Install Prebuilt Binaries With cargo-binstall
cargo binstall cargo-dist --no-symlinks
(Without --no-symlinks
this may fail on Windows)
Build From Source With Cargo
cargo install cargo-dist --profile=dist
(--profile=dist
may get you a slightly more optimized binary.)
Download Prebuilt Binaries From Github Releases
Use The Installer Scripts
NOTE: these installer scripts are currently under-developed and will place binaries in $HOME/.cargo/bin/
without properly informing Cargo of the change, resulting in cargo uninstall cargo-dist
and some other things not working. They are however suitable for quickly bootstrapping cargo-dist in temporary environments (like CI) without any other binaries being installed.
Linux and macOS:
curl --proto '=https' --tlsv1.2 -L -sSf https://github.com/axodotdev/cargo-dist/releases/download/v0.0.1/installer.sh | sh
Windows PowerShell:
irm 'https://github.com/axodotdev/cargo-dist/releases/download/v0.0.1/installer.ps1' | iex
Setup
Once cargo-dist is installed, you can set it up in your cargo project by running
cargo dist init --ci=github
This will:
- Add a
dist
build profile to your Cargo.toml (with recommended default build flags) - Add a
.github/workflows/release.yml
file to your project (only if you pass--ci=...
)
These changes should be checked in to your repo for whenever you want to cut a release.
If you don't want ci scripting generated, but just want the dist
profile you can do:
cargo dist init
If you want to just (re)generate the ci scripts, you can do:
cargo dist generate-ci github
See the next section ("Usage (CI)") for how the github workflow is triggered and what it does.
Configuring Installers
If you would like to generate (still under development) installer scripts, you can pass --installer
flags
to either init
or generate-ci
:
cargo dist init --ci=github --installer=github-shell --installer=github-powershell
This will result in installer.sh
and installer.ps1
being generated which fetch from a Github Releaseā¢ļø and copy the binaries to $HOME/.cargo/bin/
on the assumption that this is on your PATH. The scripts are currently brittle and won't properly tell Cargo about the installation (making cargo uninstall
and some other commands behave incorrectly). As such they're currently only really appropriate for setting up temporary environments like CI without any other binaries. This will be improved in the future.
Configuring Targets
By default, init
and generate-ci
will assume you want to build for a "standard desktop suite of targets". This is currently:
- x86_64-pc-windows-msvc
- x86_64-unknown-linux-gnu
- x86_64-apple-darwin
(In The future arm64 counterparts and linux-musl will probably join this, but unfortunately we currently don't support cross-compilation.)
If you would like to manually specify the targets, you can do this with --target=...
which can be passed any number of times. If this flag is passed then the defaults will be cleared.
Other commands like cargo dist build
(bare cargo dist
) will always default to only using the current host target, and may need more manual target specification. This is handled automatically if you're using dist's generated CI scripts.
cargo-dist does not currently support specifying additional targets based on different --features
or anything else, this will change in the future. See issue #22 for discussion.
Usage (CI)
Once you've completed setup (run cargo dist init --ci=...
), you're ready to start cutting releases!
The github workflow will trigger whenever you push a git tag to the main branch of your repository that looks like a version number (v1
, v1.2.0
, v0.1.0-prerelease2
, etc.).
You might do that with something like this:
# <first manually update the version of your crate, run tests, etc>
# then:
git commit -am "chore: Release version 0.1.0"
git tag v0.1.0
cargo publish
git push
git push --tags
That's a bunch of junk to remember to do, so we recommend using cargo-release to streamline all of that:
cargo release 0.1.0
NOTE: this will do nothing unless you also pass
--execute
, this is omitted intentionally!
ALSO NOTE: if your application is part of a larger workspace, you may want to configure cargo-release with things like
shared-version
andtag-name
to get the desired result. In the future the CI scripts we generate may be smarter and able to detect things like "partial publishes of the workspace". For now we assume you're always publishing the entire workspace!
cargo-release will then automatically:
- Bump all your version numbers
- Make a git commit
- Make a git tag
- Publish to crates.io (disable this with
--no-publish
) - Push to your repo's main branch
When you do push a tag (and the commit it points to) the CI will take over and do the following:
- Create a draft Github Releaseā¢ļø (with taiki-e/create-github-release-action)
- Build your application for all the target platforms, wrap them in zips/tars, and upload them to the Github Releaseā¢ļø
- (Optional, see setup) Build installer scripts that fetch from the Github Releaseā¢ļø
- Generate a dist-manifest.json describing all the artifacts and upload it to the Github Releaseā¢ļø
- On success of all the previous tasks, mark the Github Releaseā¢ļø as a non-draft
The reason we do this extra dance with drafts is that we don't want to notify anyone of the release until it's Complete, but also don't want to lose anything if some only some of the build tasks failed.
taiki-e/create-github-release-action has some support for automatically parsing a CHANGELOG.md file to populate the text of the release. For now you will need to manually enable that by editing release.yml
.
Usage (Local Builds)
When you run bare
cargo dist
this is actually a synonym forcargo dist build
. For the sake of clarity these docs will prefer this longer form.
The happy path of cargo-dist is to just have its generated CI scripts handle all the details for you, so you never really need to run cargo dist build
if you're happy to leave it to the CI. But there's plenty of reasons to want to do a local build, or to just want to understand what the builds do, so here's the docs for that!
At a high level, cargo dist build
will:
- create a
target/distrib/
folder - run
cargo build --profile=dist
on your workspace - copy built-assets reported by
cargo
intotarget/distrib/
- copy static-assets like README.md
- bundle things up into zips/tarballs ("Artifacts")
- give you paths to all the final Artifacts for you to do whatever with
If you pass --output-format=json
it will also produce a machine-readable dist-manifest.json describing all of this.
If you pass --installer=...
it will also produce that installer artifact (see Configuring Installers).
If you pass --target=...
it will build for that target instead of the host one (see Configuring Targets).
If you pass --no-builds
you can make it skip cargo builds and just focus on generating artifacts that don't require a build (like install scripts).
If you run cargo dist manifest --output-format=json
it will skip generating artifacts and just produce dist-manifest.json
. Notably, if you pass every --installer
and --target
flag at once to this command you will get a unified manifest for everything you should produce on every platform. --no-local-paths
will strip away the actual paths pointing into target
, which would otherwise become giberish if the artifacts get moved to another system.
For further details, see Concepts and Build Flags.
Concepts
cargo-dist views the world as follows:
- You are trying to publish Applications (e.g. "ripgrep" or "cargo-binstall")
- An Application has Releases, which are a specific version of an Application (e.g. "ripgrep 1.0.0" or "cargo-binstall 0.1.0-prerelease")
- A Release has Artifacts that should be built and uploaded:
- platform-specific "executable-zips" (
ripgrep-1.0.0-x86_64-apple-darwin.tar.xz
) - "symbols" (debuginfo/sourcemaps) for those executables (
ripgrep-1.0.0-x86_64-pc-windows-msvc.pdb
) - installers (
installer.sh
,installer.ps1
) - machine-readable manifests (
dist-manifest.json
)
- platform-specific "executable-zips" (
- Artifacts may have Assets stored inside them:
- built-assets (
ripgrep.exe
,ripgrep.pdb
) - static-assets (
README.md
,LICENSE.md
)
- built-assets (
- Artifacts may also have a list of Targets (triples) that they are intended for (multi-arch binaries/installers are possible)
We'll eventually make this more properly configurable, but currently cargo-dist computes this from a combination of CLI flags and your Cargo workspace:
- Every binary target your workspace can build is given its own Application. Properties like "github repository", "README", "version", and so on are inherited from the parent Cargo package (Cargo packages can have multiple binaries they produce, whether that's a good idea is up to you).
- Each Application will get one Release -- for its current version
- Each Release will get the following artifacts:
- An executable-zip for each target platform (see Configuring Targets)
- Symbols for each target platform (if the platform supports it, currently only
windows-msvc => pdb
is enabled) - Installers if requested (see Configuring Installers)
- dist-manifest.json describing all of this (emitted on stdout if
--output-format=json
is passed)
- Each executable-zip will automatically include local files with special names like README.md (eventually this will be configurable...)
In the future we might support things like "hey this application actually wants to bundle up several binaries" or "ignore this binary". Similarly we might allow you to specify that multiple versions of an application should be published with different feature flags. This is all up in the air for now, we're just trying to get the simple happy path working right now.
A current key property of cargo-dist's design is that it can compute all of these facts on any host platform before running any builds. cargo dist manifest --output-format=json
does exactly this.
(Applications only really exist implicitly -- in practice cargo-dist on really ever talks about Releases, since that's just An Application With A Version, and we always have some version.)
Build Flags
cargo-dist changes a bunch of the default build flags you would get with cargo build --release
, so here's what we change and why!
Most of the settings we change are baked into your Cargo.toml when you run cargo dist init
in the form of a dist
profile. This lets you see them and change them if you disagree with them! Here's the current default:
[profile.dist]
inherits = "release"
debug = true
split-debuginfo = "packed"
inherits = "release"
-- release generally has the right idea, so we start with its flags!debug = true
-- enables full debuginfo, which release builds normally disable (because it would bloat the binary)split-debuginfo = "packed"
-- tells the compiler to rip all of the debuginfo it can out of the final binary and put it into a single debuginfo file (aka "symbols", aka "sourcemap")
We also secretly modify RUSTFLAGS as follows (unfortunately not yet configurable):
- on
*-windows-msvc
targets we append-Ctarget-feature=+crt-static"
to RUSTFLAGS. Unlike other platforms, Microsoft doesn't consider libc ("crt", the C RunTime) to be a fundamental part of the platform. There are more fundamental DLLs on the OS that libc is implemented on top of. As such, libc isn't actually guaranteed to exist on the system, and Microsoft actually wants you to statically link it! (Or have an installer wizard which downloads the version you need, which you may have seen a game do for C++ when it says "Installing Visual C++ Redistributable".) Really Rust should have defaulted to this setting but Mistakes Happen so we're fixing it for you. See The RFC for more details.
In the future we'll probably also turn on these settings:
profile.dist.lto="fat"
-- further optimize the binary in a way that's only practical for shippable releasesRUSTFLAGS="-Csymbol-mangling-version=v0"
-- use the Fancier symbol mangling that preserves more info for debuggersRUSTFLAGS="-Cforce-frame-pointers=yes"
-- enable frame pointers, making debuggers and profilers more reliable and effecient in exchange for usually-negligible perf lossesRUSTFLAGS="--remap-path-prefix=..."
-- try to strip local paths from the debuginfo/binary
In a similar vein to the crt-static
change, we may also one day prefer linux-musl
over linux-gnu
to produce more portable binaries. Currently the only mechanism we have to do this is "try to run builds on Github's older linux images so the minimum glibc version isn't too high". This is a place where we lack expertese and welcome recommendations! (This is blocked on supporting cross-compilation.)
Compatibility With Other Tools
cargo-dist can used totally standalone (well, you need Cargo), but is intended to be a cog in various machines. Here's some things that work well with it:
- CI Scripts should be automatically triggered by simple uses of cargo-release
- If you set
repository
in your Cargo.toml, then cargo-binstall should automagically find, download, and install binaries from the Github Releasesā¢ļø we produce without any further configuration - FUTURE AXODOTDEV TOOL will be able to consume dist-manifest.json and DO COOL THINGS
Contributing
Updating Snapshots
cargo-dist's tests rely on cargo-insta for snapshot testing various
outputs. This allows us to both catch regressions and also more easily review UI/output changes. If a snapshot
test fails, you will need to use the cargo insta
CLI tool to update them:
cargo install cargo-insta
One installed, you can review and accept the changes with:
cargo insta review
If you know you like the changes, just use cargo insta accept
to auto-apply all changes.
(If you introduced brand-new snapshot tests you will also have to git add
them!)
NOTE: when it succeeds, cargo-dist-schema's
emit
test will actually commit the results back to disk tocargo-dist-schema/cargo-dist-schema.json
as a side-effect. This is a janky hack to make sure we have that stored and up to date at all times (the test also uses an insta snapshot but insta snapshots include an extra gunk header so it's not something we'd want to link end users). The file isn't even used for anything yet, I just want it to Exist because it seems useful and important. In the future we might properly host it and have our outputs link it via a$schema
field.
Cutting Releases
cargo-dist is self-hosting, so just follow the usual usage instructions and publish with cargo release
!
The CI is (re)generated with:
cargo dist generate-ci github --installer=github-shell --installer=github-powershell
Including the installers is very important, as all the CI scripts cargo-dist generates for other projects will bootstrap dist with those installers.
Note that as a consequence of the way we self-host, cargo-dist's published artifacts will always be built/generated by a previous version of itself. This can be problematic if you make breaking changes to cargo-dist-schema's format... so don't! Many things in the schema are intentionally optional to enable forward and backward compatibility, so this should hopefully work well!
(Future work: mark cargo release
do more magic like cutting CHANGELOG.md and whatnot.)